Underwater image enhancement method based on multi-flow feature fusion

Abstract

The application provides an underwater image enhancement method based on multi-flow feature fusion, comprising the following steps: constructing a data set, wherein the data set comprises Q image pairs, and each image pair comprises a degraded underwater image and a corresponding reference image; dividing a training set from the data set; constructing an underwater image enhancement network model; training the underwater image enhancement network model by using the training set to obtain a trained underwater image enhancement network model; inputting a to-be-processed degraded underwater image into the trained underwater image enhancement network model; and outputting an enhanced image by the trained underwater image enhancement network model, so that the position-sensitive image features can be fully extracted while the original image details and textures are properly maintained, the key features are selectively highlighted, the color correction, contrast enhancement and deblurring are realized, and the high-quality enhanced image close to the reference image is generated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to an underwater image enhancement method based on multi-stream feature fusion. Background Technology

[0002] The unknown ocean world and abundant marine resources have become a hot topic of research for scholars, and images and videos are one of the important means of acquiring marine information. However, due to the complex underwater environment and lighting conditions, the collected underwater images are severely degraded after being affected by wavelength-related absorption and scattering, specifically exhibiting problems such as severe color cast, low contrast, insufficient brightness, and poor clarity. Degraded underwater images are difficult to directly apply to marine exploration and research, which hinders the further development of marine-related tasks. Therefore, inventing a reasonable underwater image processing technology to effectively enhance degraded underwater images has significant practical application value.

[0003] In recent years, various underwater image processing methods have been proposed and applied, achieving certain results, but many problems and significant room for improvement remain. Existing underwater image processing methods are mainly divided into traditional underwater image enhancement methods and deep learning-based underwater image enhancement methods. Traditional underwater image enhancement methods can be further divided into non-physical model methods and physical model-based methods.

[0004] Non-physical model-based methods improve image contrast and correct color casts by directly adjusting the pixel distribution of degraded images, such as histogram equalization, dynamic pixel range stretching, and image fusion. These methods can often quickly improve the quality of degraded underwater images, but they rely heavily on manually designed criteria, making them prone to over-enhancement or under-enhancement, thus affecting the overall visual effect.

[0005] Physical model-based methods utilize prior knowledge and assumptions to simulate the underwater imaging process. These methods treat underwater image degradation as a reversible process, solving for the high-quality image before degradation by inverting the underwater imaging model. Prior knowledge such as dark channel priors and minimum loss priors, as well as imaging models such as atmospheric scattering models and Retinex models, are widely used in this field. Physical model-based methods can effectively enhance images in specific scenes; however, the assumptions and models proposed by these methods have low robustness, and the models contain many parameters that are difficult to estimate accurately. This results in limited generalization ability, poor flexibility, and difficulty in adapting to complex and changing underwater scenes.

[0006] On the other hand, with the improvement of computing performance and the abundance of data samples, deep learning-based underwater image enhancement methods have attracted increasing attention from professionals. Under the supervision of a large number of data samples and various loss functions, deep learning-based underwater image enhancement methods can correctly establish the mapping between low-quality degraded images and high-quality images, thus significantly improving the quality of underwater images. Deep learning-based underwater image enhancement methods mainly include methods based on convolutional neural networks (CNNs) and methods based on generative adversarial networks (GANs). Both types of methods have achieved considerable performance in the field of underwater image enhancement. Most deep learning-based methods extract features from underwater images through convolution operations, change the size of the feature map through downsampling operations to extract high-level features under different receptive fields, and finally obtain the enhanced image through upsampling or deconvolution operations. During the process of changing the feature map size through downsampling and upsampling, deep learning-based methods inevitably destroy the connections between image pixels, resulting in the loss of original image details and leading to unsatisfactory restored underwater images.

[0007] To address the problems of the aforementioned methods, and in order to reasonably restore degraded image details, correct the color of underwater images, and enhance the contrast of underwater images, this invention provides an underwater image enhancement method based on multi-flow feature fusion. Summary of the Invention

[0008] To address the shortcomings mentioned above, this invention provides an underwater image enhancement method based on multi-stream feature fusion. The invention includes: constructing an underwater image enhancement network model comprising three feature processing streams, each processing feature maps at different resolutions. Each feature processing stream consists of a residual coordinate attention module and an adaptive feature fusion module. The residual coordinate attention module extracts image features from different locations, while the adaptive feature fusion module fuses complementary image information extracted by different processing streams. This process corrects color cast in the underwater image while preserving its original detail, thereby improving the image's contrast.

[0009] The technical means employed in this invention are as follows:

[0010] This invention provides an underwater image enhancement method based on multi-flow feature fusion, comprising:

[0011] Construct a dataset comprising Q image pairs, each image pair including a degraded underwater image and its corresponding reference image;

[0012] The dataset is divided into a training set and a test set;

[0013] Construct an underwater image enhancement network model;

[0014] The underwater image enhancement network model is trained using the training set to obtain the trained underwater image enhancement network model.

[0015] Select one of the degraded underwater images from the test set and input it into the trained underwater image enhancement network model. The trained underwater image enhancement network model outputs an enhanced image.

[0016] Furthermore, constructing the underwater image enhancement network model includes:

[0017] First convolutional module;

[0018] The input of the first feature enhancement module is connected to the output of the first convolution module, and the output of the first feature enhancement module is connected to the input of the first element-wise addition module.

[0019] The input of the first element-wise addition module is connected to the output of the first feature enhancement module and the output of the first convolution module, respectively, and the output of the first element-wise addition module is connected to the input of the second feature enhancement module.

[0020] The input of the second feature enhancement module is connected to the output of the first element-by-element addition module, and the output of the second feature enhancement module is connected to the input of the second element-by-element addition module.

[0021] The input of the second element-wise addition module is connected to the output of the second feature enhancement module and the output of the first element-wise addition module, respectively, and the output of the second element-wise addition module is connected to the input of the third feature enhancement module;

[0022] The input of the third feature enhancement module is connected to the output of the second element-by-element addition module, and the output of the third feature enhancement module is connected to the input of the third element-by-element addition module.

[0023] The input of the third element-wise addition module is connected to the output of the third feature enhancement module and the output of the second element-wise addition module, respectively, and the output of the third element-wise addition module is connected to the input of the second convolution module.

[0024] The input of the second convolution module is connected to the third element-wise addition module, and the output of the second convolution module is connected to the input of the fourth element-wise addition module.

[0025] The input of the fourth element-wise addition module is connected to the output of the second convolution module and the input of the first convolution module, respectively.

[0026] Furthermore, the first feature enhancement module includes a first multi-stream feature estimation module, a second multi-stream feature estimation module, and a third convolution module connected in sequence;

[0027] The first multi-stream feature estimation module includes: a first feature processing stream, a second feature processing stream, and a third feature processing stream; the first feature processing stream takes the feature map at the original resolution as input; the second feature processing stream takes the feature map downsampled by a factor of two as input; and the third feature processing stream takes the feature map downsampled by a factor of four as input.

[0028] The first feature processing stream includes:

[0029] The input to the first residual coordinate attention module is the feature map at the original resolution, and the output of the first residual coordinate attention module is connected to the input of the first adaptive feature fusion module.

[0030] The input of the first adaptive feature fusion module is connected to the output of the first residual coordinate attention module and the output of the third adaptive feature fusion module, respectively, and the output of the first adaptive feature fusion module is connected to the input of the second residual coordinate attention module.

[0031] The input of the second residual coordinate attention module is connected to the output of the first adaptive feature fusion module, and the output of the second residual coordinate attention module is connected to the input of the second adaptive feature fusion module.

[0032] The input of the second adaptive feature fusion module is connected to the output of the second residual coordinate attention module and the output of the fourth adaptive feature fusion module, respectively; the output of the second adaptive feature fusion module is connected to the input of the fourth convolution module.

[0033] The input of the fourth convolutional module is connected to the output of the second adaptive feature fusion module, and the output of the fourth convolutional module is connected to the input of the fifth element-wise addition module.

[0034] The input of the fifth element-wise addition module is connected to the output of the fourth convolution module and the feature map at the original resolution, respectively.

[0035] The second feature processing stream includes:

[0036] The input to the third residual coordinate attention module is the feature map that has been downsampled by two times, and the output of the third residual coordinate attention module is connected to the input of the third adaptive feature fusion module.

[0037] The input of the third adaptive feature fusion module is connected to the output of the third residual coordinate attention module and the output of the fifth residual coordinate attention module, respectively. The output of the third adaptive feature fusion module is connected to the input of the fourth residual coordinate attention module and the input of the first adaptive feature fusion module, respectively.

[0038] The input of the fourth residual coordinate attention module is connected to the output of the third adaptive feature fusion module, and the output of the fourth residual coordinate attention module is connected to the input of the fourth adaptive feature fusion module.

[0039] The input of the fourth adaptive feature fusion module is connected to the output of the fourth residual coordinate attention module and the output of the sixth residual coordinate attention module, respectively, and the output of the fourth adaptive feature fusion module is connected to the input of the second adaptive feature fusion module.

[0040] The third feature processing stream includes:

[0041] The input of the fifth residual coordinate attention module is the feature map downsampled by four times, and the output of the fifth residual coordinate attention module is connected to the input of the third adaptive feature fusion module and the input of the sixth residual coordinate attention module, respectively.

[0042] The input of the sixth residual coordinate attention module is connected to the output of the fifth residual coordinate attention module, and the output of the sixth residual coordinate attention module is connected to the input of the fourth adaptive feature fusion module. Further, the first residual coordinate attention module includes:

[0043] Fifth convolution module;

[0044] The input of the first LeakyReLU activation function calculation module is connected to the output of the fifth convolution module, and the output of the first LeakyReLU activation function calculation module is connected to the input of the sixth convolution module.

[0045] The input of the sixth convolution module is connected to the output of the first LeakyReLU activation function calculation module, and the output of the sixth convolution module is connected to the input of the first element-wise multiplication module and the input of the coordinate attention mechanism, respectively.

[0046] The input of the coordinate attention mechanism is connected to the output of the sixth convolution module, and the output of the coordinate attention mechanism is connected to the input of the first element-wise multiplication module.

[0047] The input of the first element-wise multiplication module is connected to the output of the sixth convolution module and the output of the coordinate attention mechanism, respectively; the output of the first element-wise multiplication module is connected to the input of the second LeakyReLU activation function calculation module.

[0048] The input of the second LeakyReLU activation function calculation module is connected to the output of the first element-wise multiplication module, and the output of the second LeakyReLU activation function calculation module is connected to the input of the sixth element-wise addition module;

[0049] The input of the sixth element-wise addition module is connected to the output of the second LeakyReLU activation function calculation module and the input of the fifth convolution module, respectively.

[0050] Furthermore, the coordinate attention mechanism includes:

[0051] The input of the horizontal average pooling module is connected to the output of the sixth convolution module, and the output of the horizontal average pooling module is connected to the input of the first element-wise concatenation module.

[0052] The input of the vertical average pooling module is connected to the output of the sixth convolution module, and the output of the vertical average pooling module is connected to the input of the first element-wise concatenation module.

[0053] The input of the first element-wise concatenation module is connected to the output of the horizontal average pooling module and the output of the vertical average pooling module, respectively, and the output of the first element-wise concatenation module is connected to the input of the seventh convolution module.

[0054] The input of the seventh convolution module is connected to the output of the first element-wise concatenation module, and the output of the seventh convolution module is connected to the input of the third LeakyReLU activation function calculation module.

[0055] The input of the third LeakyReLU activation function calculation module is connected to the output of the seventh convolution module, and the output of the third LeakyReLU activation function calculation module is connected to the input of the eighth convolution module and the input of the ninth convolution module, respectively.

[0056] The input of the eighth convolution module is connected to the output of the third LeakyReLU activation function calculation module, and the output of the eighth convolution module is connected to the input of the first Sigmoid activation function calculation module.

[0057] The input of the first Sigmoid activation function calculation module is connected to the output of the eighth convolution module, and the output of the first Sigmoid activation function calculation module is connected to the input of the first element-wise multiplication module.

[0058] The input of the ninth convolution module is connected to the output of the third LeakyReLU activation function calculation module, and the output of the ninth convolution module is connected to the input of the second Sigmoid activation function calculation module.

[0059] The input of the second Sigmoid activation function calculation module is connected to the output of the ninth convolution module, and the output of the second Sigmoid activation function calculation module is connected to the input of the first element-wise multiplication module.

[0060] Furthermore, the first adaptive feature fusion module includes:

[0061] The input to the seventh element-wise addition module is the first feature map and the second feature map, and the output of the seventh element-wise addition module is connected to the input of the global average pooling module and the input of the global max pooling module, respectively.

[0062] The global average pooling module, the tenth convolution module, the first GELU activation function calculation module, and the eleventh convolution module are connected in sequence. The input of the global average pooling module is connected to the output of the seventh element-wise addition module, and the output of the eleventh convolution module is connected to the input of the second element-wise concatenation module.

[0063] The global max pooling module, the twelfth convolution module, the second GELU activation function calculation module, and the thirteenth convolution module are connected in sequence. The input of the global max pooling module is connected to the output of the seventh element-wise addition module, and the output of the thirteenth convolution module is connected to the input of the second element-wise concatenation module.

[0064] The second element-wise concatenation module, the fourteenth convolution module, the fifteenth convolution module, and the third GELU activation function calculation module are connected sequentially. The input of the second element-wise concatenation module is connected to the output of the eleventh convolution module and the output of the thirteenth convolution module, respectively. The output of the third GELU activation function calculation module is connected to the input of the sixteenth convolution module and the input of the seventeenth convolution module, respectively.

[0065] The input of the SoftMax module is connected to the output of the sixteenth convolution module and the output of the seventeenth convolution module, respectively. The output of the SoftMax module is connected to the input of the second element-wise multiplication module and the input of the third element-wise multiplication module, respectively.

[0066] The input of the second element-wise multiplication module is connected to the first feature map and the output of the sixteenth convolution module, respectively, and the output of the second element-wise multiplication module is connected to the input of the eighth element-wise addition module.

[0067] The input of the third element-wise multiplication module is connected to the output of the second feature map and the seventeenth convolution module, respectively, and the output of the second element-wise multiplication module is connected to the input of the eighth element-wise addition module.

[0068] The input of the eighth element-wise addition module is connected to the output of the sixteenth convolution module and the output of the seventeenth convolution module, respectively.

[0069] Furthermore, the loss function used in training the underwater image enhancement network model is calculated as follows:

[0070] L MSFFNet =λ1L char (G(X), Y)+λ2L ms (G(X), Y)+λ3L per (G(X), Y);

[0071] Among them, L char (G(X), Y) represents the Charbonnier loss, L ms (G(X), Y) represents the multi-scale structural similarity loss, L per (G(X), Y) represents the perceptual loss, λ1 is the first weight, λ2 is the second weight, and λ3 is the third weight.

[0072] Furthermore, the Charbonnier loss is calculated as follows:

[0073]

[0074] Wherein, G(X) is the enhanced image, Y is the reference image, and ε is the regularization term for mitigating non-differentiable points.

[0075] Furthermore, the multi-scale structural similarity loss is calculated as follows:

[0076]

[0077] Where M represents different scales, μ G μ is the mean of the enhanced image. Y σ is the mean of the reference image. G σ is the standard deviation of the enhanced image. Y σ is the standard deviation of the reference image. GY β is the covariance between the enhanced image and the reference image. m As the first relative importance, γ m For the second relative importance, C1 is the first constant, C1≠0, and C2 is the second constant, C2≠0.

[0078] Furthermore, the perceived loss is calculated as follows:

[0079]

[0080] Where N is the number of feature maps extracted, Φj This represents the feature map calculated using the j-th convolution of the VGG-16 network.

[0081] Compared with the prior art, the present invention has the following advantages:

[0082] The underwater image enhancement method based on multi-stream feature fusion provided by this invention constructs a dataset comprising Q image pairs, each pair including a degraded underwater image and its corresponding reference image; a training set is partitioned from the dataset; an underwater image enhancement network model is constructed; the underwater image enhancement network model is trained using the training set to obtain the trained underwater image enhancement network model; the degraded underwater image to be processed is input into the trained underwater image enhancement network model, and the trained underwater image enhancement network model outputs the enhanced image. This method achieves the ability to fully extract position-sensitive image features while appropriately preserving the original image details and textures, selectively highlighting key features, and realizing color correction, contrast enhancement, and deblurring to generate a high-quality enhanced image close to the reference image. Attached Figure Description

[0083] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0084] Figure 1 This is a flowchart illustrating an underwater image enhancement method based on multi-flow feature fusion provided by the present invention.

[0085] Figure 2 This is a schematic diagram of a network model for underwater image enhancement.

[0086] Figure 3 This is a schematic diagram of the structure of the attention module for the first residual coordinate.

[0087] Figure 4 This is a schematic diagram of the structure of the first adaptive feature fusion module.

[0088] Figure 5 This is a subjective evaluation comparison chart of the underwater image enhancement method based on multi-flow feature fusion and the comparison method provided by the present invention on a test set.

[0089] Figure 6 This is another subjective evaluation comparison chart of the underwater image enhancement method based on multi-flow feature fusion provided by the present invention and the comparison method on the test set.

[0090] In the diagram: 1. First convolutional module; 2. First feature enhancement module; 3. First element-wise addition module; 4. Second feature enhancement module; 5. Second element-wise addition module; 6. Third feature enhancement module; 7. Third element-wise addition module; 8. Second convolutional module; 9. Fourth element-wise addition module; 10. First multi-stream feature estimation module; 11. Second multi-stream feature estimation module; 12. Third convolutional module; 13. Eighth element-wise addition module; 14. First feature processing stream; 15. Second feature processing stream; 16. Third feature processing stream; 17. First residual coordinate attention module; 18. 19. Adaptive Feature Fusion Module; 20. Second Residual Coordinate Attention Module; 21. Second Adaptive Feature Fusion Module; 22. Fourth Convolution Module; 23. Fifth Element-by-Element Addition Module; 24. Third Residual Coordinate Attention Module; 25. Third Adaptive Feature Fusion Module; 26. Fourth Adaptive Feature Fusion Module; 27. Fifth Residual Coordinate Attention Module; 28. Sixth Residual Coordinate Attention Module; 29. ​​Fifth Convolution Module; 30. First Leaky ReLU Activation Function Calculation Module; 31. Sixth Convolution Module; 32. Coordinate Attention Machine Modules: 33. First element-wise multiplication module; 34. Second LeakyReLU activation function calculation module; 35. Sixth element-wise addition module; 36. Horizontal average pooling module; 37. Vertical average pooling module; 38. First element-wise concatenation module; 39. Seventh convolution module; 40. Third LeakyReLU activation function calculation module; 41. Eighth convolution module; 42. Ninth convolution module; 43. First Sigmoid activation function calculation module; 44. Second Sigmoid activation function calculation module; 45. Seventh element-wise addition module; 46. Global average pooling module. 47. Tenth Convolution Module; 48. First GELU Activation Function Calculation Module; 49. Eleventh Convolution Module; 50. Global Max Pooling Module; 51. Twelfth Convolution Module; 52. Second GELU Activation Function Calculation Module; 53. Thirteenth Convolution Module; 54. Second Element-by-Element Concatenation Module; 55. Fourteenth Convolution Module; 56. Fifteenth Convolution Module; 57. Third GELU Activation Function Calculation Module; 58. Sixteenth Convolution Module; 59. Seventeenth Convolution Module; 60. SoftMax Module; 61. Second Element-by-Element Multiplication Module; 62. Third Element-by-Element Multiplication Module. Detailed Implementation

[0091] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0092] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0093] Combination Figure 1 , Figure 1 This is a flowchart illustrating a method for underwater image enhancement based on multi-flow feature fusion provided by the present invention, used to illustrate a specific embodiment of the method, characterized by including:

[0094] S1: Construct a dataset consisting of Q image pairs, each image pair consisting of a degraded underwater image and its corresponding reference image;

[0095] S2: Divide the dataset into training and test sets;

[0096] S3: Construct an underwater image enhancement network model;

[0097] S4: The underwater image enhancement network model is trained using the training set to obtain the trained underwater image enhancement network model;

[0098] S5: Select any degraded underwater image from the test set and input it into the trained underwater image enhancement network model. The trained underwater image enhancement network model outputs the enhanced image.

[0099] It should be noted that in this embodiment, the datasets used are the UIEB underwater dataset and the EUVP underwater dataset. The datasets are divided into training and testing sets. The training set includes 800 image pairs randomly selected from the UIEB underwater dataset and 1500 image pairs randomly selected from the EUVP underwater dataset. The testing set includes the remaining 90 image pairs from the UIEB underwater dataset and the remaining 500 image pairs from the EUVP underwater dataset. There are no duplicate images between the training and testing sets. Of course, this is not a limitation, and the construction of the datasets can be set according to actual needs. This embodiment does not impose specific restrictions on this.

[0100] Understandably, the underwater image enhancement network model was built using the PyTorch deep learning framework; it was trained on a Linux workstation using a GTX 3080 graphics processing unit; the AdamW algorithm was used to optimize the training process; and the initial learning rate was 2×10⁻⁶. -4 The minimum learning rate is 1×10. -6 The input degraded underwater image is 256×256×3 pixels, with pixel values ​​normalized to [0, 1]. The batch size is 1, and the training iterations are 300. The underwater image enhancement network model is trained on the training set to obtain the fitted weight parameters. A degraded underwater image is randomly selected from the test set and input into the trained underwater image enhancement network model. The trained model outputs an enhanced image. Through this training, the color cast of the degraded underwater image can be effectively corrected, while restoring image details and generating a high-quality, clear image with rich colors.

[0101] In some alternative embodiments, refer to Figure 2 , Figure 2 This is a schematic diagram of an underwater image enhancement network model. Constructing the underwater image enhancement network model includes:

[0102] First convolutional module 1;

[0103] The input of the first feature enhancement module 2 is connected to the output of the first convolution module 1, and the output of the first feature enhancement module 2 is connected to the input of the first element-wise addition module 3.

[0104] The input of the first element-wise addition module 3 is connected to the output of the first feature enhancement module 2 and the output of the first convolution module 1, respectively, and the output of the first element-wise addition module 3 is connected to the input of the second feature enhancement module 4.

[0105] The input of the second feature enhancement module 4 is connected to the output of the first element-by-element addition module 3, and the output of the second feature enhancement module 4 is connected to the input of the second element-by-element addition module 5.

[0106] The input of the second element-wise addition module 5 is connected to the output of the second feature enhancement module 4 and the output of the first element-wise addition module 3, respectively. The output of the second element-wise addition module 5 is connected to the input of the third feature enhancement module 6.

[0107] The input of the third feature enhancement module 6 is connected to the output of the second element-by-element addition module 5, and the output of the third feature enhancement module 6 is connected to the input of the third element-by-element addition module 7.

[0108] The input of the third element-wise addition module 7 is connected to the output of the third feature enhancement module 6 and the output of the second element-wise addition module 5, respectively. The output of the third element-wise addition module 7 is connected to the input of the second convolution module 8.

[0109] The input of the second convolution module 8 is connected to the third element-wise addition module 7, and the output of the second convolution module 8 is connected to the input of the fourth element-wise addition module 9.

[0110] The input of the fourth element-wise addition module 9 is connected to the output of the second convolution module 8 and the input of the first convolution module 1, respectively.

[0111] It is understandable that the first feature enhancement module 2, the second feature enhancement module 4, and the third feature enhancement module 6 have the same structure. The first convolution module 1 and the second convolution module 8 are both 3×3 convolutions. Skip connections are used between the input and output of each feature enhancement module to facilitate information transfer. Finally, the image residual information processed by the three feature enhancement modules is summed with the input original degraded underwater image to achieve underwater image enhancement. The principle of the underwater image enhancement method based on multi-stream feature fusion is as follows:

[0112] I Enhanced =I Raw +R;

[0113] Among them, I Enhanced To enhance the image, I Raw For degraded underwater images, R image residual information.

[0114] In some alternative embodiments, reference continues to be made to... Figure 2 The first feature enhancement module 2 includes a first multi-stream feature estimation module 10, a second multi-stream feature estimation module 11, and a third convolution module 12 connected in sequence.

[0115] The first multi-stream feature estimation module 10 includes: a first feature processing stream 14, a second feature processing stream 15, and a third feature processing stream 16; the first feature processing stream takes the feature map at the original resolution as input; the second feature processing stream takes the feature map downsampled by a factor of two as input; and the third feature processing stream takes the feature map downsampled by a factor of four as input.

[0116] The first feature processing stream 14 includes:

[0117] The input to the first residual coordinate attention module 17 is the feature map at the original resolution, and the output of the first residual coordinate attention module 17 is connected to the input of the first adaptive feature fusion module 18.

[0118] The input of the first adaptive feature fusion module 18 is connected to the output of the first residual coordinate attention module 17 and the output of the third adaptive feature fusion module 24, respectively, and the output of the first adaptive feature fusion module 18 is connected to the input of the second residual coordinate attention module 19.

[0119] The input of the second residual coordinate attention module 19 is connected to the output of the first adaptive feature fusion module 18, and the output of the second residual coordinate attention module 19 is connected to the input of the second adaptive feature fusion module 20.

[0120] The input of the second adaptive feature fusion module 20 is connected to the output of the second residual coordinate attention and the output of the fourth adaptive feature fusion module 26, respectively. The output of the second adaptive feature fusion module 20 is connected to the input of the fourth convolution module 21.

[0121] The input of the fourth convolution module 21 is connected to the output of the second adaptive feature fusion module 20, and the output of the fourth convolution module 21 is connected to the input of the fifth element-wise addition module 22.

[0122] The inputs of the fifth element-wise addition module 22 are connected to the feature maps of the original resolution output of the fourth convolution module 21.

[0123] The second feature processing stream 15 includes:

[0124] The input to the third residual coordinate attention module 23 is a feature map downsampled by two times, and the output of the third residual coordinate attention module 23 is connected to the input of the third adaptive feature fusion module 24.

[0125] The input of the third adaptive feature fusion module 24 is connected to the output of the third residual coordinate attention module 23 and the output of the fifth residual coordinate attention module 27, respectively. The output of the third adaptive feature fusion module 24 is connected to the input of the fourth residual coordinate attention module 25 and the input of the first adaptive feature fusion module 18, respectively.

[0126] The input of the fourth residual coordinate attention module 25 is connected to the output of the third adaptive feature fusion module 24, and the output of the fourth residual coordinate attention module 25 is connected to the input of the fourth adaptive feature fusion module 26.

[0127] The input of the fourth adaptive feature fusion module 26 is connected to the output of the fourth residual coordinate attention module 25 and the output of the sixth residual coordinate attention module 28, respectively, and the output of the fourth adaptive feature fusion module 26 is connected to the input of the second adaptive feature fusion module 20.

[0128] The third feature processing stream 16 includes:

[0129] The input to the fifth residual coordinate attention module 27 is a feature map downsampled by four times, and the output of the fifth residual coordinate attention module 27 is connected to the input of the third adaptive feature fusion module 24 and the input of the sixth residual coordinate attention module 28, respectively.

[0130] The input of the sixth residual coordinate attention module 28 is connected to the output of the fifth residual coordinate attention module 27, and the output of the sixth residual coordinate attention module 28 is connected to the input of the fourth adaptive feature fusion module 26.

[0131] It is understandable that the first multi-stream feature estimation module 10 and the second multi-stream feature estimation module 11 have the same structure, and the third convolution module 12 is a 3×3 convolution. The first residual coordinate attention module 17, the second residual coordinate attention module 19, the third residual coordinate attention module 23, the fourth residual coordinate attention module 25, the fifth residual coordinate attention module 27, and the sixth residual coordinate attention module 28 have the same structure. The first adaptive feature fusion module 18, the second adaptive feature fusion module 20, the third adaptive feature fusion module 24, and the fourth adaptive feature fusion module 26 have the same structure, and the fourth convolution module 21 is a 3×3 convolution.

[0132] The multi-stream feature estimation module employs a parallel three-processing-stream architecture to process feature maps at the original resolution, downsampled by two times, and downsampled by four times, respectively. Each feature processing stream fully extracts position-sensitive image features through two residual coordinate attention modules. Then, an adaptive feature fusion module adaptively fuses features from adjacent branches, highlighting key image features and suppressing noise. During the fusion process, some features require upsampling. Finally, the multi-scale feature information extracted by the two multi-stream feature estimation modules is added to the input features through a 3×3 convolution to obtain the output features.

[0133] In some alternative embodiments, refer to Figure 2 and Figure 3 , Figure 3 This is a schematic diagram of a structure of the first residual coordinate attention module. The first residual coordinate attention module 17 includes:

[0134] Fifth convolution module 29;

[0135] The input of the first LeakyReLU activation function calculation module 30 is connected to the output of the fifth convolution module 29, and the output of the first LeakyReLU activation function calculation module 30 is connected to the input of the sixth convolution module 31.

[0136] The input of the sixth convolution module 31 is connected to the output of the first LeakyReLU activation function calculation module 30, and the output of the sixth convolution module 31 is connected to the input of the first element-wise multiplication module 33 and the input of the coordinate attention mechanism 32, respectively.

[0137] The input of coordinate attention mechanism 32 is connected to the output of the sixth convolution module 31, and the output of coordinate attention mechanism 32 is connected to the input of the first element-wise multiplication module 33.

[0138] The input of the first element-wise multiplication module 33 is connected to the output of the sixth convolution module 31 and the output of the coordinate attention mechanism 32, respectively. The output of the first element-wise multiplication module 33 is connected to the input of the second LeakyReLU activation function calculation module 34.

[0139] The input of the second LeakyReLU activation function calculation module 34 is connected to the output of the first element-wise multiplication module 33, and the output of the second LeakyReLU activation function calculation module 34 is connected to the input of the sixth element-wise addition module 35.

[0140] The input of the sixth element-wise addition module 35 is connected to the output of the second LeakyReLU activation function calculation module 34 and the input of the fifth convolution module 29, respectively.

[0141] The coordinate attention mechanism 32 includes:

[0142] The input of the horizontal average pooling module 36 is connected to the output of the sixth convolution module 31, and the output of the horizontal average pooling module 36 is connected to the input of the first element-wise concatenation module 38.

[0143] The input of the vertical average pooling module 37 is connected to the output of the sixth convolution module 31, and the output of the vertical average pooling module 37 is connected to the input of the first element-wise concatenation module 38.

[0144] The input of the first element-wise concatenation module 38 is connected to the output of the horizontal average pooling module 36 and the output of the vertical average pooling module 37, respectively, and the output of the first element-wise concatenation module 38 is connected to the input of the seventh convolution module 39.

[0145] The input of the seventh convolution module 39 is connected to the output of the first element-wise concatenation module 38, and the output of the seventh convolution module 39 is connected to the input of the third LeakyReLU activation function calculation module 40.

[0146] The input of the third LeakyReLU activation function calculation module 40 is connected to the output of the seventh convolution module 39, and the output of the third LeakyReLU activation function calculation module 40 is connected to the input of the eighth convolution module 41 and the input of the ninth convolution module 42, respectively.

[0147] The input of the eighth convolution module 41 is connected to the output of the third LeakyReLU activation function calculation module 40, and the output of the eighth convolution module 41 is connected to the input of the first Sigmoid activation function calculation module 43.

[0148] The input of the first Sigmoid activation function calculation module 43 is connected to the output of the eighth convolution module 41, and the output of the first Sigmoid activation function calculation module 43 is connected to the input of the first element-wise multiplication module 33.

[0149] The input of the ninth convolution module 42 is connected to the output of the third LeakyReLU activation function calculation module 40, and the output of the ninth convolution module 42 is connected to the input of the second Sigmoid activation function calculation module 44.

[0150] The input of the second Sigmoid activation function calculation module 44 is connected to the output of the ninth convolution module 42, and the output of the second Sigmoid activation function calculation module 44 is connected to the input of the first element-wise multiplication module 33.

[0151] Understandably, the fifth convolutional module 29 and the sixth convolutional module 31 are both 3×3 convolutions, while the seventh convolutional module 39, the eighth convolutional module 41, and the ninth convolutional module 42 are all 1×1 convolutions. In the residual coordinate attention module, the input features are first extracted into image features through a convolutional layer containing two 3×3 convolutions and a Leaky ReLU activation function. Secondly, using the coordinate attention mechanism 32, the extracted feature maps are weighted and redistributed along the horizontal and vertical directions, respectively. Finally, the weighted image features are summed with the input features to obtain residual information incorporating positional information.

[0152] In some alternative embodiments, refer to Figure 4 , Figure 4 This is a schematic diagram of a first adaptive feature fusion module. The first adaptive feature fusion module 18 first adds the first feature map F1 and the second feature map F2 of adjacent processing streams to obtain the input feature F. InNext, global and salient information are obtained through Global Average Pooling (GAP) and Global Max Pooling (GMP), respectively. Then, the global and salient information are concatenated and exchanged between different channels using two 1×1 convolutions and a GELU activation function. Finally, two adaptive weight maps are obtained using two 1×1 convolutions and a Softmax function. These weight maps are then adaptively fused with the input first feature map F1 and the second feature map F2 to obtain the fused feature map F. Out .

[0153] The first adaptive feature fusion module 18 includes:

[0154] The input of the seventh element-wise addition module 45 is the first feature map and the second feature map, and the output of the seventh element-wise addition module 45 is connected to the input of the global average pooling module 46 and the input of the global max pooling module 50, respectively.

[0155] The global average pooling module 46, the tenth convolution module 47, the first GELU activation function calculation module 48, and the eleventh convolution module 49 are connected in sequence. The input of the global average pooling module 46 is connected to the output of the seventh element-wise addition module 45, and the output of the eleventh convolution module 49 is connected to the input of the second element-wise concatenation module 54.

[0156] The global max pooling module 50, the twelfth convolution module 51, the second GELU activation function calculation module 52, and the thirteenth convolution module 53 are connected in sequence. The input of the global max pooling module 50 is connected to the output of the seventh element-wise addition module 45, and the output of the thirteenth convolution module 53 is connected to the input of the second element-wise concatenation module 54.

[0157] The second element-wise concatenation module 54, the fourteenth convolution module 55, the fifteenth convolution module 56, and the third GELU activation function calculation module 57 are connected in sequence. The input of the second element-wise concatenation module 54 is connected to the output of the eleventh convolution module 49 and the output of the thirteenth convolution module 53, respectively. The output of the third GELU activation function calculation module 57 is connected to the input of the sixteenth convolution module 58 and the input of the seventeenth convolution module 59, respectively.

[0158] The inputs of SoftMax module 60 are connected to the outputs of the sixteenth convolution module 58 and the seventeenth convolution module 59, respectively, and the outputs of SoftMax module 60 are connected to the inputs of the second element-wise multiplication module 61 and the third element-wise multiplication module 62, respectively.

[0159] The input of the second element-wise multiplication module 61 is connected to the output of the first feature map and the sixteenth convolution module 58, respectively, and the output of the second element-wise multiplication module 61 is connected to the input of the eighth element-wise addition module 13.

[0160] The input of the third element-wise multiplication module 62 is connected to the output of the second feature map and the seventeenth convolution module 59, respectively, and the output of the second element-wise multiplication module 61 is connected to the input of the eighth element-wise addition module 13.

[0161] The input of the eighth element-wise addition module 13 is connected to the output of the sixteenth convolution module 58 and the output of the seventeenth convolution module 59, respectively. The output of the eighth element-wise addition module 13 is the fused feature map F. Out Understandably, the input to the global average pooling module 46 is F. In The output of the global average pooling module 46 is connected to the input of the tenth convolution module 47;

[0162] The input of the tenth convolution module 47 is connected to the input of the global average pooling module 46, and the output of the tenth convolution module 47 is connected to the input of the first GELU activation function calculation module 48.

[0163] The input of the first GELU activation function calculation module 48 is linked to the output of the tenth convolution module 47, and the output of the first GELU activation function calculation module 48 is linked to the input of the eleventh convolution module 49.

[0164] The input of the eleventh convolution module 49 is connected to the output of the first GELU activation function calculation module 48, and the output of the eleventh convolution module 49 is connected to the input of the second element-wise concatenation module 54.

[0165] The input to the global max pooling module 50 is F. In The output of the global max pooling module 50 is connected to the input of the twelfth convolution module 51;

[0166] The input of the twelfth convolution module 51 is connected to the input of the global maximum pooling module 50, and the output of the twelfth convolution module 51 is connected to the input of the second GELU activation function calculation module 52.

[0167] The input of the second GELU activation function calculation module 52 is linked to the output of the twelfth convolution module 51, and the output of the second GELU activation function calculation module 52 is linked to the input of the thirteenth convolution module 53.

[0168] The input of the thirteenth convolution module 53 is connected to the output of the second GELU activation function calculation module 52, and the output of the thirteenth convolution module 53 is connected to the input of the second element-wise concatenation module 54.

[0169] The input of the fourteenth convolution module 55 is connected to the output of the second element-wise concatenation module 49, and the output of the fourteenth convolution module 55 is connected to the input of the fifteenth convolution module 56.

[0170] The input of the fifteenth convolution module 56 is connected to the output of the fourteenth convolution module 55, and the output of the fifteenth convolution module 56 is connected to the input of the third GELU activation function calculation module 57.

[0171] The input of the third GELU activation function calculation module 57 is connected to the output of the fifteenth convolution module 56, and the output of the third GELU activation function calculation module 57 is connected to the input of the sixteenth convolution module 58 and the input of the seventeenth convolution module 59, respectively.

[0172] In some optional embodiments, the loss function used during training of the underwater image enhancement network model is calculated as follows:

[0173] L MSFFNet =λ1L char (G(X), Y)+λ2L ms (G(X), Y)+λ3L per (G(X), Y);

[0174] Among them, L char (G(X), Y) represents the Charbonnier loss, L ms (G(X), Y) represents the multi-scale structural similarity loss, L per (G(X), Y) represents the perceptual loss, λ1 is the first weight, λ2 is the second weight, and λ3 is the third weight.

[0175] Understandably, {X, Y} represents the input image pair, where X is the degraded underwater image, Y is the reference image corresponding to X, and G(X) is the augmented image. To constrain the learning process of the underwater image augmentation network model and promote the correct learning of the mapping relationship between low-quality and high-quality images, a loss function is used to supervise and constrain the underwater image augmentation network model. In this embodiment, λ1 = 0.5, λ2 = 0.5, and λ3 = 0.2; however, this embodiment does not impose specific limitations on these values.

[0176] In some optional embodiments, the Charbonnier loss is calculated as follows:

[0177]

[0178] Where G(X) is the enhanced image, Y is the reference image, and ε is the regularization term for mitigating non-differentiable points.

[0179] Understandably, the Charbonnier loss is used to calculate the pixel-level differences between the enhanced image and the reference image. This loss can effectively mitigate the interference of outliers and extreme pixel values, promoting a pixel distribution in the enhanced image that is consistent with the reference image. In this embodiment, ε = 10.-3 .

[0180] In some optional embodiments, the multi-scale structural similarity loss is calculated as follows:

[0181]

[0182] Where M represents different scales, μ G To enhance the mean of the image, μ Y σ is the mean of the reference image. G To enhance the standard deviation of the image, σ Y σ is the standard deviation of the reference image. GY To enhance the covariance between the image and the reference image, β m As the first relative importance, γ m For the second relative importance, C1 is the first constant, C1≠0, and C2 is the second constant, C2≠0.

[0183] It is understandable that multi-scale structural similarity loss can constrain the generated image from multiple scales, such as brightness, contrast and structure, so that the enhanced image is more similar to the reference image. In this embodiment, C1 = 0.02 and C2 = 0.03. This embodiment does not impose specific restrictions on this.

[0184] In some optional embodiments, the perceived loss is calculated as follows:

[0185]

[0186] Where N is the number of feature maps extracted, Φ j This represents the feature map calculated using the j-th convolution of the VGG-16 network.

[0187] It is understandable that perceptual loss can enhance the high-level semantic and detail information of an image by constraining it at the feature level. In this embodiment, N is set to 3, and the perceptual loss is calculated using the output features of relu1-2, relu2-2 and relu3-3 in the pre-trained VGG-16 network, respectively.

[0188] In summary, under the common constraints of the three loss functions, the method of this invention can effectively learn the mapping relationship between degraded underwater images and their reference images, generating high-quality enhanced images with vivid colors, high contrast, and clarity.

[0189] In some alternative embodiments, Figure 5 This is a subjective evaluation comparison chart of the underwater image enhancement method based on multi-flow feature fusion and the comparison method provided by this invention on a test set. Figure 6This is another subjective evaluation comparison chart on a test set between the underwater image enhancement method based on multi-flow feature fusion provided by this invention and the comparison method. Figure 5 The selected degraded underwater images belong to the UIEB dataset. Figure 6 The selected degraded underwater images belong to the EUVP dataset, and are obtained through... Figure 5 and Figure 6 As can be seen, the underwater image enhancement method based on multi-flow feature fusion provided by the present invention can appropriately improve the brightness of dim images, while removing blur and noise in the images and generating enhanced images with rich textures.

[0190] In some optional embodiments, four objective evaluation metrics are used to quantitatively analyze the results of the proposed method and all comparative methods. First, the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), two full-reference image quality evaluation metrics, are used to measure the similarity between the enhanced result and the reference image. Higher PSNR and SSIM values ​​indicate greater similarity between the enhanced image and the reference image in terms of structure, detail, and contrast. Second, Mean Squared Error (MSE) is used to measure the closeness between the enhanced image and the reference image; a lower MSE value indicates a closer similarity in pixel distribution. Finally, Blur is used to measure the sharpness of the enhanced image; a higher Blur value indicates a sharper enhanced image. The quantitative analysis results of the proposed method and each comparative method on the test set are shown in Tables 1 and 2, respectively. The image pairs in the test set used in Table 1 all belong to the UIEB dataset, and the image pairs in the dataset used in Table 2 all belong to the EUVP dataset, as follows:

[0191] Table 1. Comparison of the objective evaluation of the method of the present invention and the comparative method on the UIEB dataset.

[0192]

[0193] Table 2. Comparison of objective evaluations of the method of this invention and the comparative method on the EUVP dataset.

[0194]

[0195] According to Tables 1 and 2, the method provided by this invention has achieved significant advantages on both datasets and four evaluation metrics. This indicates that the method of this invention can better correct color cast, improve contrast, restore details, and obtain higher quality enhanced images.

[0196] In summary, the method provided by this invention can fully extract position-sensitive image features while properly preserving the original image details and textures, selectively highlight key features, achieve color correction, contrast enhancement and deblurring, and generate a high-quality image close to the reference image.

[0197] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0198] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on multi-flow feature fusion, characterized in that, include: Construct a dataset comprising Q image pairs, each image pair including a degraded underwater image and its corresponding reference image; The dataset is divided into a training set and a test set; Constructing an underwater image enhancement network model; constructing the underwater image enhancement network model includes: First convolutional module; The input of the first feature enhancement module is connected to the output of the first convolution module, and the output of the first feature enhancement module is connected to the input of the first element-wise addition module. The input of the first element-wise addition module is connected to the output of the first feature enhancement module and the output of the first convolution module, respectively, and the output of the first element-wise addition module is connected to the input of the second feature enhancement module. The input of the second feature enhancement module is connected to the output of the first element-by-element addition module, and the output of the second feature enhancement module is connected to the input of the second element-by-element addition module. The input of the second element-wise addition module is connected to the output of the second feature enhancement module and the output of the first element-wise addition module, respectively, and the output of the second element-wise addition module is connected to the input of the third feature enhancement module; The input of the third feature enhancement module is connected to the output of the second element-by-element addition module, and the output of the third feature enhancement module is connected to the input of the third element-by-element addition module. The input of the third element-wise addition module is connected to the output of the third feature enhancement module and the output of the second element-wise addition module, respectively, and the output of the third element-wise addition module is connected to the input of the second convolution module. The input of the second convolution module is connected to the third element-wise addition module, and the output of the second convolution module is connected to the input of the fourth element-wise addition module. The input of the fourth element-wise addition module is connected to the output of the second convolution module and the input of the first convolution module, respectively. The first feature enhancement module includes a first multi-stream feature estimation module, a second multi-stream feature estimation module, and a third convolution module connected in sequence; the first multi-stream feature estimation module and the second multi-stream feature estimation module have the same structure. The first multi-stream feature estimation module includes: a first feature processing stream, a second feature processing stream, and a third feature processing stream; the first feature processing stream takes the original resolution feature map as input; the second feature processing stream takes the 2x downsampled feature map as input; and the third feature processing stream takes the 4x downsampled feature map as input. The first feature processing stream includes: The input to the first residual coordinate attention module is the feature map at the original resolution, and the output of the first residual coordinate attention module is connected to the input of the first adaptive feature fusion module. The input of the first adaptive feature fusion module is connected to the output of the first residual coordinate attention module and the output of the third adaptive feature fusion module, respectively, and the output of the first adaptive feature fusion module is connected to the input of the second residual coordinate attention module. The input of the second residual coordinate attention module is connected to the output of the first adaptive feature fusion module, and the output of the second residual coordinate attention module is connected to the input of the second adaptive feature fusion module. The input of the second adaptive feature fusion module is connected to the output of the second residual coordinate attention module and the output of the fourth adaptive feature fusion module, respectively; the output of the second adaptive feature fusion module is connected to the input of the fourth convolution module. The input of the fourth convolutional module is connected to the output of the second adaptive feature fusion module, and the output of the fourth convolutional module is connected to the input of the fifth element-wise addition module. The input of the fifth element-wise addition module is connected to the output of the fourth convolution module and the feature map at the original resolution, respectively. The second feature processing stream includes: The input to the third residual coordinate attention module is the feature map that has been downsampled by two times, and the output of the third residual coordinate attention module is connected to the input of the third adaptive feature fusion module. The input of the third adaptive feature fusion module is connected to the output of the third residual coordinate attention module and the output of the fifth residual coordinate attention module, respectively. The output of the third adaptive feature fusion module is connected to the input of the fourth residual coordinate attention module and the input of the first adaptive feature fusion module, respectively. The input of the fourth residual coordinate attention module is connected to the output of the third adaptive feature fusion module, and the output of the fourth residual coordinate attention module is connected to the input of the fourth adaptive feature fusion module. The input of the fourth adaptive feature fusion module is connected to the output of the fourth residual coordinate attention module and the output of the sixth residual coordinate attention module, respectively, and the output of the fourth adaptive feature fusion module is connected to the input of the second adaptive feature fusion module. The third feature processing stream includes: The input of the fifth residual coordinate attention module is the feature map downsampled by four times, and the output of the fifth residual coordinate attention module is connected to the input of the third adaptive feature fusion module and the input of the sixth residual coordinate attention module, respectively. The input of the sixth residual coordinate attention module is connected to the output of the fifth residual coordinate attention module, and the output of the sixth residual coordinate attention module is connected to the input of the fourth adaptive feature fusion module. The underwater image enhancement network model is trained using the training set to obtain the trained underwater image enhancement network model. Select one of the degraded underwater images from the test set and input it into the trained underwater image enhancement network model. The trained underwater image enhancement network model outputs an enhanced image.

2. The underwater image enhancement method based on multi-flow feature fusion according to claim 1, characterized in that, The first residual coordinate attention module includes: Fifth convolution module; The input of the first LeakyReLU activation function calculation module is connected to the output of the fifth convolution module, and the output of the first LeakyReLU activation function calculation module is connected to the input of the sixth convolution module. The input of the sixth convolution module is connected to the output of the first LeakyReLU activation function calculation module, and the output of the sixth convolution module is connected to the input of the first element-wise multiplication module and the input of the coordinate attention mechanism, respectively. The input of the coordinate attention mechanism is connected to the output of the sixth convolution module, and the output of the coordinate attention mechanism is connected to the input of the first element-wise multiplication module. The input of the first element-wise multiplication module is connected to the output of the sixth convolution module and the output of the coordinate attention mechanism, respectively; the output of the first element-wise multiplication module is connected to the input of the second LeakyReLU activation function calculation module. The input of the second LeakyReLU activation function calculation module is connected to the output of the first element-wise multiplication module, and the output of the second LeakyReLU activation function calculation module is connected to the input of the sixth element-wise addition module; The input of the sixth element-wise addition module is connected to the output of the second LeakyReLU activation function calculation module and the input of the fifth convolution module, respectively.

3. The underwater image enhancement method based on multi-flow feature fusion according to claim 2, characterized in that, The coordinate attention mechanism includes: The input of the horizontal average pooling module is connected to the output of the sixth convolution module, and the output of the horizontal average pooling module is connected to the input of the first element-wise concatenation module. The input of the vertical average pooling module is connected to the output of the sixth convolution module, and the output of the vertical average pooling module is connected to the input of the first element-wise concatenation module. The input of the first element-wise concatenation module is connected to the output of the horizontal average pooling module and the output of the vertical average pooling module, respectively, and the output of the first element-wise concatenation module is connected to the input of the seventh convolution module. The input of the seventh convolution module is connected to the output of the first element-wise concatenation module, and the output of the seventh convolution module is connected to the input of the third LeakyReLU activation function calculation module. The input of the third LeakyReLU activation function calculation module is connected to the output of the seventh convolution module, and the output of the third LeakyReLU activation function calculation module is connected to the input of the eighth convolution module and the input of the ninth convolution module, respectively. The input of the eighth convolution module is connected to the output of the third LeakyReLU activation function calculation module, and the output of the eighth convolution module is connected to the input of the first Sigmoid activation function calculation module. The input of the first Sigmoid activation function calculation module is connected to the output of the eighth convolution module, and the output of the first Sigmoid activation function calculation module is connected to the input of the first element-wise multiplication module. The input of the ninth convolution module is connected to the output of the third LeakyReLU activation function calculation module, and the output of the ninth convolution module is connected to the input of the second Sigmoid activation function calculation module. The input of the second Sigmoid activation function calculation module is connected to the output of the ninth convolution module, and the output of the second Sigmoid activation function calculation module is connected to the input of the first element-wise multiplication module.

4. The underwater image enhancement method based on multi-flow feature fusion according to claim 1, characterized in that, The first adaptive feature fusion module includes: The input to the seventh element-wise addition module is the first feature map and the second feature map, and the output of the seventh element-wise addition module is connected to the input of the global average pooling module and the input of the global max pooling module, respectively. The global average pooling module, the tenth convolution module, the first GELU activation function calculation module, and the eleventh convolution module are connected in sequence. The input of the global average pooling module is connected to the output of the seventh element-wise addition module, and the output of the eleventh convolution module is connected to the input of the second element-wise concatenation module. The global max pooling module, the twelfth convolution module, the second GELU activation function calculation module, and the thirteenth convolution module are connected in sequence. The input of the global max pooling module is connected to the output of the seventh element-wise addition module, and the output of the thirteenth convolution module is connected to the input of the second element-wise concatenation module. The second element-wise concatenation module, the fourteenth convolution module, the fifteenth convolution module, and the third GELU activation function calculation module are connected sequentially. The input of the second element-wise concatenation module is connected to the output of the eleventh convolution module and the output of the thirteenth convolution module, respectively. The output of the third GELU activation function calculation module is connected to the input of the sixteenth convolution module and the input of the seventeenth convolution module, respectively. The input of the SoftMax module is connected to the output of the sixteenth convolution module and the output of the seventeenth convolution module, respectively. The output of the SoftMax module is connected to the input of the second element-wise multiplication module and the input of the third element-wise multiplication module, respectively. The input of the second element-wise multiplication module is connected to the output of the first feature map and the sixteenth convolution module, respectively, and the output of the second element-wise multiplication module is connected to the input of the eighth element-wise addition module. The input of the third element-wise multiplication module is connected to the output of the second feature map and the seventeenth convolution module, respectively, and the output of the second element-wise multiplication module is connected to the input of the eighth element-wise addition module. The input of the eighth element-wise addition module is connected to the output of the sixteenth convolution module and the output of the seventeenth convolution module, respectively.

5. The underwater image enhancement method based on multi-flow feature fusion according to claim 1, characterized in that, The loss function used in training the underwater image enhancement network model is calculated as follows: ； in, For Charbonnier loss, For multi-scale structural similarity loss, In order to perceive loss, As the first weight, As the second weight, As the third weight, For the enhanced image, The reference image is shown.

6. The underwater image enhancement method based on multi-flow feature fusion according to claim 5, characterized in that, The Charbonnier loss is calculated as follows: ； in, For the enhanced image, The reference image, To alleviate the regularization of non-differentiable points.

7. The underwater image enhancement method based on multi-flow feature fusion according to claim 5, characterized in that, The multi-scale structural similarity loss is calculated as follows: ； in, For different scales, The mean of the enhanced image. The mean of the reference image. The standard deviation of the enhanced image is... The standard deviation of the reference image is... The covariance between the enhanced image and the reference image. As the first relative importance, As the second most important, It is the first constant. , It is the second constant. .

8. The underwater image enhancement method based on multi-flow feature fusion according to claim 5, characterized in that, The perceived loss is calculated as follows: ； in, The number of feature maps to be extracted, This represents the feature map calculated using the j-th convolution of the VGG-16 network.

Images