Underwater image enhancement method and system based on multi-modal degradation feature learning

Abstract

The invention relates to the technical field of image processing, and provides an underwater image enhancement method and system based on multi-modal degradation feature learning. The method comprises the following steps: collecting underwater image data of different modes to construct an underwater image data set, and dividing the underwater image data set into a training set and a test set; preprocessing the training set to obtain an image text degradation pair data set; constructing a multi-modal visual language MD-CLIP network framework, and training the MD-CLIP network framework by using the image text degradation pair data set; the MD-CLIP network framework comprises an image coding controller ContNet module, a pre-trained CLIP module and a comparative learning module; integrating the trained image coding controller ContNet module into an NAFNet network structure, generating an image enhancement network, and training by using the test set to obtain an underwater image enhancement model; and inputting a to-be-enhanced underwater degraded image into the underwater image enhancement model to generate an enhancement result graph of the to-be-enhanced underwater degraded image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an underwater image enhancement method and system based on multimodal degradation feature learning. Background Art

[0002] Early research methods for underwater image enhancement are mainly divided into two types: non-physical model and physical model-based. Non-physical model methods aim to modify image pixel values ​​to improve the contrast of underwater blurred images, but non-physical models are prone to over-enhancement or under-enhancement. Physical model-based methods regard underwater image enhancement as an uncertain inverse problem and use hand-crafted prior knowledge to estimate the potential parameters of the image formation model. For example, Chiang et al. and Drews et al. achieved underwater image enhancement by modifying the dark channel prior (DCP) proposed by He et al. In addition, Li et al. proposed a hybrid method (1: Li, C., Guo, J., Guo, C., Cong, R., Gong, J.: A hybrid method for underwater image correction [J]. Journal of Visual Communication and Image Representation, 2017, 48, 151-160.), including color correction and underwater image deblurring, to improve the visual quality of degraded underwater images.

[0003] In recent years, exploring the underwater world has become a popular way. Underwater image enhancement, as an important preprocessing process, aims to improve the visual quality of low-quality images. Various deep learning-based methods have been proposed, which can be classified into two categories, namely CNN-based algorithms and GAN-based algorithms. In addition, underwater image enhancement based on deep learning can be divided into 5 methods according to their architectural differences (2: S. Anwar and C. Li, "Diving deeper into underwater image enhancement: Asurvey," Signal Processing: Image Communication, vol. 89, p. 115978, 2020. 3): encoder-decoder model, which is a well-known architecture for improving image quality in many low-level tasks; modules or blocks with the same structure are designed and repeatedly applied in the network to improve the feature extraction ability of the network; the designed multi-branch aims to learn different features through separate branches; the enhancement and restoration performance is improved by predicting the depth map or transmission map of the underwater image; multiple generators are used to infer the improved image. The above-mentioned underwater image enhancement method has weak generalization ability, which results in good performance in specific underwater scenes, but poor results in other scenes, which limits the versatility of the method to a certain extent. Summary of the invention

[0004] In order to solve the problem that the underwater image enhancement methods in the prior art have weak generalization ability and their versatility is limited, the present invention provides an underwater image enhancement method based on multimodal degradation feature learning. By constructing a multimodal visual language MD-CLIP network framework and training it, and then combining it with other reconstruction network models, the generalization ability and performance of the model can be improved, and the problems of low model performance and poor adaptability can be solved.

[0005] In a first aspect, the present invention provides an underwater image enhancement method based on multimodal degradation feature learning, comprising:

[0006] Step 1: Collect underwater image data of different modalities to construct an underwater image dataset, and divide the underwater image dataset into a training set and a test set; wherein the underwater image dataset includes corresponding underwater low-quality images and target clear images;

[0007] Step 2: Preprocess the training set to obtain an image-text degradation pair data set;

[0008] Step 3: construct a multimodal visual language MD-CLIP network framework, and use the image-text degradation pair dataset to train the MD-CLIP network framework; the MD-CLIP network framework includes an image coding controller ContNet module, a pre-trained CLIP module and a contrastive learning module;

[0009] Step 4: Integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model;

[0010] Step 5: Input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhancement result image of the underwater degraded image to be enhanced.

[0011] Furthermore, the underwater image data of different modes include low-light images, overall greenish images and overall bluish images.

[0012] Furthermore, the training set is preprocessed using a visual language pre-trained BLIP model to obtain an image-text degradation pair dataset; wherein the image-text degradation pair includes a clear target image, an underwater low-quality image, and text information, and the text information includes an image text description and the blur level of the image

[0013] Furthermore, the pre-trained CLIP module includes a text encoder and an image encoder.

[0014] Furthermore, step 3 specifically includes:

[0015] Input the text information and the target clear image into the pre-trained CLIP module, and output text features and image features;

[0016] Inputting the underwater low-quality image into the ContNet module, and outputting image degradation features and text degradation features;

[0017] The image degradation feature and the image feature, the text degradation feature and the text feature are respectively subjected to contrast learning in the contrast learning module.

[0018] Furthermore, in step 3, when constructing the MD-CLIP network framework, Vision Transformer is used as the backbone of the image encoder and the ContNet module.

[0019] Furthermore, in step 4, the basic encoder-decoder architecture and hierarchical skip connection mode of the NAFNet network structure are retained, and the output of the pre-trained image encoding controller ContNet module is fused into the NAFNet Block module of the decoder through a cross-attention mechanism.

[0020] Furthermore, the NAFNet Block module enhances the underwater image by combining the image degradation features and text degradation features output by the ContNet module; the NAFNet Block module includes a first LayerNorm layer, a 3×3 depth convolution and a 1×1 convolution layer, a first SimpleGate module, a first SCA module, a first 1×1 convolution layer, a second LayerNorm layer, a second 1×1 convolution layer, a second SimpleGate module and a third 1×1 convolution layer;

[0021] The specific processing flow of the NAFNet Block module is as follows:

[0022] The original features input into the NAFNet Block module are normalized by the first LayerNorm layer, and the local spatial features of the image are extracted and fused through 3×3 deep convolution and 1×1 convolution layers. Then, the weights of different feature channels are adjusted through the SimpleGate module, and the text degradation features are introduced. Then, the features are input into the SCA module, and the intermediate features are obtained through further channel fusion and information integration through the first 1×1 convolution layer.

[0023] The intermediate features are then residually connected with the original features to obtain the intermediate output, which is sent to the second LayerNorm layer. The image degradation features are introduced through the second 1×1 convolution processing, and then passed through the SimpleGate module. Finally, further integration and channel processing are performed through 1×1 convolution, and the intermediate output is residually connected to obtain the output of the NAFNetBlock module.

[0024] Furthermore, by setting an image quality evaluation index, the effect of the underwater image enhancement model is quantitatively evaluated, and the model parameters are optimized according to the feedback of the image quality evaluation index result.

[0025] In a second aspect, the present invention provides an underwater image enhancement system based on multimodal degradation feature learning, characterized in that it includes:

[0026] A data set construction module is used to collect underwater image data of different modalities to construct an underwater image data set, and divide the underwater image data set into a training set and a test set; wherein the underwater image data set includes corresponding underwater low-quality images and target clear images;

[0027] A data set preprocessing module, used to preprocess the training set to obtain image-text degradation pairs;

[0028] An MD-CLIP network framework construction module is used to construct a multimodal visual language MD-CLIP network framework and train the MD-CLIP network framework using the image-text degradation pair; the MD-CLIP network framework includes an image coding controller ContNet module, a pre-trained CLIP module and a contrastive learning module;

[0029] An image enhancement model building module is used to integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model;

[0030] The result output module is used to input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhancement result image of the underwater degraded image to be enhanced.

[0031] Beneficial effects of the present invention:

[0032] The present invention designs an MD-CLIP network framework to train the image coding controller ContNet module therein. The trained ContNet module can extract text features and image features of underwater degraded images. Then the ContNet module is combined with the image enhancement model for reconstruction to guide the model to learn image reconstruction.

[0033] The method provided by the present invention can not only significantly improve the performance of the visual language model in low-level visual tasks, but also provide a general framework that can effectively extend the ability of the trained image feature extraction model to downstream tasks of unlabeled or zero-sample prediction. The present invention applies the trained ContNet module to the NAFNet network model to complete the final network model design for underwater image enhancement, improves the performance and generalization ability of the model, and solves the problems of low performance and poor adaptability of the underwater enhanced image model. Through a large number of experiments, the present invention proves the effectiveness of the multimodal degradation features predicted by the ContNet module trained by the MD-CLIP network in improving the performance of the image restoration model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic flow chart of an underwater image enhancement method based on multimodal degradation feature learning provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the MD-CLIP network framework structure provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of a training set provided by an embodiment of the present invention being preprocessed by a BLIP model;

[0037] Figure 4 A schematic diagram of the structure of the image coding controller ContNet based on Vision Transformer provided in an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of an underwater image enhancement model provided by an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of visual comparison results of different underwater image enhancement methods provided by an embodiment of the present invention on the RUIE_Color90 dataset;

[0040] Figure 7 A schematic diagram of the visual comparison results of different underwater image enhancement methods provided by an embodiment of the present invention on the U45 dataset. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, an embodiment of the present invention provides an underwater image enhancement method based on multimodal degradation feature learning, comprising:

[0043] Step 1: Collect underwater image data of different modalities to construct an underwater image dataset, and divide the underwater image dataset into a training set and a test set; the underwater image dataset includes corresponding underwater low-quality images and target clear images.

[0044] Specifically, the collected underwater image data of different modalities include low-light images, overall greenish images, and overall bluish images.

[0045] It is understandable that environmental factors in underwater photography can affect the color of images. For example, in deep water areas, since red light is absorbed in large quantities, the images captured often appear blue-green in color.

[0046] The embodiment of the present invention selects underwater image data of multiple different modalities to construct an underwater image dataset. As shown in Table 1, underwater_dark, underwater_imagenet and underwater_scenes in the UIEB dataset, LSUI dataset and EUVP dataset are selected to construct an underwater image dataset. The datasets are all paired data, including input and GT images, where the input is a low-quality underwater image and the GT is the corresponding clear target image.

[0047] Table 1 Details of the datasets used for training and testing MD-CLIP

[0048]

[0049] Step 2: Preprocess the training set to obtain the image-text degradation pair dataset;

[0050] Step 3: Construct a multimodal visual language MD-CLIP network framework and use the image-text degradation dataset to train the MD-CLIP network framework; Figure 2 As shown, the MD-CLIP network framework includes the image encoding controller ContNet module, the pre-trained CLIP module and the contrastive learning module;

[0051] Step 4: Integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model;

[0052] Step 5: Input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhanced result image of the underwater degraded image to be enhanced.

[0053] The method provided by the embodiment of the present invention designs an MD-CLIP network framework to train the image coding controller ContNet module therein, and the trained ContNet module can extract text features and image features of underwater degraded images; then the ContNet module is combined with the image enhancement model for reconstruction to guide the model to learn image reconstruction.

[0054] As an underwater image enhancement method based on multimodal degradation feature learning in an embodiment of the present invention, further, Figure 3As shown in the figure, the training set is preprocessed using the visual language pre-trained BLIP model to obtain an image-text degradation pair dataset; the image-text degradation pair includes a clear target image, an underwater low-quality image, and text information, and the text information includes an image text description and the blur degree of the image. The visual language pre-trained BLIP model can generate an image text description for the clear target image in the training set, and calculate the blur degree (PSNR, SSIM value) of the underwater low-quality image based on the clear target image and the underwater low-quality image.

[0055] As an underwater image enhancement method based on multimodal degradation feature learning in an embodiment of the present invention, further, Figure 2 As shown, the pre-trained CLIP module includes a text encoder and an image encoder. Step 3 specifically includes:

[0056] Input text information and target clear image into the pre-trained CLIP module, and output text features and image features;

[0057] Input the underwater low-quality image into the ContNet module and output the image degradation features and text degradation features;

[0058] The image degradation features and image features, and the text degradation features and text features are contrastively learned in the contrastive learning module respectively.

[0059] Specifically, training the MD-CLIP network framework is the process of training the image coding controller ContNet. The image coding controller ContNet learns to predict the multimodal degradation features of the input image through a large amount of data training, namely, text degradation features and image degradation features. The image-text degradation pairs obtained by preprocessing the training set are input into the MD-CLIP network. ContNet receives the input underwater low-quality images and learns to predict the text degradation features and image degradation features of the input underwater low-quality images, namely, multimodal degradation features. The MD-CLIP network framework references the pre-trained CLIP model. The image encoder of the pre-trained CLIP model encodes the image into the corresponding image features under the joint action of the input target clear image and the control factors from the image coding controller ContNet, and compares and learns with the image degradation features predicted by ContNet; the text encoder Text Encoder of the pre-trained CLIP model encodes the input text information into text features, and compares and learns with the text degradation features predicted by ContNet.

[0060] As an underwater image enhancement method based on multimodal degradation feature learning in an embodiment of the present invention, further, in step 3, as Figure 4As shown in Figure 2, the Vision Transformer is used as the backbone of the image encoder and ContNet module when constructing the MD-CLIP network framework.

[0061] Specifically, the image encoder controller ContNet is a copy of the image encoder Image Encoder of CLIP, wrapped with some zero-initialized connections to add control to the image encoder, manipulating the outputs of all encoder blocks to control the prediction of the image encoder. In this embodiment, Vision Transformer is used as the default backbone of the image encoder and controller ContNet. Figure 4 As shown in the figure, the control process is shown, where the output of the image encoding controller ContNet during training consists of two parts: multimodal degradation features and hidden controls. The hidden controls are parameters, which are added to the parameter calculation of Image Encode during training, affecting and controlling the output of the image encoder Image Encode. It should be noted that the hidden controls include outputs from all transformer blocks, which are then added to the corresponding encoder blocks to control the prediction results. The connections between transformer blocks are simple dense neural networks, and all parameters are initialized to zero, which gradually affect the image encoder Image Encode during training.

[0062] It can be understood that Vision Transformer is a deep learning model based on the Transformer architecture, which is a conventional deep learning model in this field and will not be elaborated in detail here.

[0063] As an underwater image enhancement method based on multimodal degradation feature learning in an embodiment of the present invention, further, in step 4, as Figure 5 As shown in (a), the basic encoder-decoder architecture and hierarchical skip connection mode of the NAFNet network structure are retained, and the output of the trained image coding controller ContNet module is fused into the decoder's NAFNet Block module through a cross-attention mechanism. The prediction result of the trained image coding controller ContNet module is a multimodal degradation feature composed of image degradation features and text degradation features. The output of the image coding controller ContNet module (multimodal degradation features) is inserted into the NAFNet Block module of the NAFNet model using a cross-attention mechanism.

[0064] It is understandable that embedding these multimodal degradation features at different points in the NAFNet framework will result in different performance levels. Ablation experiments were conducted to determine the most favorable insertion points for these multimodal degradation features. The model graph provided by the embodiment of the present invention is the most favorable insertion point for multimodal features, which improves the performance of the network on image restoration tasks.

[0065] Specifically, Figure 5 As shown in (b), the NAFNet Block module combines the image degradation features and text degradation features output by the ContNet module to enhance the underwater image; the NAFNet Block module includes the first LayerNorm layer, 3×3 depth convolution and 1×1 convolution layer, the first SimpleGate module, the first SCA module, the first 1×1 convolution layer, the second LayerNorm layer, the second 1×1 convolution layer, the second SimpleGate module and the third 1×1 convolution layer;

[0066] The specific processing flow of the NAFNet Block module is as follows:

[0067] The original features input to the NAFNet Block module are normalized by the first LayerNorm layer, and the local spatial features of the image are extracted and fused through 3×3 deep convolution and 1×1 convolution layers. Then, the weights of different feature channels are adjusted through the first SimpleGate module, and text degradation features are introduced. Then, the features are input to the first SCA module, and the intermediate features are obtained through further channel fusion and information integration through the first 1×1 convolution layer.

[0068] The intermediate features are then residually connected with the original features to obtain the intermediate output, which is sent to the second LayerNorm layer. The image degradation features are introduced through the second 1×1 convolution, and then passed through the second SimpleGate module. Finally, the third 1×1 convolution is used for further integration and channel processing, and the intermediate output is residually connected to obtain the output of the NAFNet Block module.

[0069] Specifically, the original features input to the NAFNet Block module are normalized through the first LayerNorm layer. This processing step can adjust the features of the image so that its mean is close to 0 and its variance is close to 1, thereby providing a more stable input for subsequent convolution operations. Then, it is processed through 3×3 deep convolution and 1×1 convolution layers, where 3×3 deep convolution is used to extract local spatial features of the image to help capture low-level texture and edge information, while 1×1 convolution further processes the channel dimension so that information can be better transmitted and fused between different channels. The features after the convolution operation pass through the first SimpleGate module. The SimpleGate module is a gating mechanism that selectively adjusts the weights of different features, which helps the model to enhance or suppress certain features as needed when reprocessing the image, thereby enhancing the model's adaptability. Then, text degradation features are introduced, which provide the model with context for image degradation and help the network to make appropriate enhancement strategies based on different degradation types of the image. Next, we enter the first SCA (Selective Channel Attention) module. The SCA module is a channel attention mechanism that can assign different weights to each feature channel, helping the model focus on more important channel features, thereby suppressing irrelevant or redundant channels and improving the efficiency and effect of image enhancement. The features processed by the first SCA module are further channel-fused and information-integrated through the first 1×1 convolutional layer to obtain intermediate features.

[0070] Residual connection of intermediate features with original features can help alleviate the gradient vanishing problem in deep networks and promote the flow of information, so that the network can better maintain the structure of the original image while performing effective enhancement. The features that have been residually connected generate an intermediate output. The intermediate output is fed into the second LayerNorm layer and then further processed by the second 1×1 convolution layer. Next, the image degradation features are introduced for feature fusion. In this step, the image degradation features further optimize the global information of the image, such as helping to restore details and improve contrast. Then, the second SimpleGate module performs gate processing again to further optimize the selection of features. Finally, the features are further integrated and channel processed by the third 1×1 convolution layer to obtain the enhanced result. The enhanced result is residually connected again with the previously generated intermediate output to ensure the preservation of image structure and information, while making the final output more refined and accurate.

[0071] It can be understood that the image degradation features and text degradation features output by the ContNet module constitute multimodal degradation features, and the unique mode of the embodiment of the present invention synergizes and promotes the ability of the network to enhance image restoration. Figure 5(b) shows in detail the specific structural configuration after the ContNet module is integrated into the NAFNet Block module, which enables the seamless integration and processing of multimodal information in the NAFNet framework, thus confirming the innovative adaptation and its potential benefits for the target image restoration task. It should be noted that among these multimodal degradation features, the embedding of text degradation features is specifically designed to enhance the network's ability to identify and process different types of degradation in a unified image restoration environment. This targeted enhancement is crucial to improving the adaptability and accuracy of the network in classifying and correcting various forms of image degradation.

[0072] The core function of the NAFNet Block module provided by the embodiment of the present invention is that it can combine image degradation features and text degradation features, and automatically learn and restore the details of underwater images through multi-level convolution, attention mechanism and gating module, especially when dealing with underwater image degradation (such as blur, color distortion, etc.). It shows excellent results. By introducing text degradation features, the module can adjust the enhancement strategy according to different image degradation types, making image restoration more targeted and adaptive. Ultimately, the NAFNet Block module can not only improve the details, contrast and clarity of degraded images, but also effectively deal with noise and blur problems in degraded images, especially in complex underwater environments, showing powerful image restoration (enhancement) capabilities.

[0073] As an underwater image enhancement method based on multimodal degradation feature learning in an embodiment of the present invention, further, by setting an image quality evaluation index, the effect of the underwater image enhancement model is quantitatively evaluated, and the model parameters are optimized according to the feedback of the image quality evaluation index results.

[0074] The embodiment of the present invention further provides an underwater image enhancement system based on multimodal degradation feature learning, comprising:

[0075] A data set construction module is used to collect underwater image data of different modalities to construct an underwater image data set, and divide the underwater image data set into a training set and a test set; wherein the underwater image data set includes corresponding underwater low-quality images and target clear images;

[0076] The data set preprocessing module is used to preprocess the training set to obtain image-text degradation pairs;

[0077] MD-CLIP network framework building module, used to build a multimodal visual language MD-CLIP network framework and train the MD-CLIP network framework using image-text degradation pairs; the MD-CLIP network framework includes an image encoding controller ContNet module, a pre-trained CLIP module, and a contrastive learning module;

[0078] The image enhancement model building module is used to integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model;

[0079] The result output module is used to input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhancement result image of the underwater degraded image to be enhanced.

[0080] In order to verify the effectiveness of the proposed method, the proposed method (NAFNet+MD-CLIP) was compared with five other advanced underwater image enhancement methods (UT-UIE, SyreaNet, WaterNet, SGUIE-Net, and RAUNE-Net). Figure 6 As shown in the figure, the experiment verifies the image restoration effect of different underwater image enhancement methods on the RUIE_Color90 dataset. The first column is the original input underwater blurred image, and the last six columns are the results obtained using different underwater image enhancement methods. It can be seen that the method of the present invention has the best enhancement effect on the input image, which can correct the color deviation of the underwater image, improve the contrast and detail clarity of the image, and enhance the overall visual effect of the image. However, the image enhanced by other methods will have a dark background or an overall green or blue bias.

[0081] like Figure 7 As shown, the experiment also tested the visual comparison of different underwater image enhancement methods on the U45 dataset, and the method of the present invention also achieved the best visual effect. Based on the above experiments, not only the performance improvement of the method of the present invention compared with other methods is verified, but also the excellent generalization ability of the method of the present invention is verified.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on multimodal degradation feature learning, characterized in that: include: Step 1: Collect underwater image data of different modalities to construct an underwater image dataset, and divide the underwater image dataset into a training set and a test set; The underwater image dataset includes corresponding underwater low-quality images and target clear images; Step 2: Preprocess the training set to obtain an image-text degradation pair data set; Step 3: construct a multimodal visual language MD-CLIP network framework, and use the image-text degradation pair dataset to train the MD-CLIP network framework; the MD-CLIP network framework includes an image coding controller ContNet module, a pre-trained CLIP module and a contrastive learning module; Step 4: Integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model; Step 5: Input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhancement result image of the underwater degraded image to be enhanced.

2. The underwater image enhancement method based on multimodal degradation feature learning according to claim 1, characterized in that: The underwater image data of different modes include low-light images, overall greenish images and overall bluish images.

3. The underwater image enhancement method based on multimodal degradation feature learning according to claim 1, characterized in that: The training set is preprocessed using a visual language pre-trained BLIP model to obtain an image-text degradation pair dataset; wherein the image-text degradation pair includes a clear target image, an underwater low-quality image and text information, and the text information includes an image text description and the blur degree of the image.

4. The underwater image enhancement method based on multimodal degradation feature learning according to claim 3 is characterized in that: The pre-trained CLIP module includes a text encoder and an image encoder.

5. The underwater image enhancement method based on multimodal degradation feature learning according to claim 3 is characterized in that: Step 3 specifically includes: Input the text information and the target clear image into the pre-trained CLIP module, and output text features and image features; Inputting the underwater low-quality image into the ContNet module, and outputting image degradation features and text degradation features; The image degradation feature and the image feature, the text degradation feature and the text feature are respectively subjected to contrast learning in the contrast learning module.

6. The underwater image enhancement method based on multimodal degradation feature learning according to claim 4, characterized in that: In step 3, the Vision Transformer is used as the backbone of the image encoder and the ContNet module when constructing the MD-CLIP network framework.

7. The underwater image enhancement method based on multimodal degradation feature learning according to claim 5, characterized in that: In step 4, the basic encoder-decoder architecture and hierarchical skip connection mode of the NAFNet network structure are retained, and the output of the trained image encoding controller ContNet module is fused into the NAFNetBlock module of the decoder through the cross attention mechanism.

8. The underwater image enhancement method based on multimodal degradation feature learning according to claim 7, characterized in that: The NAFNet Block module enhances the underwater image by combining the image degradation features and text degradation features output by the ContNet module; the NAFNet Block module includes a first LayerNorm layer, a 3×3 depth convolution and a 1×1 convolution layer, a first SimpleGate module, a first SCA module, a first 1×1 convolution layer, a second LayerNorm layer, a second 1×1 convolution layer, a second SimpleGate module and a third 1×1 convolution layer; The specific processing flow of the NAFNet Block module is as follows: The original features input into the NAFNet Block module are normalized by the first LayerNorm layer, local spatial features of the image are extracted and fused by the 3×3 deep convolution and 1×1 convolution layers, and then the weights of different feature channels are adjusted by the first SimpleGate module, and the text degradation features are introduced, and then enter the first SCA module, and further channel fusion and information integration are performed by the first 1×1 convolution layer to obtain intermediate features; The intermediate features are then residually connected with the original features to obtain an intermediate output, which is sent to the second LayerNorm layer. The image degradation features are introduced through the second 1×1 convolution processing, and then passed through the second SimpleGate module. Finally, the third 1×1 convolution is used for further integration and channel processing, and a residual connection is performed with the intermediate output to obtain the output of the NAFNet Block module.

9. The underwater image enhancement method based on multimodal degradation feature learning according to any one of claims 1 to 8, characterized in that: By setting the image quality evaluation index, the effect of the underwater image enhancement model is quantitatively evaluated, and the model parameters are optimized according to the feedback of the image quality evaluation index result.

10. An underwater image enhancement system based on multimodal degradation feature learning, characterized in that: include: A data set construction module, used to collect underwater image data of different modalities to construct an underwater image data set, and divide the underwater image data set into a training set and a test set; The underwater image dataset includes corresponding underwater low-quality images and target clear images; A data set preprocessing module, used to preprocess the training set to obtain image-text degradation pairs; An MD-CLIP network framework construction module is used to construct a multimodal visual language MD-CLIP network framework and train the MD-CLIP network framework using the image-text degradation pair; the MD-CLIP network framework includes an image coding controller ContNet module, a pre-trained CLIP module and a contrastive learning module; An image enhancement model building module is used to integrate the trained image encoding controller ContNet module into the NAFNet network structure, generate an image enhancement network and train it using the test set to obtain an underwater image enhancement model; The result output module is used to input the underwater degraded image to be enhanced into the underwater image enhancement model to generate an enhancement result image of the underwater degraded image to be enhanced.

Images