Underwater image enhancement quality evaluation method based on quality perception domain adaptation

Abstract

The invention provides an underwater image enhancement quality evaluation method based on quality perception domain adaptation. The method comprises the following steps: S1, acquiring a plurality of original underwater images to generate a training set and a test set; s2, constructing an underwater image enhancement quality evaluation network by using a deep learning framework; s3, training the underwater image enhancement quality evaluation network by using the training set, calculating a root-mean-square error between a prediction quality score and a label corresponding to each first enhancement image, and obtaining network loss in combination with binary cross entropy loss; s4, repeating the step S3 for multiple rounds to obtain an underwater image enhancement quality evaluation network model; and S5, testing each second enhanced image in the test set by using the underwater image enhancement quality evaluation network model to obtain a corresponding predicted quality score as an underwater image enhancement quality evaluation result. The method has the beneficial effects that the quality of the underwater enhanced image can be efficiently and accurately predicted.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image enhancement quality evaluation, and in particular to an underwater image enhancement quality evaluation method based on quality perception domain adaptation. Background Art

[0002] Underwater image enhancement quality evaluation aims to evaluate the quality of images generated by different underwater image enhancement algorithms. As a very important quality evaluation method, underwater image enhancement quality evaluation has been widely used in various fields of underwater image processing, such as underwater image enhancement, underwater image detection, underwater image classification and other tasks. In recent years, with the rapid development of deep learning, convolutional neural networks, Transformer and other technologies have achieved good results in image quality evaluation tasks. However, the labels of underwater image enhancement quality evaluation datasets are difficult to obtain, and underwater image enhancement quality evaluation algorithms trained on known underwater scenes are often difficult to be effectively promoted and applied in unknown underwater scenes.

[0003] Existing image quality assessment methods have widely used deep learning for modeling and have achieved good results. However, this method usually requires a large number of MOS (Mean Opinion Score) labels and consumes a lot of manpower and material resources. This dependence limits the practicality and scalability of the technology. At present, there are also some quality assessment algorithm models designed specifically for underwater enhanced images, which can generally achieve good performance. However, these algorithm models usually have performance degradation problems when dealing with some challenging scenes. That is, the existing underwater image enhancement quality assessment model only performs well on data with the same distribution as the training set. When the data distribution changes, the generalization ability of the model will decrease, affecting its applicability in different scenarios. Therefore, there is an urgent need to study a method that can go beyond the limitations of existing technologies and accurately predict the quality of images with different distributions from the training set without the need for MOS labels. Summary of the invention

[0004] The technical problem to be solved by the present invention is to be able to efficiently and accurately predict the quality of underwater enhanced images. In order to overcome the defects of the above-mentioned prior art (or related technology), the present invention provides an underwater image enhancement quality evaluation method based on quality perception domain adaptation.

[0005] The present invention provides an underwater image enhancement quality evaluation method based on quality perception domain adaptation, comprising the following steps: Step S1, obtaining a plurality of original underwater images from the outside, then performing enhancement processing on each of the original underwater images and dividing them into a source domain dataset and a target domain dataset, and preprocessing the source domain dataset to obtain a training set, and preprocessing the target domain dataset to obtain a test set, wherein the training set includes a plurality of first enhanced images and labels corresponding to each of the first enhanced images, and the test set includes a plurality of second enhanced images; Step S2, using a deep learning framework to build an underwater image enhancement quality evaluation network, the underwater image enhancement quality evaluation network includes a visual feature extraction network, a quality regression network, a text guidance network, a cross-attention network and an adversarial domain adaptation network, the visual feature extraction network is used to extract features from the input image to obtain a feature map and feature map , the quality regression network is used to The text-guided network is used to generate a feature map based on the external input text template. , and based on the cosine similarity calculation method according to the feature map And the feature map Get output text , the cross-attention network is used to calculate the feature map and the output text An output tensor F_cross_after is obtained, and the adversarial domain adaptation network is used to obtain a binary cross entropy loss according to the output tensor F_cross_after and a pre-acquired target label dc_target; Step S3, using the training set to train the underwater image enhancement quality evaluation network, calculating the root mean square error between the predicted quality score and the label corresponding to each of the first enhanced images, and combining the binary cross entropy loss to obtain the network loss, and adjusting and optimizing the underwater image enhancement quality evaluation network according to the network loss; Step S4, repeating step S3 for multiple rounds to obtain an underwater image enhancement quality evaluation network model; Step S5: Use the underwater image enhancement quality evaluation network model to test each of the second enhanced images in the test set to obtain the corresponding predicted quality score as the underwater image enhancement quality evaluation result.

[0006] Compared with the prior art, the underwater image enhancement quality evaluation method based on quality perception domain adaptation of the present invention has the following advantages: In the present invention, the training set and the test set are acquired through step S1, the underwater image enhancement quality evaluation network is constructed through step S2, the underwater image enhancement quality evaluation network is trained through step S3, the underwater image enhancement quality evaluation network model is trained and generated through step S4, and the underwater image enhancement quality evaluation network model is tested through step S5. In the present invention, the original image is processed by an enhancement algorithm to obtain an enhanced image and a label, which reduces the dependence on the annotation of the data set and reduces the cost of manual annotation, can effectively improve the generalization ability of the underwater image enhancement quality evaluation network model in different underwater environments, and enhances the adaptability and robustness of the underwater image enhancement quality evaluation network model to unknown data. In addition, the underwater image enhancement quality evaluation network constructed in the present invention realizes the joint utilization of text information and image information by introducing a text-guided network, gives full play to the complementary characteristics of text and image, enables the adversarial domain adaptation network to pay more attention to the features related to the quality characteristics under the guidance of the text, thereby significantly improving the quality perception domain adaptation effect, effectively enhancing the underwater image enhancement quality evaluation network model The evaluation ability of the underwater image quality score can be effectively enhanced, and the underwater enhanced image quality can be predicted efficiently and accurately.

[0007] In a possible implementation manner, the preprocessing method adopted in step S1 is: Each of the first enhanced images and each of the second enhanced images is sequentially scaled and normalized, so that the size of each of the first enhanced images and each of the second enhanced images after preprocessing is ;as well as Scaling operation is performed on each of the labels so that the size of each of the labels after preprocessing is .

[0008] In a possible implementation, in step S2, the visual feature extraction network includes four layers of ResNet50 backbone networks connected in sequence, receiving each of the first enhanced images or each of the second enhanced images as input images through the first layer of ResNet50 backbone network, and outputting the feature map through the fourth layer of ResNet50 backbone network. And the feature map .

[0009] In a possible implementation, in step S2, the quality regression network includes a first fully connected layer, a first ReLU activation layer, a Dropout layer, and a second fully connected layer connected in sequence, and the feature map is received through the first fully connected layer. The predicted quality score is obtained by sequentially processing the first ReLU activation layer, the Dropout layer and the second fully connected layer.

[0010] In a possible implementation, the text template includes a plurality of phrases describing the guidance direction. In step S2, the text guidance network includes a text encoder and a computing block connected to each other. The text encoder is used to receive the text template input from the outside and obtain the feature map according to the encoding of each phrase in the text template. , the calculation block is used to convert the feature map As a feature map With the characteristic diagram The cosine similarity is calculated, and then the cosine similarity is processed by sigmoid to obtain the weight W, and then the weight W is compared with the feature map Multiply to get the output text .

[0011] In a possible implementation, in step S2, the cross-attention network transforms the feature map by linear transformation Converted into a query tensor Q, the output text Convert into a key tensor K and a value tensor V, calculate the dot product of the query tensor Q and the key tensor K to obtain the attention weight, then normalize the attention weight using the softmax function, apply the dropout operation to it, and use the normalized attention weight to perform weighted summation on the value tensor V to obtain the output tensor F_cross_after.

[0012] In a possible implementation, in step S2, the adversarial domain adaptation network includes a gradient reversal layer, a third fully connected layer, a batch normalization layer, a second ReLU activation layer and a fourth fully connected layer connected in sequence, and two-dimensional adaptive average pooling is applied to the output tensor F_cross_after through the gradient reversal layer, and the output tensor F_cross_after after average pooling is sequentially passed through the third fully connected layer, the batch normalization layer, the second ReLU activation layer and the fourth fully connected layer to obtain an output probability y, and the binary cross entropy loss between the output probability y and the target label dc_target is calculated.

[0013] In a possible implementation, in step S2, the binary cross entropy loss is obtained by the following calculation formula: ; in, represents the binary cross entropy loss; s represents the source domain; t represents the target domain; represents the number of distorted images from the source domain; Indicates that it has been processed by the adversarial domain adaptation network; Indicates that it has been processed across the attention network; Indicates that it has been processed by the visual feature extraction network; representing the input image of the source domain; represents said weight; It means that it has been processed by the text-guided network; represents the output text; represents the number of distorted images from the target domain; The label of the input image; The input image represents the target domain.

[0014] In a possible implementation, in step S3, the root mean square error is obtained by the following calculation formula: ; in, represents the root mean square error; represents the number of distorted images from the source domain; Indicates that it has been processed by the quality regression network; represents the predicted quality score corresponding to the first enhanced image; Indicates the label corresponding to the first enhanced image; The label of the input image.

[0015] In a possible implementation, the normalization operation includes: For each of the first enhanced images and each of the second enhanced images after the scaling operation, the pixel values ​​of all pixels in the R channel of each of the first enhanced images and each of the second enhanced images are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.225. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the steps of the present invention; Figure 2 It is a schematic diagram of the framework of the underwater image enhancement quality evaluation network of the present invention; Figure 3 A schematic diagram of a framework of a text-guided network of the present invention; Figure 4 Schematic diagram of the framework of the cross-attention network of the present invention; Figure 5 The figure shows the results of the gMAD competition between the method of the present invention and the existing method. DETAILED DESCRIPTION

[0017] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present invention, and are not intended to limit the protection scope of the embodiments of the present invention. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0018] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] See also Figure 1 The embodiment of the present invention discloses an underwater image enhancement quality evaluation method based on quality perception domain adaptation, comprising: Step S1: select or construct a data set, which includes several original images obtained by different enhancement algorithms and the labels corresponding to each first enhanced image, as a source domain data set; then select or construct another data set, which also includes several original images obtained by different enhancement algorithms and the second enhanced images, as a target domain data set; then perform preprocessing including scaling and normalization operations on each first enhanced image and the second enhanced image, so that the size of the preprocessed image is ; and preprocess each label in the dataset only including scaling operations, so that the size of the preprocessed label is ; Then all the preprocessed first enhanced images in the source domain data set and their corresponding preprocessed labels are used as the training set, and all the preprocessed second enhanced images in the target domain data set are used as the test set, such as: the training set includes two data sets with different contents and enhancement methods; wherein, in this embodiment ; Step S2: Use a deep learning framework to build a deep neural network as an underwater image enhancement quality evaluation network based on quality-aware domain adaptation. The network includes a quality assessment module and a quality-aware domain adaptation module, such as Figure 2 As shown; wherein the quality assessment module includes a visual feature extraction network of the method and quality regression network ; Quality-aware domain adaptation module includes text-guided network, cross-attention network and adversarial domain adaptation networks ; Step S3: Use the training set to train the underwater image enhancement quality assessment network based on quality-aware domain adaptation. After each round of training, the underwater image enhancement quality assessment network based on quality-aware domain adaptation outputs the root mean square error between the predicted quality score and the label of each first enhanced image in the training set and the binary cross entropy loss output by the adversarial domain adaptation network for each pair of input source domain and target domain images. Then, the network loss is calculated, which is recorded as , , using the initial learning rate of the Adam optimizer , and use a batch size of 1; Step S4: training for 100 rounds in total according to the process of step S3, and obtaining an underwater image enhancement quality evaluation network model based on quality perception domain adaptation; Step S5: Use the underwater image enhancement quality evaluation network model based on quality-aware domain adaptation to test each second enhanced image in the test set. The model outputs a corresponding predicted quality score for each second enhanced image in the test set, and compares it with the true label of the test set to calculate the evaluation index, thereby evaluating the model performance.

[0020] Continue to see Figure 1 The process of preprocessing a first enhanced image or a second enhanced image in step S1 is as follows: first, the image size is scaled to ; Secondly, the scaled image is normalized, and the pixel values ​​of all pixels in the R channel of the scaled image are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.225; The process of preprocessing a label in step S1 is: using the existing technology to scale the label size to Since the input of a deep neural network usually has a fixed form, the first enhanced image and the second enhanced image need to be preprocessed. Here, the normalization process is only performed on the first enhanced image and the second enhanced image, while the labels corresponding to the first enhanced image and the second enhanced image are not normalized.

[0021] Continue to see Figure 2The visual feature extraction network adopts a four-layer ResNet50 backbone network. The four layers in the ResNet50 backbone network are connected in sequence. The input end of the first layer of the ResNet50 backbone network receives input images from the source domain and the target domain at the same time. In order to improve the image processing effect, the input images of the source domain and the target domain are preprocessed before entering the first layer of the ResNet50 backbone network. The output end of the fourth layer of the ResNet50 backbone network is used as the output end of the visual feature extraction network. The quality regression network is mainly composed of two fully connected layers, a Dropout layer and a ReLU activation layer. The feature map at the output end of the visual feature extraction network is used as the input of the first fully connected layer. After being processed by the first fully connected layer, it passes through the first ReLU activation layer and the Dropout layer in sequence, and finally the predicted quality scores of the input images of the source domain and the target domain are obtained through the second fully connected layer. Represents the size of the feature map output by the visual feature extraction network, , and They represent the feature maps of the source domain and the target domain output by the visual feature extraction network, and their sizes are The ResNet50 backbone network is an existing structural framework, and its network structure has been made public. For example, the ResNet50 backbone network is recorded in the reference K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.

[0022] See also Figure 3 , the text-guided network includes text templates and text encoder ,The text template consists of several phrases describing the guidance direction. Since the guidance direction of this method is a quality feature, the text template is set to: "distortion types + image". The text template is concatenated by multiple words related to the guidance direction and used as the input of the text encoder. The text encoder uses the pre-trained text encoder in the CLIP model. The output of the text encoder generates a feature map, which is recorded as , and then To process, first, and The cosine similarity is calculated, and then the similarity is processed by sigmoid to make the data smoother and limited to (0, 1) as the weight W of the text prompt, and then W is compared with Multiply to get the text input across the attention network , where the feature map and The size of the weight matrix W is (batch_size, 1).

[0023] See also Figure 4 ,The cross-attention network is a multimodal fusion process. In this method, it receives two inputs: text input and image information input. The image information comes from the feature map output by the visual feature extraction network. , while the text information comes from the feature map output by the text-guided network , in the cross-attention network, is used as the key (Key, K) and value (Value, V) in the attention module, and Used as the query (Query, Q) in the attention module, these three inputs enter the cross-attention network together to filter out feature information that is more relevant to the text feature information from the image feature information.

[0024] Continue to see Figure 2 , the adversarial domain adaptation network is the process of domain discrimination of filtered image feature information. Its input is the output F_cross_after of the cross-attention network, which is input into the discriminator. First, it passes through the GradReverse layer, which applies the gradient reversal operation so that the gradient sign of the input feature is flipped during back propagation. This operation accepts feature and alpha parameters. Subsequently, the gradient-reversed feature is mapped to 1 dimension through a three-layer fully connected neural network to obtain the output value y, which is compressed to the range of (0, 1) through the Sigmoid function. Finally, the target label l_target is created. l_target is a mixture of source domain data and target domain data, marked as 0 and 1 respectively. The binary cross entropy loss function is used to calculate the loss value between the predicted output y and the target label l_target. This process enables the network to better adapt to the data distribution of the source domain and the target domain through adversarial training, thereby improving the domain adaptation performance of the model. Among them, feature is the output F_cross_after of the cross-attention network, alpha is the gradient reversal strength, and the size of y is the same as that of l_target, which is (batch_size, 1).

[0025] Continue to see Figure 1In step S1, the input images selected by the source domain and the target domain keep the original images and the enhancement algorithms different. The process of preprocessing each pair of input images of the source domain and the target domain is as follows: First, the size of the original image is scaled to ; Secondly, the scaled image is normalized, and the pixel values ​​of all pixels in the R channel of the scaled image are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.225.

[0026] Continue to see Figure 1 In step S2, the source domain and target domain images are first input into the visual feature extraction network at the same time, and then pass through the first layer, second layer, third layer, and fourth layer of the ResNet50 network, and then the output is obtained. and ,Will and Splicing as ,Right now , then, the average pooling layer of the pre-trained ResNet50 network is used to Processing, specifically, the input tensor It is sent to the average pooling layer (avgpool) in the ResNet50 network, which averages the spatial dimensions of the feature map to generate a fixed-size feature vector. Then, the tensor after average pooling is Perform a flattening operation, which is implemented by view(x.size(0), -1). The shape is adjusted to a two-dimensional tensor, where the first dimension remains as batch_size and the second dimension is adjusted to the total number of all features to obtain the feature map , the visual feature extraction network finally obtains and Two outputs, of size and .

[0027] Continue to see Figure 1 , in step S2, Figure 2 The upper part is the quality regression network, which first predicts the quality scores of the source domain image and the target domain image; then, the predicted score of the source domain image is compared with its true label, and the accuracy of the prediction is evaluated by calculating the root mean square error (RMSE). Finally, based on the calculated RMSE loss, the quality regression network is trained to optimize its parameters to improve the accuracy of the prediction. In this process, the output of the visual feature extraction network is used as the As the input of the quality-aware domain adaptation module.

[0028] Continue to see Figure 1 , in step 2, if Figure 3 , the text template T is set to "distortion type image" (where distortion type includes: color deviation, color distortion, vague, noise, unevenlight, loss of contrast, color deviation), and they are input into the text encoder pre-trained by clip to obtain 9 text features. After splicing them, they are expanded from one-dimensional tensors to two-dimensional tensors, and then converted into floating-point form to obtain the output feature map ,Will With visual features After calculating the cosine similarity, perform a sigmoid operation to obtain the input weight W of the text feature map. Finally, compare W with Multiply to get As text input to the cross-attention network.

[0029] Continue to see Figure 1 In step 2, and As the input of the cross-attention network, the number of channels of the cross-attention network is 2048 and the embedding dimension is 77, such as Figure 4 , the cross-attention network first transforms the source feature tensor by linear transformation Convert to query tensor Q, and convert the context feature tensor Convert to key tensor K and value tensor V. Then, calculate the dot product of query tensor Q and key tensor K to obtain attention weights, and apply a scaling factor, which is the inverse of the square root of the embedding dimension. If there is a padding mask, a very large negative value is applied to the attention weights of the corresponding positions to ignore these positions. Then, the attention weights are normalized using the softmax function and the dropout operation is applied to them to prevent overfitting. Finally, the normalized attention weights are used to perform weighted summation on the value tensor V to generate the final output tensor F_cross_after.

[0030] Continue to see Figure 1In step 2, the input of the adversarial domain adaptation network is F_cross_after, which is first processed: the tensor F_cross_after after the cross attention operation Apply two-dimensional adaptive average pooling (adaptive_avg_pool2d) and adjust its spatial dimension to (1, 1), that is, compress the size of each feature map into a single value. This step effectively averages all spatial positions of each feature map to obtain a fixed-size output. Subsequently, the average pooled tensor F_cross_after is reshaped and its spatial dimension is flattened so that it can be converted into a two-dimensional tensor with a size of (batch_size, 2048); the processed tensor is input into an adversarial network with a feature dimension of 2048. The network contains a sequence of fully connected layers ad_layer3 for processing hidden features. The sequence includes a fully connected layer that maps the input tensor (feature dimension is hidden_size) to 512 dimensions, followed by a batch normalization layer (BatchNorm1d), then a ReLU activation function, and finally a fully connected layer that maps the 512-dimensional features to 1 dimension. The output of the network passes through a Sigmoid activation function to limit its value range to [0, 1], in the forward propagation process, the input feature tensor xfeature undergoes a gradient reversal operation GradReverse.apply(xfeature, alpha), where alpha is set to 0.5. This operation is used to adjust the gradient during training to achieve adversarial training. After that, the processed features are passed through ad_layer3 for further processing, and the output probability is obtained through the Sigmoid function. The network calculates the binary cross entropy loss (BCELoss) between the output probability y and the target label dc_target. The target label dc_target is a tensor containing two categories of labels, where the first source_size samples are marked as 0 and the last target_size samples are marked as 1. The final loss value is used to optimize the network so that it can effectively distinguish the features of the source domain and the target domain samples.

[0031] Continue to see Figure 1 In order to further verify the feasibility and effectiveness of the method of the present invention, the method of the present invention was tested: In the experiment, the existing underwater image enhancement quality evaluation dataset, namely the SAUD dataset, was directly selected for training and testing. First, the SAUD dataset was expanded and named SAUD+. The SAUD dataset contains 100 original underwater images and 1000 enhanced images obtained by 10 different underwater image enhancement algorithms. In order to make the types of enhancement methods of the source domain and target domain datasets more diverse, the original images of the SAUD dataset were increased to 200 in the experiment, and 2 additional enhancement algorithms were added. Finally, an underwater image enhancement quality evaluation dataset containing 2400 images was obtained. In order to maintain the differences in content and enhancement algorithms between the source domain and the target domain images, 600 images obtained by processing 100 original images in the SAUD+ dataset with 6 enhancement algorithms were used as the source domain, and 600 images obtained by processing the remaining 100 original images with another 6 enhancement algorithms were used as the target domain. In this process, the source domain and the target domain were trained together, but only the source domain was supervised; and the target domain without labels was used as the test set for performance evaluation. Four widely used evaluation indicators are used, including: Spearman rank correlation coefficient (SRCC), Pearson linear correlation coefficient (PLCC), Kendall rank correlation coefficient (KRCC) and root mean square error (RMSE). Specifically, SRCC is used to measure the monotonic relationship between the predicted value and the true value to evaluate the consistency of variable ranking. PLCC is used to quantify the prediction accuracy and analyze the linear correlation between the predicted value and the true value. KRCC is used to evaluate the correlation between two ordinal variables, reflecting the accuracy of the model in processing ordinal information. Finally, RMSE is used to measure the prediction error of the model. By calculating the mean of the squared difference between the predicted value and the true value and taking the square root, it provides a quantitative standard for the prediction accuracy of the model. These indicators are used together to evaluate the quality perception ability, linear matching degree, ranking consistency and actual prediction error of the model, see the table below: Table 1 Quantitative comparison of the evaluation indicators of the proposed method and the existing methods on the SAUD test set ; Note: The source domain images are generated by 6 enhancement algorithms in SAUD+, and the target domain images are generated by another 6 enhancement algorithms; Table 1 above shows a quantitative comparison of the evaluation indicators of the method of the present invention and common image quality evaluation methods on the SAUD test set. It can be seen that the data of the method of the present invention, namely Ours, has better evaluation indicators.

[0032] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0033] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A quality assessment method for underwater image enhancement based on quality perception domain adaptation, characterized in that: The following steps are involved: Step S1, obtaining a plurality of original underwater images from the outside, then performing enhancement processing on each of the original underwater images and dividing them into a source domain dataset and a target domain dataset, and preprocessing the source domain dataset to obtain a training set, and preprocessing the target domain dataset to obtain a test set, wherein the training set includes a plurality of first enhanced images and labels corresponding to each of the first enhanced images, and the test set includes a plurality of second enhanced images; Step S2, using a deep learning framework to build an underwater image enhancement quality evaluation network, the underwater image enhancement quality evaluation network includes a visual feature extraction network, a quality regression network, a text guidance network, a cross-attention network and an adversarial domain adaptation network, the visual feature extraction network is used to extract features from the input image to obtain a feature map and feature map , the quality regression network is used to The text-guided network is used to generate a feature map based on the external input text template. , and based on the cosine similarity calculation method according to the feature map and the characteristic graph Get output text , the cross-attention network is used to calculate the feature map and the output text An output tensor F_cross_after is obtained, and the adversarial domain adaptation network is used to obtain a binary cross entropy loss according to the output tensor F_cross_after and a pre-acquired target label dc_target; Step S3, using the training set to train the underwater image enhancement quality evaluation network, calculating the root mean square error between the predicted quality score and the label corresponding to each of the first enhanced images, and combining the binary cross entropy loss to obtain the network loss, and adjusting and optimizing the underwater image enhancement quality evaluation network according to the network loss; Step S4, repeating step S3 for multiple rounds to obtain an underwater image enhancement quality evaluation network model; Step S5: Use the underwater image enhancement quality evaluation network model to test each of the second enhanced images in the test set to obtain the corresponding predicted quality score as the underwater image enhancement quality evaluation result.

2. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1 is characterized in that: The preprocessing method adopted in step S1 is: Each of the first enhanced images and each of the second enhanced images is sequentially scaled and normalized, so that the size of each of the first enhanced images and each of the second enhanced images after preprocessing is ; as well as Scaling operation is performed on each of the labels so that the size of each of the labels after preprocessing is .

3. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 2 is characterized in that: The normalization operation includes: For each of the first enhanced images and each of the second enhanced images after the scaling operation, the pixel values ​​of all pixels in the R channel of each of the first enhanced images and each of the second enhanced images are normalized to a mean of 0.485 and a variance of 0.229, the pixel values ​​of all pixels in the G channel are normalized to a mean of 0.456 and a variance of 0.224, and the pixel values ​​of all pixels in the B channel are normalized to a mean of 0.406 and a variance of 0.

225.

4. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1 is characterized in that: In step S2, the visual feature extraction network includes four layers of ResNet50 backbone networks connected in sequence, receiving each of the first enhanced images or each of the second enhanced images as input images through the first layer of ResNet50 backbone network, and outputting the feature map through the fourth layer of ResNet50 backbone network. and the characteristic graph .

5. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1 is characterized in that: In step S2, the quality regression network includes a first fully connected layer, a first ReLU activation layer, a Dropout layer, and a second fully connected layer connected in sequence, and the feature map is received through the first fully connected layer. The predicted quality score is obtained by sequentially processing the first ReLU activation layer, the Dropout layer and the second fully connected layer.

6. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1 is characterized in that: The text template includes a plurality of phrases describing the guiding direction. In step S2, the text guiding network includes a text encoder and a computing block connected to each other. The text encoder is used to receive the text template input from the outside and obtain the feature map according to the encoding of each phrase in the text template. , the calculation block is used to convert the feature map As a feature map With the characteristic diagram The cosine similarity is calculated, and then the cosine similarity is processed by sigmoid to obtain the weight W, and then the weight W is compared with the feature map Multiply to get the output text .

7. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 6 is characterized in that: In step S2, the binary cross entropy loss is obtained by the following calculation formula: ； in, represents the binary cross entropy loss; s represents the source domain; t represents the target domain; represents the number of distorted images from the source domain; Indicates that it has been processed by the adversarial domain adaptation network; Represents processing across attention networks; Indicates that it has been processed by the visual feature extraction network; representing the input image of the source domain; represents said weight; It means that it has been processed by the text-guided network; represents the output text; represents the number of distorted images from the target domain; The label of the input image; The input image represents the target domain.

8. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1 is characterized in that: In step S2, the cross-attention network transforms the feature map Converted into a query tensor Q, the output text Convert into a key tensor K and a value tensor V, calculate the dot product of the query tensor Q and the key tensor K to obtain the attention weight, then normalize the attention weight using the softmax function, apply the dropout operation to it, and use the normalized attention weight to perform weighted summation on the value tensor V to obtain the output tensor F_cross_after.

9. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1, characterized in that: In step S2, the adversarial domain adaptation network includes a gradient reversal layer, a third fully connected layer, a batch normalization layer, a second ReLU activation layer and a fourth fully connected layer connected in sequence, and two-dimensional adaptive average pooling is applied to the output tensor F_cross_after through the gradient reversal layer, and the output tensor F_cross_after after average pooling is sequentially passed through the third fully connected layer, the batch normalization layer, the second ReLU activation layer and the fourth fully connected layer to obtain an output probability y, and the binary cross entropy loss between the output probability y and the target label dc_target is calculated.

10. The underwater image enhancement quality assessment method based on quality perception domain adaptation according to claim 1, characterized in that: In step S3, the root mean square error is obtained by the following calculation formula: ； in, represents the root mean square error; represents the number of distorted images from the source domain; Indicates that it has been processed by the quality regression network; represents the predicted quality score corresponding to the first enhanced image; Indicates the label corresponding to the first enhanced image; The label of the input image.

Images