Underwater image quality evaluation system and method based on brightness and chrominance combined learning

Abstract

The invention relates to an underwater image quality evaluation system and method based on brightness and chrominance combined learning. According to the method, an underwater original image and an optimal enhanced image are used as labels to train two generative adversarial networks, and two pseudo reference images are obtained; converting the obtained two pseudo reference images and an enhanced image to be evaluated from an RGB format to a YCbCr format; and a channel Y and a chrominance channel CbCr are lightened, and image quality evaluation is performed on the enhanced image to be evaluated from two aspects of brightness and chrominance, so that brightness quality representation and chrominance quality representation can be obtained. A multi-scale feature mapping module and a parallel attention module are also applied in the invention. The method is used for extracting mapping-based quality feature representations and focusing various distortions in an image structure space, and capturing brightness features of different distributions. Compared with the prior art, the method has the advantages that the accuracy of underwater enhanced image quality evaluation is improved, and the underwater enhanced image quality evaluation is simple, convenient, comprehensive and accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an underwater image quality evaluation system and method for joint learning of brightness and chromaticity. Background Art

[0002] So far, underwater images usually have problems such as low contrast, color distortion and blur, which not only reduces the visual effect but also affects the practical application value of the image. Existing underwater image quality evaluation indicators perform poorly in evaluating the quality of enhanced images. However, existing underwater image enhancement algorithms still have the following problems:

[0003] 1. A large number of underwater image enhancement methods aim to improve the visual quality of underwater images. However, the main difficulty currently faced is that underwater images are severely degraded and true undistorted reference images are not available. The enhancement algorithms that perform well in existing synthetic datasets have poor generalization capabilities and perform poorly on real underwater images, and may suffer from over-enhancement, under-enhancement, and structural loss.

[0004] 2. Existing evaluation indicators such as UCIQE and UIQM have higher scores when evaluating enhanced images than the best enhanced images, and appear unreliable when evaluating the quality of enhanced images, which is inconsistent with human subjective visual perception evaluation.

[0005] In the above situation, the root of the problem lies in the global features of the image extracted by these indicators, which reflect the color cast and blur of the image. For underwater original images, the clearer the image, the brighter the color and the higher the contrast, the higher the visual quality. The enhancement algorithm will introduce additional distortion. The existing underwater image quality evaluation indicators only focus on the existing distortion (such as color cast and blur), ignoring the problems such as over-enhancement and structural loss that may occur after the enhancement process, which seriously affects the quality of the enhanced image.

[0006] The invention patent with the publication number CN111354048A discloses a quality evaluation method for camera-acquired images. The method extracts the brightness and chromaticity features, image noise features, image structural features, contrast features, and statistical features of the image normalization coefficient, and then calculates the difference between the image and its sparse representation, and obtains the mean, variance, kurtosis, skewness, and information entropy of the representation residual; a support vector regression method is used on the training set to learn a mapping model from image features to image quality for predicting image quality. The invention does not require reference to the original image and can achieve high prediction performance. However, it also ignores the problems of over-enhancement and structural loss that may occur after the enhancement process when evaluating the quality of the enhanced underwater image, resulting in poor performance when evaluating the quality of the enhanced underwater image. Summary of the invention

[0007] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an underwater image quality evaluation system and method for joint learning of brightness and chromaticity.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, there is provided an underwater image quality assessment system for luminance and chrominance joint learning, the system comprising a bidirectional reference generation module, a chrominance distortion assessment module, a luminance distortion assessment module, and a chrominance and luminance joint assessment module;

[0010] The bidirectional reference generation module consists of two generative adversarial networks for generating unenhanced and optimally enhanced pseudo reference images of the image to be evaluated;

[0011] The chroma distortion assessment module is composed of a convolutional layer and a three-layer multi-scale feature mapping module cascaded together to evaluate the module for detecting different types of distortion in the chroma domain and obtaining a chroma quality representation;

[0012] The brightness distortion evaluation module is composed of a parallel spatial attention module and a three-layer multi-scale feature mapping module cascaded together to evaluate the module for detecting different types of distortion in the brightness domain and obtaining brightness quality representation;

[0013] The chroma-luminance joint evaluation module is composed of three convolutional layers and dense blocks and one fully connected layer in cascade, and is used to jointly obtain the image quality representation of the enhanced image to be evaluated with the chroma quality representation and the luminance quality representation.

[0014] As a preferred technical solution, the multi-scale feature mapping module structure is specifically as follows:

[0015] Two parallel dense blocks are cascaded with the conversion layer to output two feature vectors, which are then connected in series with the convolution layer, the normalization layer, and the activation function Tanh layer to output the quality representation.

[0016] As a preferred technical solution, the dense block is composed of n-level batch normalization layers, ReLU layers, and convolutional layers in cascade, where n≥3.

[0017] As a preferred technical solution, the parallel spatial attention module consists of three parallel convolutions, with the sizes of parallel convolution kernels being 3×3, 5×5 and 7×7, and the expansion rates being 1, 2 and 3, respectively.

[0018] According to another aspect of the present invention, a method for underwater image quality assessment using joint luminance and chrominance learning is provided, wherein the method is applied to an underwater image quality assessment system using joint luminance and chrominance learning as described above, and the method steps include:

[0019] S1, collecting underwater original images, enhancing them using an enhancement algorithm, and selecting the image with the best enhancement effect as the best enhanced image;

[0020] S2, using the underwater original image and the best enhanced image as labels to train two generative adversarial networks, and obtaining two pseudo reference images, including the original image reference image and the best enhanced reference image;

[0021] S3, converting the two pseudo reference images obtained in S2 and the enhanced image to be evaluated from RGB format to YCbCr format;

[0022] S4, separating the brightness channel Y and the chrominance channel CbCr of the two pseudo reference images and the enhanced image to be evaluated, and using the constructed chrominance distortion evaluation module and the brightness distortion evaluation module to evaluate the image quality of the enhanced image to be evaluated from the brightness and chrominance aspects, respectively, to obtain the brightness quality representation and the chrominance quality representation;

[0023] S5. The image quality of the enhanced image to be evaluated is jointly represented by the brightness quality representation and the chrominance quality representation.

[0024] As a preferred technical solution, the generator G of the two generative adversarial networks in S2 adopts the DenseUNet network, and the discriminator D adopts the PatchGAN network.

[0025] As a preferred technical solution, the generator's loss function adopts L 2 and L vhg The perceptual loss function is:

[0026]

[0027] in, It is the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset;

[0028] The loss function of the discriminator adopts the WGAN-GP algorithm:

[0029]

[0030] in, is the loss function, λ is the penalty coefficient, I g Generate images for the generator, I k be the original image or the best enhanced image; is the gradient norm of the output of the discriminator D with respect to its input, is the expected value.

[0031] As a preferred technical solution, the process of evaluating the image quality of the enhanced image to be evaluated from the aspect of chromaticity in S4 is as follows:

[0032] The CbCr channels of the enhanced image to be evaluated and its unenhanced reference image are subtracted, and the CbCr channels of the unenhanced reference and the optimal enhanced reference are subtracted. The two values ​​are used as the input of the multi-scale feature mapping module. At this time, the output of each of the three levels in the multi-scale feature mapping module is the chromaticity quality representation of the image.

[0033] As a preferred technical solution, the process of evaluating the image quality of the enhanced image to be evaluated from the aspect of brightness in S4 is as follows:

[0034] The Y channel of the enhanced image to be evaluated and its corresponding original image reference image is subtracted, and the Y channel of the original image reference image and the Y channel of the best enhanced reference image are subtracted, and the two values ​​are used as the input of the parallel attention module; and the output of the parallel attention module is input to the multi-scale feature mapping module. At this time, the output of each level of the three levels in the multi-scale feature mapping module is the brightness quality representation of the image.

[0035] As a preferred technical solution, the specific steps of jointly indicating the image quality of the enhanced image to be evaluated in S5 are:

[0036] The chromaticity feature representation and brightness feature representation obtained in S4 are cascaded and input into the dense block respectively, and then the final quality score of the image is obtained by calculation. The loss function of the entire network is for:

[0037]

[0038] in, is the calculated image quality, is the average subjective score of the subjective quality label of the image.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] 1. The present invention uses an underwater original image and an optimal enhanced image as labels to train two generative adversarial networks to obtain two pseudo reference images; the obtained two pseudo reference images and the enhanced image to be evaluated are converted from RGB format to YCbCr format; and the bright channel Y and chromaticity channel CbCr of the two pseudo reference images and the enhanced image to be evaluated are separated, and the image quality of the enhanced image to be evaluated is evaluated from the brightness and chromaticity aspects, respectively, to obtain a brightness quality representation and a chromaticity quality representation; finally, the image quality of the enhanced image to be evaluated is jointly represented by the brightness quality representation and the chromaticity quality representation, so that the obtained image quality score is more consistent with the subjective perception quality, thereby improving the accuracy of underwater enhanced image quality evaluation.

[0041] 2. A multi-scale feature mapping module is designed in the present invention. The structure of the multi-scale feature mapping module is specifically a cascade of two parallel dense blocks and a conversion layer, and two feature vectors are output. The two feature vectors are connected in series by a convolution layer, a normalization layer, and a tanh layer of an activation function, and the output is a quality representation. In the multi-scale feature mapping module, the input is extracted by a dense block and a conversion layer to obtain two feature vectors, and the results obtained after the two feature vectors are convoluted, the instance normalization layer, and the activation function tanh are used to calculate the cosine function coefficient, and the coefficient is multiplied by the feature vector of the enhanced image to be evaluated, and the mapping-based quality feature representation is extracted, so that the underwater enhanced image quality assessment process is simplified.

[0042] 3. A parallel spatial attention module is designed in the present invention. The parallel spatial attention module consists of three parallel convolutions. The sizes of parallel convolution kernels are 3×3, 5×5 and 7×7, and the expansion rates are 1, 2 and 3. It is used to focus on various distortions in the image structure space and capture brightness features of different distributions, making the underwater enhanced image quality assessment more comprehensive and specific.

[0043] 4. The present invention repeats the process of extracting brightness features and chromaticity features three times at different scales to extract the chromaticity characteristics of the image from the global to the local, making the underwater enhanced image quality assessment more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of the steps of an underwater image quality assessment method for joint learning of brightness and chromaticity in the present invention;

[0045] Figure 2 A schematic diagram of the architecture of an underwater image quality assessment system for joint learning of brightness and chrominance in the present invention;

[0046] Figure 3 Schematic diagram of the multi-scale feature mapping module architecture in the present invention;

[0047] Figure 4 Schematic diagram of the parallel attention module architecture in the present invention;

[0048] Figure 5 It is a schematic diagram of the module architecture of the dense block in the present invention;

[0049] Figure 6 A schematic diagram showing the comparison of different methods for underwater image quality evaluation in visual perception metrics in the embodiments;

[0050] Figure 7 It is a schematic diagram for comparing the structural loss metric using different methods for underwater image quality evaluation in the embodiments. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] Example

[0053] In this embodiment, a method for underwater image quality assessment using joint learning of brightness and chrominance is applied. This method represents the enhancement degree and direction of the image as a feature vector by mapping, and constructs an optimal enhancement direction, maps other feature vectors to this direction, and thus obtains the corresponding image quality representation. The method steps are as follows: Figure 1 shown.

[0054] In this embodiment, it is assumed that all underwater original images are in a domain R, and their ideal undistorted images are in another domain RI. The enhanced images obtained by various underwater image enhancement methods can also be represented as domain E. Here, it can be obtained:

[0055]

[0056] in is a feature vector, which not only represents the distance and direction between the R domain and the E domain, but also represents the degree and direction of image enhancement; e is the feature in the E domain; f r is a feature in the R domain. Assume

[0057]

[0058] Among them, f i is the feature in the RI domain.

[0059] In this embodiment, since all underwater distortions in the image have been completely removed in the ideal distortion-free image domain RI, these images should be given the highest score according to the subjective experimental standard for underwater image quality assessment. Therefore, in this embodiment, the feature vector As the quality feature of the full score. For the enhanced image in any other domain, its feature vector Map to direction, a unified mapping-based image quality representation is formed:

[0060]

[0061] represents the length of the enhanced image feature vector, It is the cosine of the angle between the direction of the vector and the direction of the eigenvector in the distortion-free domain.

[0062] In this embodiment, considering that the ideal situation of no distortion is unattainable, the enhanced image with the best enhancement effect among all enhancement algorithms for each original underwater image is selected to form a domain. The domain composed of the best enhanced images should be the domain closest to the ideal undistorted image domain, so it is used as the optimal replacement. The enhancement direction of the best enhanced image domain can be used Indicates that this direction is closest to the direction of the ideal undistorted image domain Therefore, in this embodiment, the unified mapping-based image quality representation is actually:

[0063]

[0064] In this embodiment, in order to obtain the information of the best enhanced image and the underwater original image, two types of corresponding pseudo reference images are generated by Generative Adversarial Networks (GAN) to replace the real labels. The bidirectional reference generation module uses the original image and the best enhanced image as labels to train two networks. Figure 2 The bidirectional reference generation network in is composed of two generative adversarial networks. In deep learning, a label refers to the target output corresponding to the input data in the dataset, that is, the result that the model wants to predict.

[0065] In this embodiment, a generative adversarial network is used to generate an unenhanced image with consistent underwater distortion and a reference image with optimal enhanced feature distribution. In the generative adversarial network, the generator G uses a DenseUNet network and the discriminator D uses a PatchGAN network. Input a real distorted image and get:

[0066] I g =G(I d );

[0067] Among them, I g Generate images for the generator; I d is the real distorted image.

[0068] In this embodiment, when training the GAN network, the generator G inputs the distorted image I d , the purpose is to generate a label I l (original image or best enhanced image) Image I with consistent feature distribution g , the discriminator D is used to determine I g and I l Whether the distribution of image content features is consistent. As the joint loss function gradually decreases, Ig and Il will become increasingly difficult to distinguish.

[0069] In this embodiment, the loss function adopts WGAN-GP:

[0070]

[0071] in, is the loss function, λ is the penalty coefficient, I g Generate images for the generator, I l is the label (original image or best enhanced image). The gradient norm E of the output of the discriminator D relative to its input is the expected value, that is, the average value or expectation of the random variable is calculated. In deep learning and machine learning, this symbol is widely used to define loss functions or objective functions to calculate the expected loss or output of the model over the entire data distribution. D(I g ) is the I generated by the discriminator g The judgment outputs a value indicating the probability that the input data is "true" or "false". k (I l ) is the discriminator's I for the true label l Judge, output a value, indicating the probability that the input data is "true" or "false". g With label I l (original image or best enhanced image) has consistent feature distribution. In this embodiment, λ=10. The loss function of the generator adopts L 2 and L vgg The perceptual loss function is:

[0072]

[0073] in, It is the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset.

[0074] In this embodiment, the input image to be evaluated and the two pseudo reference images generated in the previous step are converted from RGB format to YCbCr format, and the bright channel Y and the chrominance channel CbCr are separated. The image quality QI is expressed as:

[0075] Q I =λ l Q l +λ c Q c ;

[0076] Q I Q is the brightness quality. c is the chromaticity quality representation, λ l and λ c They are the coefficients for brightness quality representation and chrominance quality representation respectively.

[0077] In this embodiment, the quality evaluation is performed from two aspects: brightness and chromaticity. First, in terms of chromaticity, Figure 3 The difference between the CbCr channel features of the enhanced image to be evaluated and the unenhanced pseudo reference image is sent to the dense block and the conversion layer to obtain the feature vector and Then, the features of these two vectors are passed through convolution, instance normalization layer and activation function tanh, and the cosine function is calculated to get the coefficients Then the coefficient and Multiply them together to get the chromaticity feature of the image quality at the first scale. Finally, the feature vector and Will be used as the next level input feature f b '-f r ' and f e '-f r '. This module is repeated three times on feature maps of different scales to extract the chromaticity characteristics of the image from global to local.

[0078] In this embodiment, in terms of brightness, brightness channels of three images are first separated, and the three images include an enhanced image to be evaluated, an unenhanced reference image, and an optimal enhanced reference image.

[0079] In this embodiment, the Y channel of the image to be evaluated and its unenhanced reference image are subtracted, and the Y channel of the unenhanced reference image and the Y channel of the best enhanced reference image are subtracted, and the two are sent as input to the parallel spatial attention modules of different scales, such as Figure 3 As shown in the figure, the sizes of parallel convolution kernels are set to 3×3, 5×5 and 7×7, and the dilation rates are set to 1, 2 and 3 respectively. The final output can obtain representations of structural distortions at multiple scales. The reason for this is that smaller convolution kernel sizes and dilation rates are used to capture local detail texture distortions of the image; while larger convolution kernel sizes and dilation rates can be used to capture perceptual distortions such as global blur. With this feature pyramid structure, the final output can capture structural distortion representations at different scales. These distorted representations are then sent to the above Figure 3 The network in is performed three times in the same network structure as used for chromaticity distortion evaluation.

[0080] In this embodiment, Figure 4 As shown, the chromaticity feature representation and brightness feature representation obtained at each scale of the multi-scale feature mapping module with different depths are cascaded and input into the dense block respectively. The dense block structure is as follows Figure 5 As shown, the final quality score of the image is obtained by calculation. The loss function of the entire network is It can be expressed as:

[0081]

[0082] in, is the image quality calculated by the network proposed in this invention, and is the average subjective score of the subjective quality labels of the image.

[0083] Among them, the Mean Opinion Score (MOS) is usually used to evaluate audio, video or image quality.

[0084] In this embodiment, the quality of underwater images is evaluated by joint learning based on brightness and chromaticity. In this example, 200 epochs are trained on NVIDIA 2080Super. The batch size of the training is 8, the initial learning rate is set to 0.0002, and the learning rate is reduced to 0.0001 after 100 cycles of training. The optimizer of the network is Adam optimizer. Data enhancement steps are added during training, and random horizontal flipping, vertical flipping, and rotation operations are performed on the training images.

[0085] In this embodiment, two GAN networks are first trained, with the original image and the image with the highest quality score in each group of images as labels, and the entire group of images as input for training.

[0086] In this embodiment, the network parameters trained in the first step are fixed. The generated image is converted from RGB color space to YCbCr color space as the input of the subsequent feature extraction and quality assessment network. The network is trained in an end-to-end manner.

[0087] In this embodiment, indicators such as Pearson linear correlation coefficient (PLCC), Spearman rank-order correlation coefficient (SROCC), Kendall rank-order correlation coefficient (KROCC), and root mean square error (RMSE) are used to measure the performance of the quality assessment algorithm. The higher the PLCC, SROCC, and KROCC values ​​of an algorithm and the lower the RMSE value, the better the performance of the method, and the image quality score obtained is more consistent with the subjective perceived quality.

[0088] In this embodiment, various other quality evaluation methods and the present invention perform well on different enhanced images on the SOTA test set, and it is known that the quality evaluation index of general land images is not suitable for underwater image quality evaluation, such as NIQE, which performs poorly on various underwater images; the quality evaluation method based on physics is not effective in evaluating various underwater images. For example, the underwater image quality measurement method (UIQM) and the underwater color image quality evaluation (UCIQE) are more sensitive to the vividness of the overall color of the image. When evaluating the image enhanced by the HP algorithm, the performance of various indicators is acceptable, but the performance is poor when evaluating other types of enhanced images, original images and overall images. The algorithm based on deep learning has the best comprehensive performance on the entire data set.

[0089] In this embodiment, this method is used together with other underwater image quality evaluation algorithms such as UIQM and UCIQE to compare the quality of underwater images enhanced by some underwater enhancement algorithms (Two-step, ACPAB, Water-Net, Fusion-18, HP, Dive+). Figure 6 shown.

[0090] In this embodiment, first, the test set selects two groups of underwater images of different scenes, enhances them using these enhancement algorithms, and uses this method to sort several enhanced images. After observing the sorting results, human eyes find that the scores are basically consistent with the subjective perception of quality by human eyes.

[0091] In this embodiment, Figure 6 As shown, the over-enhanced image has the lowest quality and is ranked first, and is sorted from left to right according to the decreasing degree of over-enhancement, and the under-enhanced image is ranked after it.

[0092] In this embodiment, the scores calculated by UCIQE and UIQM algorithms for the first three underwater images enhanced by different over-enhancement algorithms are 0.6930, 0.7021, 0.6023 and 3.060, 2.529, 3.018 respectively. Both algorithms give higher scores to the over-enhanced images than to the images with the best enhancement effect. This is obviously an unreasonable result, which is contrary to the subjective perception of the human eye. It can be seen that this method performs better in evaluating the quality of underwater images based on the metric of visual perception.

[0093] In this embodiment, Figure 7As shown, for the structural loss, especially the second image, some ghost images are produced on the edge. It can be seen from the rightmost image that the scores given by the UIQM and UCIQE algorithms are also higher than the score of the image with the best enhancement effect, which is unreasonable.

[0094] Through the above analysis, it is obvious that the algorithm proposed in the present invention performs better in underwater image quality evaluation based on the metric of structural loss.

[0095] Experimental results show that the algorithm has high accuracy in underwater enhanced image quality assessment and has achieved excellent performance in various indicators.

[0096] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A system for underwater image quality assessment based on joint learning of brightness and chrominance, characterized in that: The system includes a bidirectional reference generation module, a chromaticity distortion evaluation module, a luminance distortion evaluation module, and a chromaticity and luminance joint evaluation module; The bidirectional reference generation module consists of two generative adversarial networks for generating unenhanced and optimally enhanced pseudo reference images of the image to be evaluated; The chroma distortion assessment module is composed of a convolutional layer and a three-layer multi-scale feature mapping module cascaded together to evaluate the module for detecting different types of distortion in the chroma domain and obtaining a chroma quality representation; The brightness distortion evaluation module is composed of a parallel spatial attention module and a three-layer multi-scale feature mapping module cascaded together to evaluate the module for detecting different types of distortion in the brightness domain and obtaining brightness quality representation; The chroma-luminance joint evaluation module is composed of three convolutional layers and dense blocks and one fully connected layer in cascade, and is used to jointly obtain the image quality representation of the enhanced image to be evaluated with the chroma quality representation and the luminance quality representation.

2. According to claim 1, the underwater image quality assessment system of luminance and chrominance joint learning is characterized in that: The multi-scale feature mapping module structure is specifically as follows: Two parallel dense blocks are cascaded with the conversion layer to output two feature vectors, which are then connected in series with the convolution layer, the normalization layer, and the activation function Tanh layer to output a quality representation.

3. The underwater image quality assessment system of luminance and chrominance joint learning according to claim 2 is characterized in that: The dense block is composed of n-level batch normalization layers, ReLU layers, and convolutional layers in cascade, where n≥3.

4. The underwater image quality assessment system of luminance and chrominance joint learning according to claim 1 is characterized in that: The parallel spatial attention module consists of three parallel convolutions, the sizes of the parallel convolution kernels are 3×3, 5×5 and 7×7, and the expansion rates are 1, 2 and 3 respectively.

5. A method for underwater image quality assessment based on joint learning of brightness and chrominance, characterized in that: The method is applied to an underwater image quality assessment system for joint learning of brightness and chrominance as described in any one of claims 1 to 4, and the method steps include: S1, collecting underwater original images, enhancing them using an enhancement algorithm, and selecting the image with the best enhancement effect as the best enhanced image; S2, using the underwater original image and the best enhanced image as labels to train two generative adversarial networks, and obtaining two pseudo reference images, including the original image reference image and the best enhanced reference image; S3, converting the two pseudo reference images obtained in S2 and the enhanced image to be evaluated from RGB format to YCbCr format; S4, separating the brightness channel Y and the chrominance channel CbCr of the two pseudo reference images and the enhanced image to be evaluated, and using the constructed chrominance distortion evaluation module and the brightness distortion evaluation module to evaluate the image quality of the enhanced image to be evaluated from the brightness and chrominance aspects, respectively, to obtain the brightness quality representation and the chrominance quality representation; S5. The image quality of the enhanced image to be evaluated is jointly represented by the brightness quality representation and the chrominance quality representation.

6. The underwater image quality assessment method based on joint learning of luminance and chrominance according to claim 5, characterized in that: The generator G of the two generative adversarial networks in S2 adopts the DenseUNet network, and the discriminator D adopts the PatchGAN network.

7. The underwater image quality assessment method based on joint learning of luminance and chrominance according to claim 6, characterized in that: The loss function of the generator uses L2 and L vgg The perceptual loss function is: in, It is the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset; The loss function of the discriminator adopts the WGAN-GP algorithm: in, is the loss function, λ is the penalty coefficient, I g Generate images for the generator, I l be the original image or the best enhanced image; is the gradient norm of the output of the discriminator D with respect to its input, is the expected value.

8. The underwater image quality assessment method of luminance and chrominance joint learning according to claim 5 is characterized in that: The process of evaluating the image quality of the enhanced image to be evaluated from the aspect of chromaticity in S4 is as follows: The CbCr channels of the enhanced image to be evaluated and its unenhanced reference image are subtracted, and the CbCr channels of the unenhanced reference and the optimal enhanced reference are subtracted. The two values ​​are used as the input of the multi-scale feature mapping module. At this time, the output of each of the three levels in the multi-scale feature mapping module is the chromaticity quality representation of the image.

9. The underwater image quality assessment method of luminance and chrominance joint learning according to claim 5 is characterized in that: The process of evaluating the image quality of the enhanced image to be evaluated from the aspect of brightness in S4 is as follows: The Y channel of the enhanced image to be evaluated and its corresponding original image reference image is subtracted, and the Y channel of the original image reference image and the Y channel of the best enhanced reference image are subtracted, and the two values ​​are used as the input of the parallel attention module; and the output of the parallel attention module is input to the multi-scale feature mapping module. At this time, the output of each level of the three levels in the multi-scale feature mapping module is the brightness quality representation of the image.

10. The underwater image quality assessment method of luminance and chrominance joint learning according to claim 5 is characterized in that: The specific steps of jointly indicating the image quality of the enhanced image to be evaluated in S5 are: The chromaticity feature representation and brightness feature representation obtained in S4 are cascaded and input into the dense block respectively, and then the final quality score of the image is obtained by calculation. The loss function of the entire network is for: in, is the calculated image quality, is the average subjective score of the subjective quality label of the image.

Images