Underwater image quality evaluation model based on multi-feature fusion

Abstract

The invention provides an underwater image quality evaluation model based on multi-feature fusion, and belongs to the technical field of underwater image quality evaluation. The method is mainly used for solving the problem that after an underwater image is enhanced, the quality of the enhanced image cannot be objectively and accurately evaluated, so that the score is inconsistent with the human vision, and aiming at the problems that the underwater image is often blurred and the contrast ratio is low due to light absorption, scattering and color distortion, based on a brightness channel and a chromaticity channel in a YcbCr color space, the score is not consistent with the human vision. The method comprises the following steps: extracting brightness features, chrominance features and saliency features of an underwater image to carry out multi-feature fusion, carrying out model training on a data set by adopting a support vector machine sorting SVMrank model based on machine learning and utilizing the fused multi-dimensional features, and finally carrying out image quality evaluation on a to-be-detected image according to the trained model. And obtaining a quality evaluation score of a unified view angle.

Description

Technical Field

[0001] The present invention relates to the technical field of underwater image quality assessment, and in particular to an underwater image quality assessment model based on multi-feature fusion. Background Art

[0002] With the rapid advancement of science and technology, the exploration and research of the ocean has become more active. The ocean is not only one of the most important ecosystems on the earth, but also a treasure trove of resources, covering a variety of resources such as biological resources, mineral resources and energy. However, in marine scientific research, potential risks such as low visibility and high pressure in the ocean bring many difficulties to exploration activities. Therefore, underwater images are the key medium for obtaining and transmitting underwater information, and they undertake the important task of information collection and recording. Whether it is underwater resource exploration, marine environmental monitoring, or marine biological research, submarine engineering construction and other fields, the quality of underwater images directly affects the validity of data and the progress of research. Due to the particularity of the underwater environment, such as light attenuation, scattering and color distortion, underwater images usually have problems such as low contrast, blur and noise. These problems significantly reduce the visual quality of underwater images, which not only affects the difficulty of interpreting underwater images, but also limits their effectiveness in practical applications. In order to solve the quality problem of underwater images, many researchers are committed to developing underwater image enhancement (UIE) technology. Underwater image enhancement can improve the clarity and contrast of images, making underwater targets more visible. However, different UIE technologies have their own advantages and disadvantages in terms of effect. There is an urgent need for an evaluation method that can objectively and accurately evaluate the quality of underwater images in order to effectively compare and improve these algorithms. It is crucial to develop a dedicated underwater image quality assessment (UIQA) method that can more accurately reflect the quality of underwater images. Underwater image quality assessment methods can provide objective and quantitative standards for underwater image enhancement work, which is of great significance in underwater fields such as marine scientific research, resource development, and engineering construction. Summary of the invention

[0003] According to the technical problems raised above, a model for underwater image quality assessment based on multi-feature fusion is provided. The present invention is mainly used for the problem that after underwater image enhancement, the enhanced image quality cannot be objectively and accurately assessed, resulting in the problem that the score is inconsistent with human vision. In view of the problem that underwater images are often blurred and have low contrast due to light absorption, scattering and color distortion, based on the brightness channel and chromaticity channel in the YcbCr color space, the brightness features, chromaticity features and significance features of the underwater image are extracted for multi-feature fusion, and the SVMrank model based on machine learning is used to train the model on the data set using the fused multi-dimensional features. Finally, the image quality of the image to be tested is assessed based on the trained model to obtain a quality assessment score with a unified perspective.

[0004] The technical means adopted by the present invention are as follows:

[0005] An underwater image quality assessment method based on multi-feature fusion includes the following steps:

[0006] Step 01: Acquire multiple underwater original images and their corresponding enhanced image datasets, and randomly divide the multiple underwater original images and their corresponding enhanced image datasets into training set images and test set images in a ratio of 50%:50%;

[0007] Step 02: Convert the training set image from RGB color space to YCbCr color space;

[0008] Step 03: extracting brightness features on the Y channel in the YcbCr color space; the extraction includes: gray-level co-occurrence matrix texture features and local binary pattern histogram features;

[0009] Step 04: Extract chromaticity features on the Cb channel and the Cr channel in the YcbCr color space; the extraction includes: moment statistics features and red offset features;

[0010] Step 05: Convert the training set images into grayscale images and extract the significant features of the images;

[0011] Step 06: Fusing the brightness feature, the chromaticity feature and the saliency feature to obtain a combined feature vector, and performing model training on the training set images using a support vector machine ranking algorithm (SVMrank) model to obtain a trained model;

[0012] Step 07: Based on the trained model obtained in step 06, the quality of the underwater image is evaluated to obtain an image quality evaluation score.

[0013] Furthermore, the steps of converting the image color space in step 02 are as follows:

[0014] Step 21: Convert the training set image from RGB color space to YCbCr color space:

[0015] Y = 0.299R + 0.587G + 0.114B;

[0016] Cb=-0.169R-0.331G+0.5B+128;

[0017] Cr=0.5R-0.419G-0.081B+128;

[0018] Among them, Y represents the brightness component of the image in the YcbCr color space, Cb represents the blue chrominance component of the image in the YcbCr color space, and Cr represents the red chrominance component of the image in the YcbCr color space.

[0019] Furthermore, the step 03 of extracting brightness features comprises the following steps:

[0020] Step 31: Extract gray level co-occurrence matrix (GLCM) texture features on the Y component; in GLCM, each element (i, j) represents the co-occurrence frequency of two pixels with gray levels i and j in the image at a specific distance and direction; the GLCM of an image I with a size of M×N is calculated as:

[0021]

[0022] Where d and θ represent the distance and direction between pixels respectively; set d = 1, θ = 0°, 45°, 90°, 135°;

[0023] Step 32: After calculating the gray level co-occurrence matrix of the Y channel, four key features, namely contrast, correlation, energy and homogeneity, are extracted to reflect the texture properties of the image from different angles;

[0024]

[0025] Among them, μ x and μ y Respectively represent the mean of gray levels x and y, σ x and σ y Represent the standard deviation of the rows and columns of the matrix respectively; these four features are taken as the brightness feature f1, expressed as f1 = [Con Y ,Cor Y ,Ene Y ,Hom Y ];

[0026] Step 33: extract the local binary pattern histogram feature on the Y component; first perform local contrast normalization on the image to generate a mean contrast normalization map, and then calculate the gradient amplitude map and gradient direction map of the image; the mean contrast normalization map is:

[0027]

[0028] Where I(i,j) represents the pixel value at position (i,j) in the image; μ(i,j) and σ(i,j) represent the mean and standard deviation of the local area at that position, respectively; C is a constant used to avoid the denominator being zero; the local mean μ(i,j) and standard deviation σ(i,j) are calculated as follows:

[0029]

[0030] Where Y represents the brightness component of the underwater image in the YCbCr space, C represents a constant used to avoid numerical instability; μ and σ represent the mean and standard deviation of the local patch, respectively, and ω represents a Gaussian window of unit volume; N represents a local neighborhood window of image I, and ω(i,j) is a weighting coefficient used to control the influence of pixels in the neighborhood;

[0031] Step 34: The calculation formulas of the gradient magnitude map GM and the gradient direction map GO are:

[0032]

[0033] Among them, Y represents the brightness component of the underwater image in the YCbCr space, G x and G y Represents the Prewitt operator mode in the horizontal x and vertical y directions; the value range of GO(i,j) is [-180°, +180°];

[0034] Step 35: Then use the local binary pattern method to extract the local structural features of the image; by c Its neighboring pixel P i By comparison, the local binary pattern encoding of the central pixel is derived, and the calculation formula is:

[0035]

[0036] Where ψ represents P c and P i The threshold for binary relative values, P represents the number of adjacent pixels, and R represents the radius of the neighborhood;

[0037] Step 36: In order to achieve the local rotation invariance of LBP coding, a local rotation invariant uniform LBP operator is introduced. The LBP operator is based on a uniformity measure U quantization sequence ({T(p i -p c ),i=0,1,2,…,P-1}), such that:

[0038]

[0039] Step 37: Finally, the local binary pattern operator is applied to the mean-contrast normalized map, the gradient magnitude map and the gradient direction map respectively to obtain the corresponding LBP histogram;

[0040]

[0041] Among them, Λ∈{MSCN,GM,GO} indicates that the LBP operation is applied to the MSCN, GM and GO feature maps respectively; k∈{0,1,2,…,P+1} represents the grayscale of the LBP histogram, and H and W represent the height and width of the image respectively.

[0042] Finally, three LBP histograms based on MSCN, GM and GO are obtained; these three histogram features are used as brightness features f2, expressed as f2 = [H MSCN ,H GM ,H GO ].

[0043] Furthermore, the step 04 of chromaticity feature extraction comprises the following steps:

[0044] Step 41: First, calculate the moment statistics of the Cb and Cr components as the chromaticity features of the underwater image. Specifically, by calculating the mean, variance and skewness on the Cr and Cb component mappings, a three-dimensional moment statistic based on chromaticity is obtained:

[0045]

[0046]

[0047] Among them, x ij represents the first pixel of the jth pixel of the i-th component of the image, and N represents the number of pixels in the image;

[0048] The three-dimensional moment statistics based on chromaticity is taken as the chromaticity feature f3, expressed as f3 = [μ Cb ,σ Cb ,μ Cb ,μ Cr ,σ Cr ,s Cr ];

[0049] Step 42: First, the training set image is preprocessed using a color correction method, and the pixel value distribution of the image is adjusted using histogram equalization; the original histogram is calculated, and for each gray level i, the number of pixels with gray level i in the image is calculated, recorded as h(i); then the cumulative distribution function is calculated for gray level i, and the calculation formula of the cumulative distribution function CDF is:

[0050]

[0051] Normalize the value of the cumulative distribution function to the range of 0 to L-1, L = 256. The specific calculation formula is as follows:

[0052]

[0053] Finally, the normalized cumulative distribution function value is used to replace the original gray value of each pixel of the image to obtain the equalized image;

[0054] Step 43: Based on the equalized image, the ratio of red pixels to total pixels in the red channel is calculated as a quantitative index of the red offset feature;

[0055]

[0056] Among them, P represents the total number of pixels in the image, R i Represents the value of the i-th pixel in the red channel, R ki Represents the red pixel value after threshold processing; R k The calculation of is as follows:

[0057]

[0058] In order to quantify the reddish pixel value, a threshold is set, and pixels with red channel pixel values ​​greater than the threshold are regarded as reddish pixels. The red offset feature value F is obtained by calculating the proportion of pixels identified as red in the image to the total pixels. R ; shift the red feature F R As the chromaticity feature f4, it is expressed as f4=F R .

[0059] Furthermore, the step 05 of extracting significant features comprises the following steps:

[0060] Step 51: First, convert the training set image into a grayscale image Gray, and the calculation formula is as follows:

[0061]

[0062] Step 52: Then use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows:

[0063]

[0064] The calculation formula of the gradient amplitude G is as follows:

[0065]

[0066] Among them, I i,j Represents the gray value of the (i, j)th pixel in the gray image, Gx and G y Represent the gradients in the horizontal and vertical directions respectively, G is the gradient amplitude, and the gradient direction is arctan(Gy / Gx);

[0067] The gradient magnitude is normalized to the range of 0-255 and converted to an integer type; at the same time, the gradient direction is quantized to the nearest 45 degree angle;

[0068] The LBP algorithm is used to calculate the features of the integer gradient amplitude map to obtain the feature H of the saliency map. s ; The LBP histogram feature of the saliency map is taken as the saliency feature f5, expressed as f5 = H s .

[0069] Furthermore, the step 06 comprises the following steps:

[0070] Step 61: Fusing the brightness features f1, f2, the chromaticity features f3, f4 and the saliency feature f5 of the underwater image to obtain a multidimensional feature vector F, expressed as F = [f1, f2, f3, f4, f5];

[0071] Step 62: Use the multi-dimensional feature vector F of each image in the training set and the subjective score MOS of the corresponding image as input data to build a support vector machine ranking model for training.

[0072] Compared with the prior art, the present invention has the following advantages:

[0073] 1. To solve the problem that the existing underwater image quality assessment methods lack consideration of key features such as image texture, resulting in inaccurate prediction results for complex underwater scenes, the present invention uses the YCbCr color space and gray-level co-occurrence matrix to effectively separate and measure color and texture in underwater images, and improves the accurate description of image degradation characteristics.

[0074] 2. To solve the common reddish phenomenon in existing underwater image enhancement methods, this paper proposes a red offset feature extraction strategy based on color correction. By modeling the chromaticity features more finely and introducing visual saliency map features, the color distortion problem in the image is effectively improved, and the model's prediction performance for underwater image quality is further improved.

[0075] 3. Experimental results on two standard underwater image quality datasets show that the quality prediction performance of the model in a variety of complex scenarios is better than that of existing mainstream methods, significantly improving the accuracy and robustness of the evaluation.

[0076] Based on the above reasons, the present invention is particularly suitable for underwater image quality assessment in scenarios where objective prediction and scoring of enhanced images are required. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0078] Figure 1 It is a model flow chart of the present invention.

[0079] Figure 2 The six pictures selected for the present invention are underwater enhanced images with different degrees of reddishness, and the MOS value is the subjective quality score of the image; among them, (a) is MOS=8.4022; (b) is MOS=6.8333; (c) is MOS=3.0597; (d) is MOS=1.5473; (e) is MOS=0.6818; (f) is MOS=0.5410.

[0080] Figure 3 Image quality scores and their rankings are calculated for different UIQA methods in this invention.

[0081] Figure 4 The following are comparison diagrams of the effects of the present invention in different scenes in the underwater image dataset. Among them, (a) MOS = 7.4746; (b) MOS = 7.3546; (c) MOS = 6.7999; (d) MOS = 5.1210; (e) MOS = 4.4881; (f) MOS = 3.5561; (g) MOS = 2.7121; (h) MOS = 1.5771.

[0082] Figure 5 Image quality scores and their rankings are calculated for different UIQA methods of the present invention. DETAILED DESCRIPTION

[0083] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0084] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0085] like Figure 1 As shown, the present invention provides an underwater image quality assessment method based on multi-feature fusion, comprising the following steps:

[0086] Step 01: obtain multiple underwater original images and their corresponding enhanced image data sets, and randomly divide the multiple underwater original images and their corresponding enhanced image data sets into training set images and test set images in a ratio of 50%:50%; Step 02: convert the training set images from RGB color space to YcbCr color space; the steps of converting the image color space in Step 02 are:

[0087] Step 21: Convert the training set image obtained in step 01 from RGB color space to YCbCr color space. The specific color space conversion definition is as follows:

[0088] Y = 0.299R + 0.587G + 0.114B;

[0089] Cb=-0.169R-0.331G+0.5B+128;

[0090] Cr=0.5R-0.419G-0.081B+128;

[0091] Where Y represents the brightness component of the image in the YcbCr color space, Cb represents the blue chrominance component in the YcbCr color space, and Cr represents the red chrominance component in the YcbCr color space.

[0092] Step 03: extracting brightness features on the Y channel in the YcbCr color space, including gray-level co-occurrence matrix texture features and local binary pattern (LBP) histogram features; the brightness feature extraction of step 03 includes the following steps:

[0093] Step 31: Extract the gray level co-occurrence matrix (GLCM) texture features on the Y component; in GLCM, each element (i, j) represents the co-occurrence frequency of two pixels with gray levels i and j in the image at a specific distance and direction. The GLCM of an image I with a size of M×N is calculated as follows:

[0094]

[0095] Where d and θ represent the distance and direction between pixels respectively. Set d = 1, θ = 0°, 45°, 90°, 135°.

[0096] Step 32: After calculating the gray level co-occurrence matrix of the Y channel, four key features, namely contrast, correlation, energy and homogeneity, are extracted to reflect the texture properties of the image from different angles. The specific calculation formula is as follows:

[0097]

[0098] Among them, μ x and μ y are the means of gray levels x and y, σ x and σ y These four features are taken as brightness feature f1, expressed as f1 = [Con Y ,Cor Y ,Ene Y ,Hom Y ].

[0099] Step 33: Extract the local binary pattern (LBP) histogram feature on the Y component; first perform local contrast normalization on the image to generate a mean contrast normalization (MSCN) map, and then calculate the gradient magnitude (GM) map and gradient direction (GO) map of the image. The specific calculation formula of MSCN is as follows:

[0100]

[0101] Where I(i,j) represents the pixel value at position (i,j) in the image, μ(i,j) and σ(i,j) represent the mean and standard deviation of the local area at that position, respectively, and C is a constant to avoid the denominator being zero. The local mean μ(i,j) and standard deviation σ(i,j) are calculated as follows:

[0102]

[0103] Where Y is the brightness component of the underwater image in the YCbCr space, C is a constant used to avoid numerical instability. μ and σ are the mean and standard deviation of the local patch, respectively, and ω defines a Gaussian window per unit volume. N represents a local neighborhood window of image I, and ω(i,j) is a weighting coefficient used to control the influence of pixels in the neighborhood.

[0104] Step 34: The specific calculation formulas of the gradient magnitude map GM and the gradient direction map GO are as follows:

[0105]

[0106] Where Y is the brightness component of the underwater image in the YCbCr space, G x and G y Represents the Prewitt operator pattern in the horizontal x and vertical y directions. The value range of GO(i,j) is [-180°, +180°].

[0107] Step 35: Then use the local binary pattern (LBP) method to extract the local structural features of the image. Specifically, by c Its neighboring pixel P i By comparison, the LBP coding of the central pixel is derived. The calculation formula is as follows:

[0108]

[0109] Where ψ is P c and P i is the threshold for binarization of relative values, P is the number of adjacent pixels, and R is the radius of the neighborhood.

[0110] Step 36: To achieve local rotation invariance of LBP coding, we use a local rotation invariant uniform LBP operator based on a uniformity measure U quantized sequence ({T(p i -p c ),i=0,1,2,…,P-1}), such that:

[0111]

[0112] Step 37: Finally, the local binary pattern (LBP) operator is applied to the mean-contrast normalization (MSCN) map, the gradient magnitude map (GM) and the gradient direction map (GO) to obtain the corresponding LBP histogram. The specific calculation formula is as follows:

[0113]

[0114] where Λ∈{MSCN, GM, GO} denotes the application of LBP operations to the MSCN, GM, and GO feature maps, respectively, k∈{0,1,2,…,P+1} is the grayscale of the LBP histogram, and H and W are the height and width of the image, respectively.

[0115] Finally, three LBP histograms based on MSCN, GM and GO are obtained. These three histogram features are used as brightness features f2, expressed as f2 = [H MSCN ,H GM ,H GO ].

[0116] Step 04: Extract chromaticity features on the Cb channel and the Cr channel in the YcbCr color space, including moment statistics features and red offset features; the chromaticity feature extraction in step 04 includes the following steps:

[0117] Step 41: First, calculate the moment statistics of the Cb and Cr components as the chromaticity features of the underwater image. Specifically, by calculating the mean (μ, first-order moment), variance (σ, second-order moment) and skewness (s, third-order moment) on the Cr and Cb component mappings, a three-dimensional moment statistic based on chromaticity is obtained. The specific formula is as follows:

[0118]

[0119]

[0120] where x ij represents the first pixel of the jth pixel of the i-th component of the image, and N represents the number of pixels in the image.

[0121] The three-dimensional moment statistics based on chromaticity is taken as the chromaticity feature f3, expressed as f3 = [μ Cb ,σ Cb ,s Cb ,μ Cr ,σ Cr ,s Cr ].

[0122] Step 42: First, the training set image obtained in step 01 is preprocessed using the color correction method, and the pixel value distribution of the image is adjusted using histogram equalization. The original histogram is calculated, and for each gray level i, the number of pixels with gray level i in the image is calculated, recorded as h(i). Then the cumulative distribution function (CDF) is calculated, which is the cumulative sum of the histogram and can measure the cumulative distribution of gray levels. For gray level i, the calculation formula of the cumulative distribution function (CDF) is as follows:

[0123]

[0124] The value of the cumulative distribution function CDF is normalized to the range of 0 to L-1 (L=256). The specific calculation formula is as follows:

[0125]

[0126] Finally, the normalized cumulative distribution function (CDF) value is used to replace the original grayscale value of each pixel to obtain the equalized image.

[0127] Step 43: Based on the equalized image, calculate the ratio of red pixels to total pixels in the red channel as a quantitative indicator of the red offset feature. The specific calculation formula is as follows:

[0128]

[0129] Among them, P represents the total number of pixels in the image, R i Represents the value of the i-th pixel in the red channel, R ki Represents the red pixel value after threshold processing. R k The calculation of is as follows:

[0130]

[0131] In order to quantify the reddish pixel value, a threshold of 0.6 is set, and pixels with red channel pixel values ​​greater than 0.6 are regarded as reddish pixels. The red offset feature value F is obtained by calculating the proportion of pixels identified as red to the total pixels in the image. R . The red offset feature F R As the chromaticity feature f4, it is expressed as f4=F R .

[0132] Step 05: convert the training set image obtained in step 01 into a grayscale image, and extract the significant features of the image; the significant feature extraction in step 05 includes the following steps:

[0133] Step 51: First, convert the training set image obtained in step 01 into a grayscale image Gray. The calculation formula is as follows:

[0134]

[0135] Step 52: Then use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows:

[0136]

[0137] The calculation formula of the gradient amplitude G is as follows:

[0138]

[0139] Among them, I i,j Represents the gray value of the (i, j)th pixel in the gray image, G x and G y Represent the gradients in the horizontal and vertical directions, G is the gradient amplitude, and the gradient direction is arctan (Gy / Gx). In order to simplify the subsequent calculations, the gradient amplitude is normalized to the range of 0-255 and converted to an integer type. At the same time, the gradient direction is quantized to the nearest 45-degree angle for LBP calculation. Finally, through the formulas in steps 35 and 36, the LBP algorithm is used to calculate the features of the integer gradient amplitude map to obtain the feature H of the saliency map. s The LBP histogram feature of the saliency map is taken as the saliency feature f5, expressed as f5 = H s .

[0140] Step 06: According to the brightness feature, chromaticity feature and significance feature obtained in step 03, step 04 and step 05, feature fusion is performed to obtain a combined feature vector, and a support vector machine ranking (SVMrank) model is used to perform model training on the training set; step 06 includes the following steps:

[0141] Step 61: Fusing the brightness features f1, f2, the chromaticity features f3, f4 and the saliency feature f5 of the underwater image to obtain a multidimensional feature vector F, expressed as F = [f1, f2, f3, f4, f5];

[0142] Step 62: Taking the multidimensional feature vector F of each image in the training set and the subjective score MOS of the corresponding image as input data, a support vector machine ranking (SVMrank) model is constructed for training;

[0143] Step 07: Use the trained model obtained in step 06 to evaluate the quality of underwater images and obtain the image quality evaluation score of the unified perspective. Step 7 includes the following steps:

[0144] Step 71: Input a test image into the trained model to obtain a multi-dimensional feature vector F of the image;

[0145] Step 72: The model predicts the quality of the image based on the multi-dimensional feature vector F of the image to obtain an objective image quality score.

[0146] Embodiment 1:

[0147] In order to verify the generalization of the evaluation of different underwater scenes, two public underwater image datasets, SAUD dataset and UID dataset, were selected as test sets. At the same time, the experimental results of the proposed method were compared with those of UIQM (Human-visual-system-inspired underwater image quality measures) algorithm, UCIQE (An underwater color image quality evaluation metric) algorithm, CCF (An imaging-inspired no-reference underwater color image quality assessment metric), FDUM (A reference-free underwater image quality assessment metric in frequency domain), Twice Mixing (Twice mixing: a rank learning based quality assessment approach for underwater image enhancement), and ATUIQP (Underwater image quality assessment: Benchmark database and objective method) algorithm from both qualitative and quantitative aspects.

[0148] like Figure 2 , Figure 3 As shown, the present invention provides a comparative effect diagram of underwater image quality assessment with excessive enhancement and reddish phenomenon. From the experimental effect diagram, it can be seen that the subjective quality scores of the six reddish images decrease from a to f in sequence. It can be observed that the quality assessment scores of the reddish images of the present invention also decrease from a to f in sequence, which is completely consistent with the ranking of subjective scores. The algorithm of the present invention can perfectly predict the objective score of the reddish underwater enhanced image.

[0149] like Figure 4 , Figure 5 As shown, the present invention provides a comparison effect diagram of underwater image quality assessment in different scenes. From the experimental effect diagram, it can be seen that the subjective quality scores of 8 underwater images of different scenes decrease from a to h in sequence. It can be observed that the quality assessment scores of the group of images of the present invention also decrease from a to h in sequence, which is completely consistent with the ranking of subjective scores. The algorithm of the present invention can perfectly predict the objective scores of underwater images in different scenes.

[0150] This embodiment compares the experimental results of different algorithms from four objective indicators: Spearman rank correlation coefficient SROCC, Kendall rank correlation coefficient KROCC, Pearson linear correlation coefficient PLCC and root mean square error RMSE; when SROCC takes 1, it means that the algorithm performance is very good, and -1 means it is very poor. The closer the value is to 1, the better the performance of the IQA algorithm. The larger the value of KROCC, the better the correlation between the two data and the better the performance of the IQA algorithm; the smaller the value, the worse the correlation. In contrast, SROCC focuses on calculating the correlation between two vectors, while KROCC tends to evaluate the dependence strength of two vectors. PLCC describes the linear correlation between two sets of data, and its value range is -1 to 1. When the value of PLCC is zero, it means that the two sets of data are independent. The closer the RMSE root mean square error is to 0, the better the performance of the IQA algorithm. From the data in Tables 1 and 2, it can be seen that this method is superior to other mainstream underwater image quality assessment methods, and has achieved the best results in SROCC, KROCC, PLCC and RMSE indicators.

[0151] Table 1 Comparison of the processing results of the proposed model and other advanced algorithms on SAUD

[0152] Metrics Mean SROCC Mean KROCC Mean PLCC Mean RMSE UIQM 0.0267 0.0269 0.5124 1.2161 UCIQE 0.1443 0.0869 0.7464 0.9046 CCF -0.0298 -0.0237 0.6784 1.0083 FDUM 0.1553 0.0923 0.6526 1.0158 Twice Mixing 0.3760 0.2937 0.7443 0.9263 ATUIQP 0.2699 0.1967 0.6802 0.9943 Ours 0.6365 0.5050 0.8068 0.7715

[0153] Table 2 Comparison of the processing results of the proposed model and other advanced algorithms on UID

[0154]

[0155]

[0156] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image quality assessment method based on multi-feature fusion, characterized in that: The following steps are involved: Step 01: Acquire multiple underwater original images and their corresponding enhanced image datasets, and randomly divide the multiple underwater original images and their corresponding enhanced image datasets into training set images and test set images in a ratio of 50%:50%; Step 02: Convert the training set image from RGB color space to YCbCr color space; Step 03: extracting brightness features on the Y channel in the YcbCr color space; the extraction includes: gray-level co-occurrence matrix texture features and local binary pattern histogram features; Step 04: Extract chromaticity features on the Cb channel and the Cr channel in the YcbCr color space; the extraction includes: moment statistics features and red offset features; Step 05: Convert the training set images into grayscale images and extract the significant features of the images; Step 06: Fusing the brightness feature, the chromaticity feature and the saliency feature to obtain a combined feature vector, and performing model training on the training set images using a support vector machine ranking algorithm (SVMrank) model to obtain a trained model; Step 07: Based on the trained model obtained in step 06, the quality of the underwater image is evaluated to obtain an image quality evaluation score.

2. The underwater image quality assessment method based on multi-feature fusion according to claim 1 is characterized in that: The steps of step 02 to convert the image color space are as follows: Step 21: Convert the training set image from RGB color space to YCbCr color space: Y = 0.299R + 0.587G + 0.114B; Cb=-0.169R-0.331G+0.5B+128; Cr=0.5R-0.419G-0.081B+128; Among them, Y represents the brightness component of the image in the YcbCr color space, Cb represents the blue chrominance component of the image in the YcbCr color space, and Cr represents the red chrominance component of the image in the YcbCr color space.

3. The underwater image quality assessment method based on multi-feature fusion according to claim 1 is characterized in that: The step 03 of brightness feature extraction comprises the following steps: Step 31: Extract gray level co-occurrence matrix (GLCM) texture features on the Y component; in GLCM, each element (i, j) represents the co-occurrence frequency of two pixels with gray levels i and j in the image at a specific distance and direction; the GLCM of an image I with a size of M×N is calculated as: Where d and θ represent the distance and direction between pixels respectively; set d = 1, θ = 0°, 45°, 90°, 135°; Step 32: After calculating the gray level co-occurrence matrix of the Y channel, four key features, namely contrast, correlation, energy and homogeneity, are extracted to reflect the texture properties of the image from different angles; Among them, μ x and μ y Respectively represent the mean of gray levels x and y, σ x and σ y Represent the standard deviation of the rows and columns of the matrix respectively; these four features are taken as the brightness feature f1, expressed as f1 = [Con Y ,Cor Y ,Ene Y ,Hom Y ]; Step 33: extract the local binary pattern histogram feature on the Y component; first perform local contrast normalization on the image to generate a mean contrast normalization map, and then calculate the gradient amplitude map and gradient direction map of the image; the mean contrast normalization map is: Where I(i,j) represents the pixel value at position (i,j) in the image; μ(i,j) and σ(i,j) represent the mean and standard deviation of the local area at that position, respectively; C is a constant used to avoid the denominator being zero; the local mean μ(i,j) and standard deviation σ(i,j) are calculated as follows: Where Y represents the brightness component of the underwater image in the YCbCr space, C represents a constant used to avoid numerical instability; μ and σ represent the mean and standard deviation of the local patch, respectively, and ω represents a Gaussian window of unit volume; N represents a local neighborhood window of image I, and ω(i,j) is a weighting coefficient used to control the influence of pixels in the neighborhood; Step 34: The calculation formulas of the gradient magnitude map GM and the gradient direction map GO are: Among them, Y represents the brightness component of the underwater image in the YCbCr space, G x and G y Represents the Prewitt operator mode in the horizontal x and vertical y directions; the value range of GO(i,j) is [-180°, +180°]; Step 35: Then use the local binary pattern method to extract the local structural features of the image; by c Its neighboring pixel P i By comparison, the local binary pattern encoding of the central pixel is derived, and the calculation formula is: Where ψ represents P c and P i The threshold for binary relative values, P represents the number of adjacent pixels, and R represents the radius of the neighborhood; Step 36: In order to achieve the local rotation invariance of LBP coding, a local rotation invariant uniform LBP operator is introduced. The LBP operator is based on a uniformity measure U quantization sequence ({T(p i -p c ),i=0,1,2,…,P-1}), such that: Step 37: Finally, the local binary pattern operator is applied to the mean-contrast normalized map, the gradient magnitude map and the gradient direction map respectively to obtain the corresponding LBP histogram; Among them, Λ∈{MSCN,GM,GO} indicates that the LBP operation is applied to the MSCN, GM and GO feature maps respectively; k∈{0,1,2,…,P+1} represents the grayscale of the LBP histogram, and H and W represent the height and width of the image respectively. Finally, three LBP histograms based on MSCN, GM and GO are obtained; these three histogram features are used as brightness features f2, expressed as f2 = [H MSCN ,H GM ,H GO ].

4. The underwater image quality assessment method based on multi-feature fusion according to claim 1 is characterized in that: The step 04 chromaticity feature extraction comprises the following steps: Step 41: First, calculate the moment statistics of the Cb and Cr components as the chromaticity features of the underwater image. Specifically, by calculating the mean, variance and skewness on the Cr and Cb component mappings, a three-dimensional moment statistic based on chromaticity is obtained: Among them, x ij represents the first pixel of the jth pixel of the i-th component of the image, and N represents the number of pixels in the image; The three-dimensional moment statistics based on chromaticity is taken as the chromaticity feature f3, expressed as f3 = [μ Cb ,σ Cb ,s Cb ,μ Cr ,σ Cr ,s Cr ]; Step 42: First, the training set image is preprocessed using a color correction method, and the pixel value distribution of the image is adjusted using histogram equalization; the original histogram is calculated, and for each gray level i, the number of pixels with gray level i in the image is calculated, recorded as h(i); then the cumulative distribution function is calculated for gray level i, and the calculation formula of the cumulative distribution function CDF is: Normalize the value of the cumulative distribution function to the range of 0 to L-1, L = 256. The specific calculation formula is as follows: Finally, the normalized cumulative distribution function value is used to replace the original gray value of each pixel of the image to obtain the equalized image; Step 43: Based on the equalized image, the ratio of red pixels to total pixels in the red channel is calculated as a quantitative index of the red offset feature; Among them, P represents the total number of pixels in the image, R i Represents the value of the i-th pixel in the red channel, R ki Represents the red pixel value after threshold processing; R k The calculation of is as follows: In order to quantify the reddish pixel value, a threshold is set, and pixels with red channel pixel values ​​greater than the threshold are regarded as reddish pixels. The red offset feature value F is obtained by calculating the proportion of pixels identified as red in the image to the total pixels. R ; Shift the red feature F R As the chromaticity feature f4, it is expressed as f4=F R .

5. The underwater image quality assessment method based on multi-feature fusion according to claim 1 is characterized in that: The step 05 of extracting significant features comprises the following steps: Step 51: First, convert the training set image into a grayscale image Gray, and the calculation formula is as follows: Step 52: Then use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows: The calculation formula of the gradient amplitude G is as follows: Among them, I i,j Represents the gray value of the (i, j)th pixel in the gray image, G x and G y Represent the gradients in the horizontal and vertical directions respectively, G is the gradient amplitude, and the gradient direction is arctan(Gy / Gx); The gradient magnitude is normalized to the range of 0-255 and converted to an integer type; at the same time, the gradient direction is quantized to the nearest 45 degree angle; The LBP algorithm is used to calculate the features of the integer gradient amplitude map to obtain the feature H of the saliency map. s ; The LBP histogram feature of the saliency map is taken as the saliency feature f5, expressed as f5 = H s .

6. The underwater image quality assessment method based on multi-feature fusion according to claim 1 is characterized in that: The step 06 comprises the following steps: Step 61: Fusing the brightness features f1, f2, the chromaticity features f3, f4 and the saliency feature f5 of the underwater image to obtain a multidimensional feature vector F, expressed as F = [f1, f2, f3, f4, f5]; Step 62: Use the multi-dimensional feature vector F of each image in the training set and the subjective score MOS of the corresponding image as input data to build a support vector machine ranking model for training.

Images