An underwater image quality assessment model based on multi-feature fusion

Abstract

The application provides a kind of underwater image quality evaluation model based on multi-feature fusion, belongs to underwater image quality evaluation technical field.The application mainly uses in underwater image enhancement, cannot objectively and accurately evaluate the quality of enhanced image, leading to the problem that score is inconsistent with human eye vision, in view of the light absorption, scattering and color distortion make underwater image often appear fuzzy and low contrast problem, based on luminance channel and chroma channel in YcbCr color space, extract the luminance feature, chroma feature and saliency feature of underwater image for multi-feature fusion, adopt support vector machine ranking SVMrank model based on machine learning to utilize the multi-dimensional feature after fusion to carry out model training on data set, finally, according to the trained model, the image quality of the image to be tested is evaluated, and the quality evaluation score of unified perspective is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater image quality evaluation, in particular, especially relates to an underwater image quality evaluation model based on multi-feature fusion. BACKGROUND

[0002] With the rapid progress of science and technology, the exploration and research of the ocean are increasingly active. The ocean is not only one of the most important ecosystems on earth, but also a treasure trove of resources, covering biological resources, mineral resources and energy resources and various resources. However, in the research of marine science, the potential risks of low visibility and high pressure in the ocean bring many difficulties to the exploration activities. Therefore, underwater images are the key medium for obtaining and transmitting underwater information, and undertake the important task of information collection and recording. Whether it is underwater resource exploration, marine environment monitoring, or marine biology research, seabed engineering construction and other fields, the quality of underwater images directly affects the effectiveness of data and the progress of research. Due to the particularity of the underwater environment, such as light attenuation, scattering and color distortion, underwater images usually have problems such as low contrast, blur and noise. These problems significantly reduce the visual quality of underwater images, not only affecting the difficulty of underwater image interpretation, but also limiting its effectiveness in practical applications. In order to solve the quality problem of underwater images, many researchers have devoted themselves to the development of underwater image enhancement (UIE) technology. Underwater image enhancement can improve the clarity and contrast of images, making underwater targets more visible. However, different UIE technologies have their own advantages and disadvantages in effect, and there is an urgent need for an evaluation method that can objectively and accurately evaluate the quality of underwater images to effectively compare and improve these algorithms. It is crucial to develop a special underwater image quality assessment (UIQA) method to more accurately reflect the quality of underwater images. The underwater image quality assessment method can provide objective and quantitative standards for underwater image enhancement work, which is of great significance in the fields of marine science research, resource development, engineering construction and other underwater fields. SUMMARY

[0003] In view of the above technical problems, an underwater image quality evaluation model based on multi-feature fusion is provided. The present application mainly uses the problem that after underwater image enhancement, the quality of the enhanced image cannot be objectively and accurately evaluated, leading to inconsistent scores with human visual perception. In view of the problems of blur and low contrast in underwater images caused by light absorption, scattering and color distortion, the brightness feature, chroma feature and saliency feature of the underwater image are extracted based on the brightness channel and chroma channel in the YcbCr color space for multi-feature fusion. The SVMrank model based on machine learning is used to train the model on the data set using the fused multi-dimensional features. Finally, the trained model is used to evaluate the image quality of the test image, and the quality evaluation score of the unified perspective is obtained.

[0004] The technical means adopted by the present application are as follows:

[0005] A multi-feature fusion-based underwater image quality assessment method, comprising the following steps:

[0006] Step 01: Obtain a plurality of underwater original image and its corresponding enhanced image data set, according to the proportion of 50%:50%, the plurality of underwater original image and its corresponding enhanced image data set is randomly divided into training set image and test set image;

[0007] Step 02: convert the training set image from RGB color space to YcbCr color space;

[0008] Step 03: extract the brightness feature on the Y channel in YcbCr color space; the extraction includes: gray level co-occurrence matrix texture feature and local binary pattern histogram feature;

[0009] Step 04: extract the chrominance feature on the Cb channel and Cr channel in YcbCr color space; the extraction includes: moment statistics feature and red shift feature;

[0010] Step 05: convert the training set image to a gray image, and extract the saliency feature of the image;

[0011] Step 06: fuse the brightness feature, the chrominance feature and the saliency feature to obtain a combined feature vector, use support vector machine ranking algorithm (SVMrank) model to train the model on the training set image, and obtain the trained model;

[0012] Step 07: according to the trained model obtained in step 06, the quality of underwater image is evaluated, and the image quality evaluation score is obtained.

[0013] Further, the step of converting the image color space in step 02 is:

[0014] Step 21: convert the training set image from RGB color space to YcbCr color space:

[0015] Y=0.299R+0.587G+0.114B;

[0016] Cb=-0.169R-0.331G+0.5B+128;

[0017] Cr=0.5R-0.419G-0.081B+128;

[0018] Y represents the luminance component of the image in the YcbCr color space, Cb represents the blue chroma component of the image in the YcbCr color space, and Cr represents the red chroma component of the image in the YcbCr color space.

[0019] Further, the step 03 luminance feature extraction comprises the following steps:

[0020] Step 31: Extracting the gray level co-occurrence matrix (GLCM) texture feature on the Y component; in the GLCM, each element (i, j) represents the co-occurrence frequency of two pixels with gray levels i and j in the image at a certain distance and direction; the GLCM of an image I with a size of MxN is calculated as:

[0021]

[0022] where d and θ represent the distance and direction between pixels, respectively; set d = 1, θ = 0°, 45°, 90°, 135°;

[0023] Step 32: After calculating the gray level co-occurrence matrix of the Y channel, four key features of contrast, correlation, energy and homogeneity are extracted to reflect the texture properties of the image from different angles;

[0024]

[0025] where μ x and μ y represent the mean values of gray levels x and y, respectively, σ x and σ y represent the standard deviations of the rows and columns of the matrix, respectively; these four features are taken as the luminance feature f1, represented as f1 = [Con Y , Cor Y , Ene Y , Hom Y ];

[0026] Step 33: Extracting the local binary pattern histogram feature on the Y component; first, performing local contrast normalization on the image to generate a mean contrast normalized image, then calculating the gradient amplitude map and gradient direction map of the image; the mean contrast normalized image is:

[0027]

[0028] where I(i,j) denotes the pixel value at position (i,j) in the image; μ(i,j) and σ(i,j) denote the local mean and standard deviation of the position, respectively; C is a constant to avoid the case that the denominator is zero; the local mean μ(i,j) and standard deviation σ(i,j) are calculated as follows:

[0029]

[0030] where Y denotes the luminance component of the underwater image in YCbCr space; C is a constant to avoid numerical instability; μ and σ denote the local mean and standard deviation of the patch, respectively; ω denotes the unit volume of the Gaussian window; N denotes a local neighborhood window of the image I; ω(i,j) is the weighting coefficient to control the influence degree of the pixels in the neighborhood;

[0031] Step 34: The calculation formulae of the gradient magnitude map GM and the gradient orientation map GO are as follows:

[0032]

[0033] where Y denotes the luminance component of the underwater image in YCbCr space; G x and G y denote the Prewitt operator patterns in the horizontal x and vertical y directions; the value range of GO(i,j) is [-180°, +180°];

[0034] Step 35: Then, the local structural features of the image are extracted using the local binary pattern method; by comparing each center pixel P c with its adjacent pixels P i , the local binary pattern code of the center pixel is derived, and the calculation formula is as follows:

[0035]

[0036] where ψ denotes the threshold value of the relative value binary of P c and P i ; P denotes the number of adjacent pixels; R denotes the radius of the neighborhood;

[0037] Step 36: In order to realize the local rotation invariance of the LBP code, a local rotation invariant uniform LBP operator is introduced, which quantifies the sequence ({T(p i -p c ), i = 0, 1, 2, …, P-1}) based on a uniformity measure U, so that:

[0038]

[0039] Step 37: Finally, the local binary pattern operator is applied to the mean-contrast normalized image, the gradient magnitude image and the gradient orientation image respectively to obtain the corresponding LBP histogram;

[0040]

[0041] where Λ∈{MSCN, GM, GO} represents the LBP operation applied to the MSCN, GM and GO feature maps respectively; k∈{0, 1, 2, …, P+1} represents the gray levels of the LBP histogram; H and W represent the height and width of the image respectively.

[0042] Finally, three LBP histograms based on MSCN, GM and GO are obtained; the three histogram features are taken as the luminance feature f2, denoted as f2=[H MSCN ,H GM ,H GO ].

[0043] Further, the step 04 chroma feature extraction comprises the following steps:

[0044] Step 41: First, the moment statistics of the Cb and Cr components are calculated as the chroma features of the underwater image. Specifically, by calculating the mean value, variance and skewness on the Cr and Cb component maps, three-dimensional moment statistics based on chroma are obtained:

[0045]

[0046]

[0047] where x ij represents the first pixel of the jth pixel of the ith component of the image, and N represents the number of pixels in the image;

[0048] The three-dimensional moment statistics based on chroma are taken as the chroma feature f3, denoted as f3=[μ Cb ,σ Cb ,μ Cb ,μ Cr ,σ Cr ,s Cr ];

[0049] Step 42: First, the training set images are preprocessed using a color correction method, and histogram equalization is used to adjust the pixel value distribution of the image; the original histogram is calculated, and for each gray level i, the number of pixels with gray level i in the image is calculated, denoted as h(i); then the cumulative distribution function is calculated, and for gray level i, the calculation formula of the cumulative distribution function CDF is:

[0050]

[0051] The cumulative distribution function is normalized to the range of 0 to L-1, where L = 256. The specific calculation formula is as follows:

[0052]

[0053] Finally, the original gray value of each pixel in the image is replaced by the normalized cumulative distribution function value to obtain the equalized image;

[0054] Step 43: Based on the equalized image, calculate the proportion of red pixels in the red channel to the total number of pixels, as a quantitative indicator of the red shift feature.

[0055]

[0056] Where P represents the total number of pixels in the image, R i R represents the value of the i-th pixel in the red channel. ki R represents the value of the red pixel after thresholding; k The calculation is as follows:

[0057]

[0058] To quantify red-biased pixel values, a threshold is set, and pixels with red channel values ​​greater than the threshold are considered red-biased pixels. The red offset feature value F is obtained by calculating the proportion of pixels identified as red in the image. R ; the red offset feature F R As a chromaticity feature, f4 is expressed as f4 = F R .

[0059] Furthermore, step 05, salient feature extraction, includes the following steps:

[0060] Step 51: First, convert the training set images into grayscale images (Gray). The calculation formula is as follows:

[0061]

[0062] Step 52: Then, use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows:

[0063]

[0064] The formula for calculating the gradient magnitude G is as follows:

[0065]

[0066] Among them, I i,j G represents the gray value of the (i,j)th pixel in a grayscale image.x and G y Let G and Gx represent the gradients in the horizontal and vertical directions, respectively, where G is the gradient magnitude and the gradient direction is arctan(Gy / Gx).

[0067] The gradient magnitude is normalized to the range of 0-255 and converted to an integer type; at the same time, the gradient direction is quantized to the nearest 45-degree angle.

[0068] The LBP algorithm is used to calculate the features of the integerized gradient magnitude map to obtain the feature H of the saliency map. s The LBP histogram features of the saliency plot are taken as the saliency feature f5, which is expressed as f5 = H. s .

[0069] Furthermore, step 06 includes the following steps:

[0070] Step 61: Fuse the brightness features f1, f2, chromaticity features f3, f4 and saliency features f5 of the underwater image to obtain a multidimensional feature vector F, denoted as F = [f1, f2, f3, f4, f5];

[0071] Step 62: Use the multidimensional feature vector F of each image in the training set and the corresponding subjective rating MOS of the image as input data to construct a support vector machine ranking model for training.

[0072] Compared with the prior art, the present invention has the following advantages:

[0073] 1. To address the problem that existing underwater image quality assessment methods lack consideration for key features such as image texture, leading to inaccurate prediction results for complex underwater scenes, this invention utilizes the YCbCr color space and gray-level co-occurrence matrix to effectively separate and measure color and texture in underwater images, thereby improving the accurate description of image degradation characteristics.

[0074] 2. To address the common red cast issue in existing underwater image enhancement methods, this invention proposes a red offset feature extraction strategy based on color correction. By performing more refined modeling of chromaticity features and introducing visual saliency map features, the color distortion problem in the image is effectively improved, further enhancing the model's predictive performance for underwater image quality.

[0075] 3. Experimental results on two standard underwater image quality datasets show that the model outperforms existing mainstream methods in quality prediction under various complex scenarios, significantly improving the accuracy and robustness of the evaluation.

[0076] Based on the above reasons, the present invention is particularly suitable for underwater image quality assessment in scenarios where objective prediction and scoring of enhanced images are required. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 This is a flowchart of the model of the present invention.

[0079] Figure 2 The six images selected for this invention are underwater enhanced images with varying degrees of reddish tint. The MOS value is the subjective quality score of the image; where (a) has a MOS of 8.4022; (b) has a MOS of 6.8333; (c) has a MOS of 3.0597; (d) has a MOS of 1.5473; (e) has a MOS of 0.6818; and (f) has a MOS of 0.5410.

[0080] Figure 3 This invention calculates image quality scores and their rankings using different UIQA methods.

[0081] Figure 4 These are comparison images of the effects of this invention in different scenarios within an underwater image dataset. In the figures, (a) MOS = 7.4746; (b) MOS = 7.3546; (c) MOS = 6.7999; (d) MOS = 5.1210; (e) MOS = 4.4881; (f) MOS = 3.5561; (g) MOS = 2.7121; and (h) MOS = 1.5771.

[0082] Figure 5 This invention calculates image quality scores and their rankings using different UIQA methods. Detailed Implementation

[0083] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0084] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0085] like Figure 1 As shown, this invention provides an underwater image quality assessment method based on multi-feature fusion, comprising the following steps:

[0086] Step 01: Obtain multiple raw underwater images and their corresponding enhanced image datasets, and randomly divide the multiple raw underwater images and their corresponding enhanced image datasets into training set images and test set images in a 50%:50% ratio; Step 02: Convert the training set images from the RGB color space to the YcbCr color space; The steps for converting the image color space in Step 02 are as follows:

[0087] Step 21: Convert the training set images obtained in Step 01 from the RGB color space to the YcbCr color space. The specific color space conversion definition is as follows:

[0088] Y = 0.299R + 0.587G + 0.114B;

[0089] Cb=-0.169R-0.331G+0.5B+128;

[0090] Cr=0.5R-0.419G-0.081B+128;

[0091] Where Y represents the luminance component of the image in the YcbCr color space, Cb represents the blue chromaticity component in the YcbCr color space, and Cr represents the red chromaticity component in the YcbCr color space.

[0092] Step 03: Extract luminance features from the Y channel in the YcbCr color space, including gray-level co-occurrence matrix texture features and local binary mode (LBP) histogram features; Step 03, luminance feature extraction, includes the following steps:

[0093] Step 31: Extract Gray-Level Co-occurrence Matrix (GLCM) texture features from the Y component; in the GLCM, each element (i,j) represents the co-occurrence frequency of two pixels with gray levels i and j in the image at a specific distance and direction. The GLCM for an image I of size M×N is calculated as follows:

[0094]

[0095] Where d and θ represent the distance and direction between pixels, respectively. Set d = 1, θ = 0°, 45°, 90°, 135°.

[0096] Step 32: After calculating the gray-level co-occurrence matrix of the Y channel, four key features are extracted: Contrast, Correlation, Energy, and Homogeneity, to reflect the texture attributes of the image from different perspectives. The specific calculation formulas are as follows:

[0097]

[0098] Where, μ x and μ y These are the mean values ​​of gray levels x and y, respectively, and σ x and σ y Let f1 and f2 represent the standard deviations of the rows and columns of the matrix, respectively. These four features are taken as the brightness feature f1, expressed as f1 = [Con]. Y Cor Y Ene Y Hom Y ].

[0099] Step 33: Extract Local Binary Pattern (LBP) histogram features from the Y component; first, perform local contrast normalization on the image to generate a Mean Contrast Normalized (MSCN) map, then calculate the gradient magnitude (GM) map and gradient direction (GO) map of the image. The specific calculation formula for MSCN is as follows:

[0100]

[0101] Where I(i,j) represents the pixel value at position (i,j) in the image, μ(i,j) and σ(i,j) represent the mean and standard deviation of the local region at that position, respectively, and C is a constant to avoid the denominator being zero. The local mean μ(i,j) and standard deviation σ(i,j) are calculated as follows:

[0102]

[0103] Where Y is the luminance component of the underwater image in the YCbCr space, and C is a constant used to avoid numerical instability. μ and σ are the mean and standard deviation of the local patch, respectively, and ω defines a Gaussian window per unit volume. N represents a local neighborhood window of image I, and ω(i,j) is a weighting coefficient used to control the degree of influence of pixels within the neighborhood.

[0104] Step 34: The specific calculation formulas for the gradient magnitude map GM and the gradient direction map GO are as follows:

[0105]

[0106] Where Y represents the luminance component of the underwater image in the YCbCr space, and G... x and G y This represents the Prewitt operator pattern in the horizontal x and vertical y directions. The value range of GO(i,j) is [-180°, +180°].

[0107] Step 35: Then, the local binary pattern (LBP) method is used to extract the local structural features of the image. Specifically, by extracting the local structural features of each center pixel P... c Its neighboring pixel P i By comparison, the LBP code of the center pixel is derived. Its calculation formula is as follows:

[0108]

[0109] Where ψ is P c and P i The threshold for binarizing the relative values ​​between them, where P is the number of neighboring pixels and R is the radius of the neighborhood.

[0110] Step 36: To achieve local rotation invariance in LBP encoding, we use a locally rotation invariant uniform LBP operator, which is based on a uniformity measure U quantized sequence ({T(p i -p c ),i=0,1,2,…,P-1}, such that:

[0111]

[0112] Step 37: Finally, the Local Binary Pattern (LBP) operator is applied to the Mean-Contrast Normalized (MSCN) plot, Gradient Magnitude (GM) plot, and Gradient Direction (GO) plot, respectively, to obtain the corresponding LBP histograms. The specific calculation formula is as follows:

[0113]

[0114] Where Λ∈{MSCN, GM, GO} indicates that LBP operation is applied to the feature maps of MSCN, GM and GO respectively, k∈{0,1,2,…,P+1} is the gray level of the LBP histogram, and H and W are the height and width of the image respectively.

[0115] Finally, three LBP histograms based on MSCN, GM, and GO were obtained. These three histogram features were used as the brightness feature f2, expressed as f2 = [H MSCN H GM H GO ].

[0116] Step 04: Extract chromaticity features from the Cb and Cr channels in the YcbCr color space, including moment statistics features and red shift features; Step 04, chromaticity feature extraction, includes the following steps:

[0117] Step 41: First, calculate the moment statistics of the Cb and Cr components as the chromaticity features of the underwater image. Specifically, by calculating the mean (μ, first moment), variance (σ, second moment), and skewness (s, third moment) on the mapping of the Cr and Cb components, the three-dimensional moment statistics based on chromaticity are obtained. The specific formula is as follows:

[0118]

[0119]

[0120] Where x ij Let N represent the first pixel of the j-th pixel in the i-th component of the image, and let N represent the number of pixels in the image.

[0121] Using the chromaticity-based three-dimensional moment statistics as the chromaticity feature f3, it is expressed as f3=[μ Cb ,σ Cb ,s Cb ,μ Cr ,σ Cr ,s Cr ].

[0122] Step 42: First, preprocess the training set images obtained in Step 01 using color correction methods, and adjust the pixel value distribution of the images using histogram equalization. Calculate the original histogram. For each gray level i, calculate the number of pixels with gray level i in the image, denoted as h(i). Then calculate the cumulative distribution function (CDF), which is the cumulative sum of the histograms and measures the cumulative distribution of gray levels. The formula for calculating the cumulative distribution function (CDF) for gray level i is as follows:

[0123]

[0124] The cumulative distribution function (CDF) is normalized to the range of 0 to L-1 (L = 256). The specific calculation formula is as follows:

[0125]

[0126] Finally, the original grayscale value of each pixel is replaced by the normalized cumulative distribution function (CDF) value to obtain the equalized image.

[0127] Step 43: Based on the equalized image, calculate the proportion of red pixels to total pixels in the red channel, as a quantification of the red shift feature. The specific calculation formula is as follows:

[0128]

[0129] Where P represents the total number of pixels in the image, R i R represents the value of the i-th pixel in the red channel. ki This represents the value of the red pixel after thresholding. R k The calculation is as follows:

[0130]

[0131] To quantify red-biased pixel values, a threshold of 0.6 is set, classifying pixels with red channel values ​​greater than 0.6 as red-biased pixels. The red offset feature value F is then calculated by determining the proportion of pixels identified as red in the image. R The red offset feature F R As a chromaticity feature, f4 is expressed as f4 = F R .

[0132] Step 05: Convert the training set images obtained in Step 01 into grayscale images and extract the salient features of the images; Step 05, salient feature extraction, includes the following steps:

[0133] Step 51: First, convert the training set images obtained in Step 01 into grayscale images (Gray). The calculation formula is as follows:

[0134]

[0135] Step 52: Then, use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows:

[0136]

[0137] The formula for calculating the gradient magnitude G is as follows:

[0138]

[0139] Among them, I i,j G represents the gray value of the (i,j)th pixel in a grayscale image. x and G y Let G represent the gradient in the horizontal and vertical directions, respectively, where G is the gradient magnitude and the gradient direction is arctan(Gy / Gx). To simplify subsequent calculations, the gradient magnitude is normalized to the range of 0-255 and converted to an integer type. Simultaneously, the gradient direction is quantized to the nearest 45-degree angle for LBP calculation. Finally, using the formulas in steps 35 and 36, the LBP algorithm is used to calculate the features of the integerized gradient magnitude map, yielding the feature H of the saliency map. s The LBP histogram features of the saliency plot are taken as the saliency feature f5, which is expressed as f5 = H. s .

[0140] Step 06: Based on the brightness features, chromaticity features, and saliency features obtained in Steps 03, 04, and 05, feature fusion is performed to obtain a combined feature vector. The Support Vector Machine (SVMrank) model is then used to train the model on the training set. Step 06 includes the following steps:

[0141] Step 61: Fuse the brightness features f1, f2, chromaticity features f3, f4 and saliency features f5 of the underwater image to obtain a multidimensional feature vector F, denoted as F = [f1, f2, f3, f4, f5];

[0142] Step 62: Use the multidimensional feature vector F of each image in the training set and the corresponding subjective rating MOS of the image as input data to build a support vector machine ranking (SVMrank) model for training;

[0143] Step 07: Using the trained model obtained in Step 06, perform underwater image quality assessment to obtain a unified viewpoint image quality assessment score. Step 7 includes the following steps:

[0144] Step 71: Input the test image into the trained model and obtain the multidimensional feature vector F of the image;

[0145] Step 72: The model will predict the quality of the image based on the multidimensional feature vector F of the image, and obtain an objective score of image quality.

[0146] Example 1:

[0147] To verify the generalizability of this invention for evaluating different underwater scenarios, two public underwater image datasets, SAUD and UID, were selected as test sets. The experimental results were compared and analyzed qualitatively and quantitatively with those of the UIQM (Human-visual-system-inspired underwater image quality measures), UCIQE (An underwater color imagequality evaluation metric), CCF (An imaging-inspired no-reference underwater color image quality assessment metric), FDUM (A reference-free underwater image quality assessment metric in the frequency domain), Twice Mixing (Twice mixing: a rank learning based quality assessment approach for underwater image enhancement), and ATUIQP (Underwater image quality assessment: Benchmark database and objective method) algorithms.

[0148] like Figure 2 , Figure 3 As shown in the figure, this invention provides a comparative image of underwater image quality assessment for images with excessive reddish tint. The experimental results show that the subjective quality scores of the six reddish images decrease sequentially from a to f. It can be observed that the quality assessment scores of this invention for reddish images also decrease sequentially from a to f, perfectly matching the ranking of subjective scores. The algorithm of this invention can perfectly predict the objective score of reddish underwater enhanced images.

[0149] like Figure 4 , Figure 5 As shown in the figure, this invention provides a comparison of underwater image quality assessment results under different scenarios. The experimental results show that the subjective quality scores of the eight underwater images from different scenarios decrease sequentially from a to h. It can be observed that the quality assessment score of this invention for this group of images also decreases sequentially from a to h, perfectly matching the ranking of subjective scores. The algorithm of this invention can perfectly predict the objective scores of underwater images under different scenarios.

[0150] This embodiment compares the experimental results of different algorithms using four objective indicators: Spearman's Rank Correlation Coefficient (SROCC), Kendall's Rank Correlation Coefficient (KROCC), Pearson's Linear Correlation Coefficient (PLCC), and Root Mean Square Error (RMSE). A SROCC value of 1 indicates excellent algorithm performance, while -1 indicates poor performance. The closer the SROCC value is to 1, the better the IQA algorithm's performance. A larger KROCC value indicates a stronger correlation between the two data points, resulting in better IQA algorithm performance; a smaller value indicates a weaker correlation. SROCC focuses on calculating the correlation between two vectors, while KROCC tends to assess the dependence strength between them. PLCC describes the linear correlation between two sets of data, ranging from -1 to 1. A PLCC value of zero indicates that the two sets of data are unrelated. A RMSE value closer to 0 indicates better IQA algorithm performance. As shown in Tables 1 and 2, this method outperforms other mainstream underwater image quality assessment methods, achieving the best results in SROCC, KROCC, PLCC, and RMSE.

[0151] Table 1 Comparison of the results of the present invention model with other advanced algorithms on SAUD.

[0152] Metrics Mean SROCC Mean KROCC Mean PLCC Mean RMSE UIQM 0.0267 0.0269 0.5124 1.2161 UCIQE 0.1443 0.0869 0.7464 0.9046 CCF -0.0298 -0.0237 0.6784 1.0083 FDUM 0.1553 0.0923 0.6526 1.0158 Twice Mixing 0.3760 0.2937 0.7443 0.9263 ATUIQP 0.2699 0.1967 0.6802 0.9943 Ours 0.6365 0.5050 0.8068 0.7715

[0153] Table 2 Comparison of the UID results of the model of this invention with other advanced algorithms

[0154]

[0155]

[0156] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the descriptions of each embodiment have their own emphasis; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for underwater image quality assessment based on multi-feature fusion, characterized in that, Includes the following steps: Step 01: Obtain multiple original underwater images and their corresponding enhanced image datasets, and randomly divide the multiple original underwater images and their corresponding enhanced image datasets into training set images and test set images in a 50%:50% ratio; Step 02: Convert the training set images from the RGB color space to the YcbCr color space; Step 03: Extract luminance features from the Y channel in the YcbCr color space; the extraction includes: gray-level co-occurrence matrix texture features and local binary mode histogram features; Step 04: Extract chromaticity features from the Cb and Cr channels in the YcbCr color space; the extraction includes: moment statistics features and red shift features; Step 05: Convert the training set images to grayscale images and extract the salient features of the images; Step 05, salient feature extraction, includes the following steps: Step 51: First, convert the training set images into grayscale images (Gray). The calculation formula is as follows: Step 52: Then, use the Sobel operator to calculate the horizontal and vertical gradients of the image. The calculation formula is as follows: The formula for calculating the gradient magnitude G is as follows: in, Represents the grayscale image. The grayscale value of each pixel and Let G represent the gradient in the horizontal and vertical directions, respectively, where G is the gradient magnitude and the gradient direction is... ; The gradient magnitude is normalized to the range of 0-255 and converted to an integer type; at the same time, the gradient direction is quantized to the nearest 45-degree angle. The LBP algorithm is used to calculate the features of the integerized gradient magnitude map to obtain the features of the saliency map. The LBP histogram features of the saliency map are used as saliency features. , represented as ; Step 06: The brightness feature, chromaticity feature, and saliency feature are fused to obtain a combined feature vector. The Support Vector Machine (SVMrank) ranking algorithm is then used to train the model on the training set images to obtain the trained model. Step 06 includes the following steps: Step 61: Extract brightness features from underwater images , Color characteristics , and saliency features By fusing the features, a multidimensional feature vector F is obtained, denoted as: ; Step 62: Use the multidimensional feature vector F of each image in the training set and the corresponding subjective rating MOS of the image as input data to build a support vector machine ranking model for training; Step 07: Based on the trained model obtained in Step 06, perform underwater image quality assessment to obtain an image quality assessment score.

2. The underwater image quality assessment method based on multi-feature fusion according to claim 1, characterized in that, The step 02, converting the image color space, is as follows: Step 21: Convert the training set images from the RGB color space to the YcbCr color space: Where Y represents the luminance component of the image in the YcbCr color space, Cb represents the blue chromaticity component of the image in the YcbCr color space, and Cr represents the red chromaticity component of the image in the YcbCr color space.

3. The underwater image quality assessment method based on multi-feature fusion according to claim 1, characterized in that, Step 03, brightness feature extraction, includes the following steps: Step 31: Extract Gray-Level Co-occurrence Matrix (GLCM) texture features on the Y component; in the GLCM, each element... This represents the co-occurrence frequency of two pixels with gray levels i and j in an image at a specific distance and direction; for a size of The GLCM of image I is calculated as follows: ； in, and Represents the distance and direction between pixels, respectively; set ; Step 32: After calculating the gray-level co-occurrence matrix of the Y channel, extract four key features: Contrast, Correlation, Energy, and Homogeneity to reflect the texture attributes of the image from different perspectives. in, and Let x and y represent the mean values ​​of the gray levels, respectively. and Let these represent the standard deviations of the rows and columns of the matrix, respectively; these four features are used as brightness features. , represented as ; Step 33: Extract local binary pattern histogram features on the Y component; first, perform local contrast normalization on the image to generate a mean contrast normalized map, then calculate the gradient magnitude map and gradient direction map of the image; the mean contrast normalized map is as follows: in, Indicates the position in the image Pixel values; and These represent the mean and standard deviation of a local region at that location, respectively, with C being a constant to avoid the denominator being zero; local mean and standard deviation The calculation is as follows: Where Y represents the brightness component of the underwater image in the YCbCr space, and C represents a constant used to avoid numerical instability. and These represent the mean and standard deviation of local plaques, respectively. N represents a Gaussian window per unit volume; N represents a local neighborhood window of image I. These are weighting coefficients used to control the degree of influence of pixels within the neighborhood; Step 34: The formulas for calculating the gradient magnitude map GM and the gradient direction map GO are as follows: Where Y represents the luminance component of the underwater image in the YCbCr space. and Represents the Prewitt operator patterns in the horizontal x and vertical y directions; The range of values ​​is ; Step 35: Then, the local binary pattern method is used to extract the local structural features of the image; by extracting each center pixel... Its neighboring pixels By comparison, the local binary pattern encoding of the center pixel is derived, and the calculation formula is as follows: ； in, and The threshold for binarizing the relative values ​​between them, where P represents the number of neighboring pixels and R represents the radius of the neighborhood; Step 36: To achieve local rotation invariance of LBP encoding, a locally rotation invariant uniform LBP operator is introduced. This LBP operator is based on a uniformity measure U quantized sequence. , so that: ； Step 37: Finally, apply the local binary mode operator to the mean-contrast normalized map, gradient magnitude map, and gradient direction map respectively to obtain the corresponding LBP histogram; in, This indicates that LBP operations are applied to the MSCN, GM, and GO feature maps respectively; H represents the gray level of the LBP histogram, and H and W represent the height and width of the image, respectively. Finally, three LBP histograms based on MSCN, GM, and GO were obtained; the features of these three histograms were used as brightness features. , represented as .

4. The underwater image quality assessment method based on multi-feature fusion according to claim 1, characterized in that, Step 04, chromaticity feature extraction, includes the following steps: Step 41: First, calculate the moment statistics of the Cb and Cr components as chromaticity features of the underwater image; specifically, obtain the chromaticity-based three-dimensional moment statistics by calculating the mean, variance, and skewness on the Cr and Cb component mappings. in, Let N represent the first pixel of the j-th pixel in the i-th component of the image, and N represent the number of pixels in the image. Using chromaticity-based three-dimensional moment statistics as chromaticity features , represented as ; Step 42: First, preprocess the training set images using color correction methods, and use histogram equalization to adjust the pixel value distribution of the images; calculate the original histogram, for each gray level... i Calculate the gray levels in the image. i The number of pixels, denoted as Then calculate the cumulative distribution function for gray levels. i The cumulative distribution function (CDF) is calculated as follows: Normalize the value of the cumulative distribution function to 0. Within the specified range, the specific calculation formula is as follows: Finally, the original gray value of each pixel in the image is replaced by the normalized cumulative distribution function value to obtain the equalized image; Step 43: Based on the equalized image, calculate the proportion of red pixels in the red channel to the total number of pixels, as a quantitative indicator of the red shift feature. Where P represents the total number of pixels in the image. Indicates the first in the red channel The value of each pixel. This represents the value of the red pixel after thresholding. The calculation is as follows: To quantify red-biased pixel values, a threshold is set, and pixels with red channel values ​​greater than the threshold are considered red-biased pixels. The red offset feature value is obtained by calculating the proportion of pixels identified as red in the image. ; Red offset feature As a chromaticity feature , represented as .

Images