An underwater image quality assessment method based on image feature weighted fusion analysis

Abstract

The present invention provides an underwater image quality evaluation method based on image feature weighted fusion analysis, which belongs to the technical field of underwater image data processing and analysis; the present invention takes the color distortion and blur distortion of underwater images as the main quality evaluation factors of underwater images, and proposes a reference-free underwater image color evaluation method based on the Lab color model in view of the color attenuation characteristics commonly present in underwater images, and proposes to apply the image clarity based on LPC to the underwater image evaluation method in view of the blur distortion characteristics of underwater images, and then weightedly fuses the evaluation of these two parts to form a reference-free underwater image quality evaluation algorithm, which performs well in performance analysis and can accurately and effectively evaluate underwater image data. Compared with other single feature quality evaluation methods, the algorithm of the present invention has better effectiveness and robustness, and solves the problems of these evaluation methods being dependent on equipment and lacking universality.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater image data processing and analysis, and in particular relates to an underwater image quality evaluation method based on image feature weighted fusion analysis. Background Art

[0002] In today's society, the ocean is the focus of human attention, and underwater imaging technology is an important means for humans to understand, develop and protect the ocean. This technology has been widely used in the technical fields of various scenes in the water. Obtaining underwater high signal-to-noise ratio imaging is a basic requirement for ocean optical detection. A large number of activities related to underwater operations urgently need underwater high signal-to-noise ratio imaging technology to provide more effective information. However, due to the particularity and complexity of the marine environment, underwater detection operations are severely affected by the absorption and scattering characteristics of water bodies. Backscattering forms strong background noise, which causes the image contrast to decrease, thereby reducing the image signal-to-noise ratio. Forward scattering causes the points on the object surface to become spots on the image surface during imaging, making the image blurred and the resolution reduced. These characteristics are particularly prominent when detecting in turbid water areas. Therefore, effectively overcoming the influence of these effects of water media on imaging quality is a key link in underwater detection.

[0003] The research on underwater imaging technology requires the support of underwater image quality evaluation technology to be a scientific and complete process. However, due to the complexity of water bodies and the variability of underwater environments, image quality evaluation methods in the air cannot be fully applicable to underwater environments. Most of the existing underwater image quality evaluation methods are proposed for a specific underwater image restoration or enhancement algorithm. These traditional algorithms have poor adaptability. Therefore, it is difficult to obtain distortion-free images as references in underwater environments. Therefore, the research on image quality evaluation methods for acquired high-quality underwater imaging data is of great significance to image processing technology and design optimization of imaging systems. Summary of the invention

[0004] In view of the above problems, the present invention proposes an underwater image quality evaluation method based on image feature weighted fusion analysis, comprising the following steps:

[0005] Step 1, obtaining an optimized underwater image and preprocessing the image into a fixed size;

[0006] Step 2, performing color space conversion on the preprocessed underwater image, converting the RGB image into Lab color space, and obtaining a Lab color space image;

[0007] The variance and mean of the 𝑎 channel and the b channel of the obtained image are calculated respectively; and the logarithmic operation is introduced into the color measurement of the underwater image to obtain the evaluation value of the color feature of the underwater image without reference;

[0008] Step 3, the preprocessed underwater image is processed by log-Gabor filters of N scales and M directions without any downsampling process;

[0009] There are MN complex coefficients at each spatial position in all directions and all scales, and then the LPC intensity is calculated at a specific direction and a specific position in a specific space;

[0010] The obtained LPC intensity is fused with the weight allocated in the specific space to obtain the evaluation value of the clarity feature of the reference-free underwater image;

[0011] Step 4: Conduct subjective evaluation on the preprocessed underwater image, take the average score as the MOS value of the image, and perform nonlinear fitting with the objective value of the image feature to obtain the optimal weight coefficient;

[0012] Step 5, performing linear weighted summation on the evaluation value of the underwater image color feature, the evaluation value of the underwater image clarity feature, and the optimal weight coefficient to obtain an optimized evaluation score of the underwater image quality.

[0013] Preferably, the conversion of the RGB image to the Lab color space in step 2 is specifically as follows:

[0014] The first step is to convert the RGB space to the XYZ space. The conversion formula is as follows:

[0015]

[0016] Among them, matrix M is the constant matrix of color space conversion;

[0017] Then convert from XYZ space to Lab space, the conversion formula is as follows:

[0018]

[0019]

[0020]

[0021] in is the conversion function.

[0022] Preferably, the specific process of obtaining the evaluation value of the color feature of the non-reference underwater image is:

[0023] Calculate the variance and mean of the 𝑎 channel and the b channel respectively, Variance of the channel and mean The calculation formula is as follows:

[0024]

[0025]

[0026] Variance of the channel and mean The calculation formula is as follows:

[0027]

[0028]

[0029] Where N is the number of samples, and is the value of 𝑎 and b of the pth color;

[0030] The subjective feeling of the human eye is proportional to the logarithm of the objective stimulus. By introducing logarithmic operation into the color measurement of underwater images, the evaluation value of the color feature of the reference-free image based on the Lab color model is obtained:

[0031]

[0032] in, Respectively refer to The variance and mean of the channel, Respectively refer to Variance and mean of the channel.

[0033] Preferably, the specific process of performing log-Gabor filter processing on the pre-processed underwater image in step 3 with N scales and M directions is as follows:

[0034] First, for the image of size M×N obtained by preprocessing , perform Fourier transform to frequency domain, the formula is as follows:

[0035]

[0036] in, is the image pixel value in the spatial domain; is the Fourier transform result in the frequency domain; is a complex exponential function with a frequency of The sine and cosine components of ;

[0037] After that, construct Log-Gabor filters of N scales and M directions. The filters are as follows:

[0038]

[0039] in, is the center direction of the filter, is the control direction bandwidth, the center frequency Determines the frequency center and bandwidth parameters of the filter Controls the width of the frequency response;

[0040] Finally, multi-scale and multi-directional Log-Gabor filtering is performed on the frequency domain representation of the image to obtain 𝑀𝑁 complex coefficients at each spatial position in all directions and all scales, which are used Represents the complex coefficient at the 𝑖th scale, the 𝑗th direction, and the 𝑘th spatial position, and then inverse transforms each filtered result back to the spatial domain and saves the result.

[0041] Preferably, the specific process of calculating the LPC intensity in step 3 is:

[0042] In wavelet transform, position The characteristic signal is , then the signal is represented as: ,use Represents the complex wavelet transform of the signal; According to the complex wavelet transform theory and convolution theorem, the complex wavelet transform is obtained Phase :

[0043]

[0044] in is the scale factor, is the displacement factor;

[0045] In continuous wavelet transform Discrete sampling of discrete wavelet transform is performed on the , between every two adjacent scales, the scale factor Doubled, the spatial sampling rate is halved; at three consecutive scales of the two-dimensional image signal, the neighboring coefficients are , according to the principle of local coherence, from the coarser scale coefficient Get a finer scale factor The predicted phase of , where the first set of coefficients is a , the second set of coefficients is , i and j is an integer from 1 to 2, and the third set of coefficients is , i and j is an integer from 1 to 4, predicting the phase It is expressed as:

[0046]

[0047] Therefore, for Nearby Any extraction The phase of the sample coefficients and the corresponding predictions:

[0048]

[0049] Quantifying the strength of LPC for:

[0050]

[0051] Select a specific set of samples , the weight set corresponding to this sample is ,make:

[0052]

[0053] Get the weight set:

[0054]

[0055] After simplification, Directions and The LPC at each position is expressed as:

[0056]

[0057] The complex coefficients It is The scale, Directions and The complex coefficients of the spatial positions, represents the real part of a complex number; each spatial position is considered in a weighted form, with the weight being the first scale factor The magnitude of further indicates the strength of the k-th spatial LPC as:

[0058]

[0059] in is a constant, is the first scale factor Directions and The complex coefficients of the spatial positions.

[0060] Preferably, the LPC intensity obtained in step 3 is fused with the weight allocated in the specific space to obtain an evaluation value of the clarity feature of the non-reference underwater image, specifically:

[0061] is assigned to The weight of the spatial LPC value is calculated as follows:

[0062]

[0063] Finally, the evaluation value of the clarity feature of the no-reference underwater image is obtained as follows:

[0064]

[0065] K is the total number of spaces, is the adjustment coefficient.

[0066] Preferably, the specific process of step 4 is:

[0067] The image quality levels of the acquired pre-processed underwater images are divided into five levels, namely "very poor", "poor", "average", "good" and "very good"; the subjective quality evaluation score is 0-10 points, where 0-2 points represent "very poor", 2-4 points represent "poor", 4-6 points represent "average", 6-8 points represent "good", and 8-10 points represent "very good"; the testers give the subjective evaluation score of the image between 0-10 points, and the score is accurate to two decimal places; finally, the average of the testers' scores on the image is taken as the mean opinion score MOS value of the image, which is defined as the following formula:

[0068]

[0069] in, Refers to Ratings from testers, is the total number of observers; Keep two decimal places; after obtaining the MOS value, perform nonlinear fitting with the objective value of the obtained image feature.

[0070] Preferably, the specific process of step 5 is:

[0071] UCIQ is used to represent the underwater image quality evaluation score, and its expression is:

[0072]

[0073] in, is the evaluation value of the color feature of the underwater image without reference, is the evaluation value of the clarity feature of the underwater image without reference, are the weight coefficients for color evaluation and clarity evaluation respectively.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] 1. Compared with other image quality evaluation methods, the present invention can obtain objective and accurate evaluation values ​​without the need for reference images. The results are more consistent with the evaluation of human visual perception, solving the problem that it is difficult to obtain distortion-free images as reference images during underwater imaging, and has better applicability.

[0076] 2. Due to the serious color attenuation of underwater images, the present invention performs a reference-free color evaluation of underwater images based on the color distortion characteristics of underwater images and the Lab color model that is more in line with human vision. The evaluation value is more accurate than that of other color models and does not depend on the device. The performance of this algorithm is better than that of other algorithms.

[0077] 3. Due to the complexity of the underwater environment, many algorithms have poor evaluation effects on underwater images and are dependent on the image content. The present invention applies the LPC-based clarity evaluation method to underwater and conducts experiments to verify the effectiveness of the method.

[0078] 4. Compared with the subjective quality evaluation method, the algorithm of the present invention has the advantages of high efficiency and real-time performance, which solves the problem that the subjective evaluation method is time-consuming, labor-intensive and heavily dependent on manual experience.

[0079] 5. Compared with the underwater image quality evaluation method based on a single feature, the present invention considers multiple factors of quality evaluation, performs component analysis on multiple features of the image, and then performs weighted fusion analysis on the image features. It is more objective and accurate, and has better robustness.

[0080] 6. Since most underwater image quality evaluation methods are basically proposed for a specific underwater image restoration or enhancement algorithm, they lack universality and are difficult to accurately evaluate classified data sets. Compared with these traditional methods, the present invention has excellent performance and objective and accurate evaluation. Experiments have also verified the effectiveness and robustness of the present invention in evaluating acquired underwater images, providing an effective reference for underwater image analysis and having high practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 It is a flow chart of the underwater image quality evaluation method based on image feature weighted fusion analysis of the present invention.

[0082] Figure 2 This is a schematic diagram of the digital light source imaging system experimental platform;

[0083] Figure 3 It is a schematic diagram of the local phase coherent structure;

[0084] (a) is a schematic diagram of a sharp step edge; (b) is a schematic diagram of a blurred step edge.

[0085] Figure 4 Schematic diagram of discrete wavelet transform sampling grid in continuous wavelet transform domain;

[0086] Among them, (a) is a one-dimensional sampling schematic diagram; (b) is a two-dimensional sampling schematic diagram.

[0087] Figure 5 is an underwater color plate image;

[0088] Among them, (a) is a schematic diagram of an imaging distance of 60 cm, (b) is a schematic diagram of an imaging distance of 90 cm, (c) is a schematic diagram of an imaging distance of 150 cm, and (d) is a schematic diagram of an imaging distance of 280 cm.

[0089] Figure 6 Schematic diagram of underwater image and its LPC.

[0090] Figure 7 Scatter plot of objective and subjective ratings of the algorithm;

[0091] Among them, (a) is the schematic diagram of the UCIQ scatter plot; (b) is the JNBM scatter plot; (c) is the CPBD scatter plot; and (d) is the LPC scatter plot.

[0092] The main hardware description is as follows:

[0093] 1. Experimental pool; 2. Integrated platform based on digital light source imaging; 3. Underwater discrete light source array; 4. Underwater camera; 5. Schematic diagram of digital light source array light field; 6. Underwater imaging target; 7. Experimental water body. DETAILED DESCRIPTION

[0094] The invention will be further described below in conjunction with specific embodiments.

[0095] like Figure 1 As shown, a flow chart of the underwater image quality evaluation method based on image feature weighted fusion analysis of the present invention is given.

[0096] This embodiment is experimented on a data set acquired based on a digital light source imaging system. The performance of this method on this data set exceeds that of other methods. The present invention also conducts an experimental study on ocean data to verify its effectiveness and robustness.

[0097] 1. Underwater discrete light source array to build underwater imaging system based on digital light source

[0098] The underwater environment is very complex. Compared with the air, high-quality imaging is very difficult. Active illumination of underwater targets is crucial during imaging. Underwater discrete light source arrays can create a rich combination of light sources, thereby effectively suppressing the impact of backscattering in underwater imaging on underwater imaging quality. Figure 2As shown, the underwater discrete light source array, together with the underwater camera module and the fixed platform, constitutes a digital light source imaging system. The system includes six small-angle light sources and a six-channel two-degree-of-freedom pan-tilt, where each single-channel pan-tilt can rotate horizontally and vertically. The underwater camera module mainly includes a 4K monocular camera and a Gopro sports camera. The system tracks, illuminates and images underwater targets to obtain underwater data acquired by the digital light source imaging system. The following table shows the performance parameters of the digital light source array:

[0099] Table 1 Performance parameters of underwater discrete light source array

[0100]

[0101] The acquired data is preprocessed, the video data is processed into images frame by frame, and each frame is cropped to a fixed size. The preprocessed data is classified, and an underwater data set based on the digital light source imaging system is constructed for subsequent testing.

[0102] 2. Conversion between RGB color model and Lab color model:

[0103] The first step is to convert the RGB space to the XYZ space. The conversion formula is as follows:

[0104]

[0105] Among them, matrix M is the constant matrix of color space conversion, after being substituted into:

[0106]

[0107] Then convert from XYZ space to Lab space, the conversion formula is as follows:

[0108]

[0109]

[0110]

[0111] in:

[0112]

[0113] 3. Calculate the evaluation value of the color feature of the non-reference image:

[0114] The conversion from RGB color model to Lab color model is based on the research of color evaluation theory of underwater images without reference, and taking into account that the subjective feeling of human eyes is proportional to the logarithm of objective stimulus, the logarithmic operation is introduced into the color measurement of underwater images. Therefore, the color evaluation of underwater images without reference based on Lab color model is obtained as follows:

[0115]

[0116] in, Respectively refer to The variance and mean of the channel, Respectively refer to Variance and mean of the channel. Variance of the channel and mean The calculation formula is as follows:

[0117]

[0118]

[0119] Variance of the channel and mean The calculation formula is as follows:

[0120]

[0121]

[0122] Where N is the number of samples, and is the pth color a and b value.

[0123] 4. Perform log-Gabor filter processing in 𝑁 scales and 𝑀 directions:

[0124] First, for the image of size M×N obtained by preprocessing , perform Fourier transform to frequency domain, the formula is as follows:

[0125]

[0126] in, is the image pixel value in the spatial domain. is the Fourier transform result in the frequency domain. is a complex exponential function, which is the sine and cosine components of this frequency.

[0127] After that, construct Log-Gabor filters of N scales and M directions. The filters are as follows:

[0128]

[0129] in, is the center direction of the filter, is the control direction bandwidth. Center frequency Determines the frequency center and bandwidth parameters of the filter Controls the width of the frequency response.

[0130] Finally, multi-scale and multi-directional Log-Gabor filtering is performed on the frequency domain representation of the image to obtain 𝑀𝑁 complex coefficients at each spatial position in all directions and all scales, which are used Represents the complex coefficient at the 𝑖th scale, the 𝑗th direction, and the 𝑘th spatial position. Each filtered result is then inverse transformed back to the spatial domain and the result is saved.

[0131] 5. Calculate the LPC intensity value:

[0132] Complex wavelet transform at position The characteristic signal is , then the signal Can be expressed as: , the general formula of complex wavelet transform can be written as:

[0133]

[0134] in is the scale factor, is the displacement factor, wavelet basis function The mother wavelet The expression is obtained by expansion and displacement:

[0135]

[0136] in is the center frequency of the modulated bandpass filter, is a slowly varying, non-negative, symmetric envelope function.

[0137] Through the convolution theorem, and the shift and scaling properties of Fourier transform, it can be derived that:

[0138]

[0139] in yes The Fourier transform of . The phase depends on If we assume is scale-invariant and can be expressed as:

[0140]

[0141] in, yes The real function of It doesn't matter. Substitute it into the operation and we get:

[0142]

[0143] Assumptions The structure of It is expressed as:

[0144]

[0145] because and are all real numbers, and we can get The phase relationship is:

[0146]

[0147] From the above formula, we can see that there is a strong phase coherence between scale and displacement. Figure 3 As shown in (a), The isophase contour of the plane will converge exactly on the feature The phase-coherent relationship passes through the center vertical line The phase is predicted to form a subset of LPC. Figure 3 As can be seen in (b), blurring does not change the central vertical line , but distorts the LPC phase map far from the center. Phase consistency does not require scale invariance features, but the premise of the derivation of the LPC phase map is It is a scale-invariant signal. Therefore, phase consistency cannot measure blur, but phase coherence can measure image blur.

[0148] Discrete wavelet transform corresponds to continuous wavelet transform Discrete sampling of Figure 4 A typical sampling grid is shown in Figure . Figure 4 As shown in (a), between every two adjacent scales, the scale factor Double, the spatial sampling rate is halved. Consider three consecutive scales and divide the adjacent coefficients Grouping, then the coarser scale coefficients Can predict finer scale coefficients well The phase of , that is, the local phase satisfies the phase coherence relationship. The predicted phase It can be expressed as:

[0149]

[0150] In two-dimensional space, Figure 4 As shown in (b), according to the coarser scale factor The phase coefficient of the finer scale The predicted expression for the phase is:

[0151]

[0152] The prediction accuracy can be used as a local measure of the strength of LPC. The method of complex wavelet transform domain LPC based on controllable pyramid decomposition is adopted. At each spatial position, the measure of LPC strength is defined as:

[0153]

[0154] in, and Respectively Direction The actual phase and predicted phase of each coefficient. This value is maximized when the phase prediction is most accurate. This metric specifies that the weights of coefficients in different directions are summed, so directions containing more energy are given higher weights.

[0155] 6. Calculate the evaluation value of the clarity feature of the non-reference underwater image:

[0156] In one-dimensional space, pulses and step edges are the most common sharp features. In the case of is a non-zero constant, is an impulse function. For all , The Fourier transform of ,therefore and . Substituting in, we get:

[0157]

[0158] Derivation The phase can be expressed as:

[0159]

[0160] In the ideal step pulse case, ,in is a non-zero constant, is a step function.

[0161] The Fourier transform of , ,therefore , we can get:

[0162]

[0163] According to the integral characteristics of Fourier transform, we can get:

[0164]

[0165] because is There are slowly changing functions near and . So we can get:

[0166]

[0167] Therefore, the phase is calculated as:

[0168]

[0169] Combining the above derivation, we can get the LPC relationship expression:

[0170]

[0171] in is an integer that depends on the sharp feature. In particular, it is equal to 0, 2, -1, and 1 for a positive pulse, a negative pulse, an ideal rising edge step, and an ideal falling edge step, respectively. The above equation gives the phase pattern model around the sharp image feature. Nearby Any extraction The phase of the sample coefficients and the corresponding predictions:

[0172]

[0173] The strength of LPC can be quantified by evaluating how close the actual phases of these coefficients are to their optimally predicted phases. :

[0174]

[0175] in is a measure of the closeness between the actual and predicted phase samples given in vector form. The general formula stated in this equation allows for arbitrary choices in scale space In practice, a set of samples can be selected at a specific location and scale in the scale space so that the LPC calculation can be simplified. Specifically, if a set of samples is found And the corresponding weight set ,make:

[0176]

[0177] Therefore, a simplified LPC strength measurement formula can be obtained:

[0178]

[0179] The LPC value range is between -1 and 1. The obtained LPC intensity is combined with the weights allocated in this specific space and the complex coefficients obtained in the above steps. , representing the The scale, Directions and The complex coefficients of the spatial positions. Then Directions and For each position, the LPC strength can be calculated:

[0180]

[0181] in represents the real part of a complex number. This LPC measure considers each spatial position in a weighted form. The weight is determined by the first scale factor The magnitude of , so directions containing more energy are given higher weights:

[0182]

[0183] in is a constant to prevent the denominator from being zero when the coefficient amplitude is zero.

[0184] Finally, the evaluation value of the clarity feature of the no-reference underwater image is obtained as follows:

[0185]

[0186] in, is assigned to The weight of the spatial LPC value, K is the total number of spaces, is the adjustment coefficient, It is calculated by the following exponential decay function:

[0187]

[0188] 7. Calculate the MOS value of the subjectively evaluated image:

[0189] For the acquired pre-processed underwater images, the image quality levels are divided into five levels, namely "very poor", "poor", "average", "good", and "very good". The subjective quality evaluation score is 0-10 points, where 0-2 points represent "very poor", 2-4 points represent "poor", 4-6 points represent "average", 6-8 points represent "good", and 8-10 points represent "very good". The tester gives a subjective evaluation score for the image between 0 and 10 points, and the score can be accurate to two decimal places. Finally, the average of the tester's scores for the image is taken as the MOS (mean opinion score) value of the image, and the following formula is defined:

[0190]

[0191] in, Refers to Ratings from testers, is the total number of observers. Keep two decimal places. After obtaining the MOS value, perform nonlinear fitting with the objective value of the obtained image feature.

[0192] 8. Calculate the underwater image quality evaluation score UCIQ of the algorithm of the present invention:

[0193] The obtained image feature evaluation value and the obtained optimal weight coefficient are linearly weighted and the final underwater image quality evaluation score is obtained. The algorithm of the present invention is represented by UCIQ (underwater color image quality), and its expression is:

[0194]

[0195] in, is a reference-free underwater image color evaluation, It is a reference-free underwater image clarity evaluation. are the weight coefficients for color evaluation and clarity evaluation respectively.

[0196] 9. Comparison of experimental results:

[0197] In order to verify the performance of the reference-free underwater image color evaluation algorithm based on the Lab color model of the present invention, several commonly used color models are selected: HSI color model, lαβ color model, and YIQ color model for comparative experiments. Since the underwater image color evaluation algorithm is based on a color model that satisfies the human visual system, and the RGB color model does not conform to the human visual system, the RGB color space is no longer considered here. This paper selects some pictures taken underwater and conducts underwater image color evaluation experiments based on Lab, lαβ, YIQ, and HSI color models, such as Figure 5These are color plate images collected in a pool using a digital light source imaging system. The imaging distance d of (a) is 60 cm, the imaging distance d of (b) is 90 cm, the imaging distance d of (c) is 150 cm, and the imaging distance d of (d) is 280 cm. The imaging distances of the four images gradually increase. In an underwater environment, as the imaging distance increases, the color attenuation of the underwater image becomes more serious. Table 2 gives the Figure 5 The color evaluation values ​​based on different color models are as follows:

[0198] Table 2 Color scores of underwater color plate images based on different color models

[0199]

[0200] As can be seen from Table 2, the reference-free underwater image color evaluation algorithm based on the Lab color model can perform relatively accurate color distortion evaluation on both pool images and ocean images. As the imaging distances of the four images (a), (b), (c), and (d) gradually increase, the color attenuation of the underwater image gradually increases, and the color evaluation value based on the Lab color model also gradually decreases, which is consistent with the color attenuation trend of the underwater image. When the color evaluation algorithm based on the lαβ color model evaluates underwater color plate images, the color evaluation score does not completely match the color attenuation trend of the underwater image. When the color evaluation algorithm based on the YIQ color model and the HSI color model evaluates underwater images at different imaging distances, the evaluation score fluctuates very little, and the score does not match the color attenuation trend of the underwater image. Therefore, through the above verification, the underwater image color evaluation method based on the Lab color model of the present invention can effectively measure the color distortion of underwater images.

[0201] In order to verify the effectiveness of the LPC-based no-reference underwater image clarity evaluation algorithm of the present invention, the present invention conducted an underwater image clarity evaluation experiment. Figure 6 The images (a), (b), (c), and (d) are images acquired in a water pool using a digital light source imaging system, and (a´), (b´), (c´), and (d´) are the corresponding LPC images. The imaging distances of the four images (a), (b), (c), and (d) gradually increase, and d represents the imaging distance. By comparing the four images, it can be found that as the imaging distance increases, the degree of image blurring and distortion increases, and the image LPC relationship weakens. The final image severe mode is shown in Figure (d), and it can be seen from Figure (d´) that the LPC relationship is completely interrupted. The LPC-based no-reference underwater image clarity evaluation accurately verifies this point.

[0202] In order to verify the effectiveness and robustness of the underwater image quality evaluation method based on weighted fusion analysis of image features proposed in this invention, a water pool experiment was designed to collect a large number of underwater images, and then the quality evaluation was performed using the algorithm of this invention. At the same time, several classic image quality evaluation algorithms were selected for comparison, and the experimental results were analyzed to verify whether the algorithm of this invention can provide an objective, effective and accurate evaluation for the study of underwater imaging. The selected evaluation indicators are as follows:

[0203] (1) Spearman rank correlation coefficient (SROCC)

[0204] SROCC can measure the monotonicity of the algorithm prediction. SROCC performs linear correlation analysis on the ranks of two vectors and uses vector and vector Represent the subjective score of human eyes and the objective score of algorithm respectively. First, vector and Sort the elements in from largest to smallest, and then calculate SROCC according to the following formula:

[0205]

[0206] in, Representative is The position after sorting, represent The position after sorting.

[0207] (2) Pearson linear correlation coefficient (PLCC)

[0208] PLCC measures the accuracy of the algorithm by calculating the minimum average error rate between the algorithm score and the subjective score of the human eye. The calculation formula is as follows:

[0209]

[0210] in, and Respectively The mean and standard deviation of and Respectively The mean and standard deviation of .

[0211] (3) Kendall rank correlation coefficient (KROCC)

[0212] KROCC also measures the monotonicity of the algorithm's predictions. Its calculation formula is as follows:

[0213]

[0214] in, represents the number of harmonious pairs in the data sequence, Represents the number of discordant pairs in the data sequence. Assume and is any pair of data in two data sequences, if it satisfies and , or satisfy and , they are said to be harmonious; if and , or satisfy and , then it becomes disharmony; if it satisfies or , it is neither harmonious nor disharmonious.

[0215] (4) Mean square error (RMSE)

[0216] RMSE calculates the absolute error between the subjective and objective ratings:

[0217]

[0218] in, MOS or DMOS values ​​represent the subjective rating of human eyes. Represents the objective score of the algorithm. Before calculating RMSE, the objective score of the algorithm and the subjective score of the human eye need to be normalized.

[0219] When calculating the two evaluation indicators PLCC and RMSE, regression analysis is required to establish a nonlinear mapping relationship between the subjective evaluation value of the human eye and the objective evaluation value of the algorithm. The closer the SROCC, KROCC, and PLCC values ​​are to 1, and the closer the RMSE is to 0, the closer the algorithm evaluation results are to the subjective scores of the human eye, indicating that the algorithm has a higher accuracy, and vice versa.

[0220] In addition, the present invention compares two classic blur underwater quality assessment algorithms, a wavelet domain image blind watermarking algorithm based on just noticeable distortion model (JNBM) and a reference-free sharpening factor (CPBD), which can be used to quantitatively evaluate the blurriness of an image.

[0221] The performance of the algorithm is better measured by calculating the above four image quality evaluation indicators (SROCC, KROCC, PLCC, RMSE). At the same time, it is compared with the fuzziness evaluation method JNBM, CPBD and the unimproved LPC algorithm. The comparison results are shown in Table 3.

[0222] Table 3 Comparison results of underwater image test obtained by digital light source imaging system

[0223]

[0224] As can be seen from Table 3, the SROCC, KROCC, and PLCC values ​​of the UCIQ algorithm are higher than those of the JNBM algorithm, the CPBD algorithm, and the LPC algorithm, and the RMSE value is smaller than that of the other three algorithms. SROCC, KROCC, and PLCC all calculate correlation coefficients. The higher the correlation coefficient, the stronger the correlation between the algorithm evaluation value and the subjective score of the human eye. The RMSE value calculates the absolute error between the algorithm score and the subjective score of the human eye. The smaller the RMSE value, the smaller the error. From the comparison results, it can be seen that when used for underwater image quality evaluation, the performance of this algorithm is better than the other three algorithms.

[0225] like Figure 7 As shown in the figure, the scatter plot of the subjective score of the image collected in the pool by the digital light source imaging system and the objective score of the algorithm is given, and the nonlinear fitting is performed using the logistic function to generate the curve as shown in the figure. For the convenience of comparison, the scores of the four algorithms are normalized to the range of 0-1. Figure 7 (a) is the result of the underwater image quality evaluation algorithm proposed in this paper; Figure 7 (b) in the figure is the result of the JNBM algorithm; Figure 7 (c) in the figure is the evaluation result using the CPBD algorithm; Figure 7 (d) in the figure is the evaluation result using the unmodified LPC algorithm. It can be seen that the scatter point fitting curve in Figure (a) is smoother, and the scatter points are more concentrated around the curve. Therefore, compared with the other three algorithms, the accuracy and monotonicity of the method for evaluating the quality of underwater images are the best. Through the above experiments, the effectiveness and robustness of the algorithm of the present invention for evaluating underwater images are verified.

[0226] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0227] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An underwater image quality assessment method based on image feature weighted fusion analysis, characterized in that: The following steps are involved: Step 1, obtaining an optimized underwater image and preprocessing the image into a fixed size; Step 2: Perform color space conversion on the preprocessed underwater image, convert the RGB image to Lab color space, and obtain a Lab color space image. First, convert the RGB space to XYZ space. The conversion formula is as follows: Among them, matrix M is the constant matrix of color space conversion; Then convert from XYZ space to Lab space; Calculate the obtained images separately The variance and mean of the a channel and the b channel are calculated; and the logarithmic operation is introduced into the color measurement of the underwater image to obtain the evaluation value of the color characteristics of the underwater image without reference; the specific process is: Calculate separately The variance and mean of the a channel and b channel, Variance of the channel and mean The calculation formula is as follows: Variance of the channel and mean The calculation formula is as follows: Where N is the number of samples, and is the value of 𝑎 and b of the pth color; The subjective feeling of the human eye is proportional to the logarithm of the objective stimulus. By introducing logarithmic operation into the color measurement of underwater images, the evaluation value of the color feature of the reference-free image based on the Lab color model is obtained: in, Respectively refer to The variance and mean of the channel, Respectively refer to Variance and mean of the channel; Step 3, the preprocessed underwater image is processed by log-Gabor filters of N scales and M directions without any downsampling process; There are MN complex coefficients at each spatial position in all directions and all scales, and then the LPC intensity is calculated at a specific direction and a specific position in a specific space; The obtained LPC intensity is fused with the weight allocated in the specific space to obtain the evaluation value of the clarity feature of the reference-free underwater image; Step 4: Conduct subjective evaluation on the preprocessed underwater image, take the average score as the MOS value of the image, and perform nonlinear fitting with the objective value of the image feature to obtain the optimal weight coefficient; Step 5, performing linear weighted summation on the evaluation value of the underwater image color feature, the evaluation value of the underwater image clarity feature, and the optimal weight coefficient to obtain an optimized evaluation score of the underwater image quality.

2. The underwater image quality assessment method based on image feature weighted fusion analysis according to claim 1, characterized in that: The specific process of performing log-Gabor filter processing on the pre-processed underwater image in step 3 with N scales and M directions is as follows: First, for the image of size M×N obtained by preprocessing , perform Fourier transform to frequency domain, the formula is as follows: in, is the image pixel value in the spatial domain; is the Fourier transform result in the frequency domain; is a complex exponential function with a frequency of The sine and cosine components of ; After that, construct Log-Gabor filters of N scales and M directions. The filters are as follows: in, is the center direction of the filter, is the control direction bandwidth, the center frequency Determines the frequency center and bandwidth parameters of the filter Controls the width of the frequency response; Finally, multi-scale and multi-directional Log-Gabor filtering is performed on the frequency domain representation of the image to obtain 𝑀𝑁 complex coefficients at each spatial position in all directions and all scales, which are used Represents the complex coefficient at the 𝑖th scale, the 𝑗th direction, and the 𝑘th spatial position, and then inverse transforms each filtered result back to the spatial domain and saves the result.

3. The underwater image quality assessment method based on image feature weighted fusion analysis according to claim 2, characterized in that: The specific process of calculating the LPC intensity in step 3 is: In wavelet transform, position The characteristic signal is , then the signal is represented as: ,use Represents the complex wavelet transform of a signal; According to the complex wavelet transform theory and convolution theorem, we can get the complex wavelet transform Phase : in is the scale factor, is the displacement factor; In continuous wavelet transform Discrete sampling of discrete wavelet transform is performed on the , between every two adjacent scales, the scale factor Doubled, the spatial sampling rate is halved; at three consecutive scales of the two-dimensional image signal, the neighboring coefficients are , according to the principle of local coherence, from the coarser scale coefficient Get a finer scale factor The predicted phase of , where the first set of coefficients is a , the second set of coefficients is , i and j is an integer from 1 to 2, and the third set of coefficients is , i and j is an integer from 1 to 4, predicting the phase It is expressed as: Therefore, for Nearby Any extraction The phase of the sample coefficients and the corresponding predictions: Quantifying the strength of LPC for: Select a specific set of samples , the weight set corresponding to this sample is ,make: Get the weight set: After simplification, Directions and The LPC at each position is expressed as: The complex coefficients It is The scale, Directions and The complex coefficients of the spatial positions, represents the real part of a complex number; each spatial position is considered in a weighted form, with the weight being the first scale factor The magnitude of further indicates the strength of the k-th spatial LPC as: in is a constant, is the first scale factor Directions and The complex coefficients of the spatial positions.

4. The underwater image quality assessment method based on image feature weighted fusion analysis as claimed in claim 3, characterized in that: The LPC intensity obtained in step 3 is fused with the weight allocated in the specific space to obtain the evaluation value of the clarity feature of the non-reference underwater image, specifically: is assigned to The weight of the spatial LPC value is calculated as follows: Finally, the evaluation value of the clarity feature of the no-reference underwater image is obtained as follows: K is the total number of spaces, is the adjustment coefficient.

5. The underwater image quality assessment method based on image feature weighted fusion analysis according to claim 1, characterized in that: The specific process of step 4 is as follows: For the acquired pre-processed underwater images, the image quality levels are divided into five levels, namely "very poor", "poor", "average", "good", and "very good"; the subjective quality evaluation score is 0-10 points, where 0-2 points represent "very poor", 2-4 points represent "poor", 4-6 points represent "average", 6-8 points represent "good", and 8-10 points represent "very good"; the testers give the subjective evaluation score of the image between 0-10 points, and the score is accurate to two decimal places; finally, the average of the testers' scores on the image is taken as the mean opinion score MOS value of the image, which is defined as the following formula: in, Refers to Ratings from testers, is the total number of observers; Keep two decimal places; after obtaining the MOS value, perform nonlinear fitting with the objective value of the obtained image feature.

6. The underwater image quality assessment method based on image feature weighted fusion analysis according to claim 1, characterized in that: The specific process of step 5 is as follows: UCIQ is used to represent the underwater image quality evaluation score, and its expression is: in, is the evaluation value of the color feature of the underwater image without reference, is the evaluation value of the clarity feature of the underwater image without reference, are the weight coefficients for color evaluation and clarity evaluation respectively.

Images