Image quality evaluation method based on human visual characteristics

Abstract

The invention relates to the technical field of computer vision and information processing research, in particular to an image quality evaluation (IQA) method based on human visual characteristics, which comprises the following steps of: 1, performing color space transformation on an original image and a distorted image; 2, carrying out sub-block division on the YCrCb color space three-component graph; 3, processing all the sub-blocks through an intensity perception function of the HVS to obtain an intensity subjective perception result of the image; 4, carrying out DCT (Discrete Cosine Transform) on the perception result, carrying out processing and inverse transformation in combination with a CSF model, and sorting and synthesizing all processed sub-blocks to obtain a human eye perception image; and 5, evaluating a distorted image perceived by human eyes by adopting an SSIM mode, and taking a result as an IQA score. The IQA method based on the human visual characteristics has a good effect on solving the problem that the precision is low when an SSIM model evaluates image quality such as color saturation distortion and contrast distortion, and the defects of the SSIM model are overcome.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and information processing research, and in particular to an image quality evaluation method based on human visual characteristics. Background Art

[0002] Images often need to be stored, processed, and transmitted. However, during the storage, processing, and transmission of images, the images are often distorted or degraded due to performance limitations such as storage devices, processing methods, and transmission environments. This has a certain impact on user perception, use, and services. In particular, in recent years, users have increasingly high requirements, and image quality is of particular concern. Therefore, there is an urgent need for an effective, practical, simple, and easy-to-use Image Quality Assessment (IQA) criterion, which can not only evaluate the quality of images during use, but also monitor, track, and optimize image processing, transmission, and storage process parameters in real time to achieve better service quality. [2] At present, IQA has achieved a lot of research results. However, it is still far from meeting the growing needs in real life. This is mainly because there is a certain gap between the proposed methods and models and the actual evaluation results. [1-3] Therefore, it is necessary to study an effective, reliable, convenient and practical IQA method that is as consistent as possible with the perception of the human visual system (HVS) to provide a better control basis for image processing, storage and transmission operations.

[0003] In recent years, combined with the practical application of IQA, a lot of research has been done at home and abroad, and many measurement criteria have been reported. [4]. However, considering the performance of IQA model such as accuracy, generalization and computational complexity, the SSIM (Structural Similarity Index) image quality assessment model based on structural similarity proposed by Wang Z et al. (Wang Z, Bovik AC, Sheikh HR, et al. Image quality assessment: from error visibility to structural similarity [J]. IEEE Trans. Image Processing, 2004, 13 (4): 600-612.) is currently generally recognized. The proposed SSIM model has been applied to broadcasting, cable TV and the Internet. In essence, it is a numerical description of the similarity between two images. It is believed that structural similarity is a perceptual characteristic of HVS. Different from brightness and contrast, it can better perceive the structural attributes of target information. It also uses brightness, contrast and structure to describe the information of HVS perceived images in a certain way, that is, covariance is used to describe the similarity of structure, standard deviation reflects the effect of contrast, and mean represents the estimation of brightness. For two images x and y, i.e., source image and distorted image, SSIM is calculated as shown in formula (1).

[0004]

[0005] In the formula, μ x and μ y are the average values ​​of images x and y, respectively. and is the variance of x and y, and δ xy is the covariance of x and y; c1=(k1L) 2 , c2=(k2L) 2 , L is the dynamic range of pixel brightness, k1 = 0.01, k2 = 0.03, where c1 and c2 are constants to maintain stability [5] SSIM∈[0,1], that is, when two images are exactly the same, SSIM is 1, and if there is no relationship, it is 0.

[0006] Its comprehensive performance is better than common models such as SNR, PSNR and VSNR, but the accuracy of SSIM is still not high and needs to be improved. Therefore, a lot of research has been done on SSIM in recent years, and many improvement methods have been proposed, but the comprehensive benefits are still not high. Analyzing the SSIM model, it has some defects or shortcomings, which are mainly manifested as follows:

[0007] (1) SSIM has poor effect on image quality evaluation of certain distortion types. By analyzing the IQA results in CSIQ, LIVE and TID2013 databases (References: Sheikh HR, Wang Z, Bovik A C. LIVE Image and Video Quality Assessment Database [OL]. Available: http: / / live.ece.utexas.edu / research / quality., Ponomarenko N, Jin L, Ieremeiev O, et al. Tampere Image Database 2013 TID2013, version 1.0 [OL]. Available: http: / / www.ponomarenko.info / tid2013.htm., Larson EC, Chandler D M. The CSIQ image database [OL]. Available: http: / / vision.okstate.edu / ?loc=csiq.), it is found that SSIM has poor evaluation effect on some types of distorted images. For example, in the TID2013 image database, the evaluation effect of Contrast change (CC) and Change of color is poor. For the evaluation of saturation (CCS) distorted images, the Pearson linear correlation of subjective and objective evaluations is only 0.6398 and 0.2102, and the Spearman rank correlation can only reach 0.4561 and 0.1907.

[0008] (2) Visual redundancy is not considered well. SSIM mainly focuses on the structural information characteristics of HVS, without considering the actual situation of visual redundancy (Reference: Liu Guojun, Gao Lixia, Chen Liqi. Generalized average full reference image quality evaluation pooling strategy [J]. Optics and Precision Engineering, 2017, 25(3): 742-748.). Regarding visual redundancy, in image perception: HVS often has a certain threshold for image distortion. When the distortion is lower than the distortion perception threshold, HVS believes that the image is not distorted, and the subjective evaluation score remains basically unchanged, that is, MOS (mean opinion score) or DMOS (differenceMOS) remains unchanged; but if SSIM evaluation is used, as long as the image is distorted, its IQA score will change, resulting in a certain difference in the subjective and objective quality evaluation results of the image.

[0009] (3) Insufficient integration of visual characteristics. The object of SSIM evaluation is real images, which are quite different from images perceived by the human eye. Real images generally have more redundancy. The human eye is a natural high-quality filter that can effectively remove redundancy from perceived images. Therefore, the consistency of SSIM subjective and objective evaluation is not high, and the effect is poor. Therefore, it is necessary to further introduce other visual characteristics when improving SSIM.

[0010] (4) Generalization performance issues. The experimental results of using SSIM on images in the LIVE, CSIQ and TID2013 databases show that the evaluation accuracy PLCC in the LIVE database reaches 0.9449, while the PLCC in TID2008 is only 0.7732, showing poor generalization performance. In addition, when designing IQA models, the problem of model complexity has been around for a long time, especially when combining the IQA construction with the human visual system (HVS), which is even more difficult. Because, combining HVS and its mathematical model can greatly improve the accuracy of the IQA model, but because it increases the amount of calculation, the IQA algorithm is also relatively complex, with typical models such as VSI, VSNR and FSIM; therefore, how to combine the HVS characteristics and its model is a key issue.

[0011] Based on the above analysis, an IQA model is proposed based on SSIM, taking into account the issues of IQA model accuracy, computational complexity and generalization performance, combined with the contrast sensitivity characteristics of HVS. Summary of the invention

[0012] The purpose of the present invention is to provide an image quality assessment method based on human visual characteristics to solve the problems raised by the above background technology that image quality assessment in the current market is insufficient in terms of model accuracy, computational complexity and generalization performance.

[0013] To achieve the above object, the present invention provides the following technical solution: an image quality evaluation method based on human visual characteristics, the steps of the image quality evaluation method are as follows:

[0014] 1. Import original and distorted images;

[0015] 2. Perform color space transformation and brightness and chromaticity separation on the RGB image to obtain YC r C b Color space diagram;

[0016] 3. Take 8 pixels × 8 pixels as the target size and adjust the YC r C b The color space diagram is divided into sub-blocks;

[0017] 4. Through the HVS perception function of intensity I, each pixel on each sub-block is processed to obtain the HVS subjective perception result P of the brightness and chromaticity intensity of the image L and P rg , P by ;

[0018] 5. Perform DCT transformation on each sub-block of the subjective perception value P of image brightness and chromaticity, and perform frequency shift so that the central spectrum is zero, and obtain the spatial frequency f of each point;

[0019] 6. Calculate the angular frequency of each point on the sub-block transform domain, calculate the contrast perception threshold of each point on the sub-block, and filter the image;

[0020] 7. Based on the calculated contrast perception threshold, the contrast sensitivity value S of each point on the three-component image is calculated. L and S rg , S by , and normalize it;

[0021] 8. Then, the contrast sensitivity value is used as the weight factor corresponding to each point on the sub-block after filtering, and the product is calculated;

[0022] 9. Perform IDCT transformation on the multiplied value to obtain YC r C b The sub-blocks of the image that the human eye perceives in color space;

[0023] 10. Sort and synthesize all perceptual sub-blocks to obtain YC r C b The human eye perceives the image in the color space, and performs color space transformation to obtain the image perceived by the human eye in the RGB color space, recorded as RGB P ;

[0024] 11. The perceived images of the original image and the distorted image are recorded as RGB respectively Po , RGB Pd ;

[0025] 12. Use SSIM to evaluate the distorted image perceived by the human eye, and the calculated value is used as the quality evaluation score of the distorted image.

[0026] Preferably, the image quality evaluation method is used in combination with a method for applying a visual model in IQA, and the method for applying a visual model in IQA includes a human eye contrast sensitivity visual model and a method for combining CSF in IQA.

[0027] Preferably, in the method of combining CSF in the IQA, after the image is subjected to DCT, a transform domain image is obtained, which presents an energy redistribution diagram of different frequencies and different intensities, wherein the frequency is generally referred to as the spatial frequency f, which is defined as the number of periodic light and dark fringes per unit length, and the angular frequency f θ , a variable in the CSF model, is defined as the number of periods of the grating stripes that stimulate the HVS within a unit visual angle, and is expressed as in Equation (5).

[0028]

[0029] Where θ is the viewing angle and N is the number of periods of fringes within θ.

[0030] Preferably, the YC in step 2 r C b The color space diagram is a diagram of brightness, red-green and blue-yellow components, and the step 3 divides the brightness, red-green and blue-yellow components into sub-blocks respectively.

[0031] Preferably, the perception function of the HVS to the intensity I in step 4 is the combination formula (7).

[0032] P L =K L log 10 I L ,P rg =K rg log 10 I rg ,P by =K by log 10 I by (7)

[0033] Preferably, in step 6, the angular frequency of each point on the sub-block transform domain is calculated according to the relationship between the two frequencies when the HVS observes the image, i.e., equation (6), and is substituted into the luminance and chrominance CSF expressions (equations (2), (3) and (4)) to calculate the contrast perception threshold C of each point on the sub-block. T-L and C T-rg , C T-by , and use this threshold as the boundary to achieve filtering processing of the image.

[0034] CSF L (f θ ) = a·f θ ·exp(-b·f θ )[1+0.06·exp(b·f θ )] 0.5 (2)

[0035] Where a = 540·(1+0.7 / L)-0.2 ·{1+12[w·(1+f θ / 3) 2 ] -1}, b = 0.3·(1+100 / L) 0.15 .

[0036] Where: f θ is the angular frequency, w is the spatial viewing angle, and L is the average brightness of the perceived target. CSF L It quantitatively describes the contrast perception threshold of HVS when perceiving brightness targets, and its reciprocal represents the brightness contrast perception sensitivity of the human eye.

[0037] Red and green CSF rg :

[0038] Blue and yellow CSF by :

[0039] Where: CSF by and CSF rg They represent the chromaticity contrast perception thresholds of the human eye for blue-yellow and red-green opposing color targets respectively.

[0040]

[0041] Preferably, the filtering process is performed on the perceived results of image brightness and chromaticity using a contrast perception threshold to eliminate visual redundancy.

[0042] Preferably, the steps 7 and 8 are weighted using the normalized contrast sensitivity value as a weight factor of the filtering processing result, and the weighted sum is used as the perceived image of the real image perceived by the human eye.

[0043] Preferably, the calculation formula in step 10 is formula (8) and (9).

[0044]

[0045] Preferably, in step 12, the images x and y in SSIM (formula (1)) are represented by RGB Po and RGB Pd Substitute, and record this evaluation method as SSIMcsf, and its calculation method is as shown in formula (10).

[0046]

[0047]

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. This image quality evaluation method based on human visual characteristics aims at the problem that the structural similarity model SSIM has low accuracy in evaluating image quality such as color saturation distortion and contrast distortion. Combining the image contrast distortion characteristics and the contrast sensitivity characteristics of the human eye, and drawing on the SSIM modeling method, an image quality evaluation model is proposed to solve the shortcomings of SSIM.

[0050] 2. This image quality evaluation method based on human visual characteristics introduces CSF to preprocess the image and then uses SSIM for evaluation, which greatly improves the model performance and has good generalization performance and accuracy. This shows that CSF preprocessing of the image helps to improve the evaluation performance of SSIM. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flow chart of an image quality evaluation method based on human visual characteristics of the present invention;

[0052] Figure 2 This is a precision diagram of the proposed model and SSIM based on the IQA results in three databases for an image quality assessment method based on human visual characteristics of the present invention;

[0053] Figure 3 This is a diagram showing the accuracy of the proposed model and five classic models based on the PLCC values ​​of 24 distortion type images evaluated in the TID2013 library in an image quality evaluation method based on human visual characteristics of the present invention;

[0054] Figure 4 This is a diagram showing the accuracy of the proposed model and five classic models based on the PLCC values ​​of the image quality evaluation method based on human visual characteristics in the CSIQ library;

[0055] Figure 5 This is a diagram showing the accuracy of the proposed model compared with five classic models based on the PLCC values ​​of five types of distortion images evaluated in the LIVE library by an image quality evaluation method based on human visual characteristics of the present invention;

[0056] Figure 6 This is a performance diagram of the image quality evaluation method based on human visual characteristics of the present invention, comparing the evaluation results of two types of distorted images, CC and CCS, in the TID2013 database, and comparing SSIM with the proposed model;

[0057] Figure 7 This is a performance diagram of an image quality evaluation method based on human visual characteristics in the present invention, comparing the evaluation results of CC distorted images in the CSIQ database with those of SSIM and the proposed model. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] See also Figure 1-7 The present invention provides a technical solution: an image quality evaluation method based on human visual characteristics, the steps of the image quality evaluation method are:

[0060] 1. Import original and distorted images;

[0061] 2. Perform color space transformation and brightness and chromaticity separation on the RGB image to obtain YC r C b Color space diagram;

[0062] 3. Take 8 pixels × 8 pixels as the target size and adjust the YC r C b The color space diagram is divided into sub-blocks;

[0063] 4. Through the HVS perception function of intensity I, each pixel on each sub-block is processed to obtain the HVS subjective perception result P of the brightness and chromaticity intensity of the image L and P rg , P by ;

[0064] 5. Perform DCT transformation on each sub-block of the subjective perception value P of image brightness and chromaticity, and perform frequency shift so that the central spectrum is zero, and obtain the spatial frequency f of each point;

[0065] 6. Calculate the angular frequency of each point on the sub-block transform domain, calculate the contrast perception threshold of each point on the sub-block, and filter the image;

[0066] 7. Based on the calculated contrast perception threshold, the contrast sensitivity value S of each point on the three-component image is calculated. L and S rg , S by , and normalize it;

[0067] 8. Then, the contrast sensitivity value is used as the weight factor corresponding to each point on the sub-block after filtering, and the product is calculated;

[0068] 9. Perform IDCT transformation on the multiplied value to obtain YC r C b The sub-blocks of the image that the human eye perceives in color space;

[0069] 10. Sort and synthesize all perceptual sub-blocks to obtain YC r C b The human eye perceives the image in the color space, and performs color space transformation to obtain the image perceived by the human eye in the RGB color space, recorded as RGB P ;

[0070] 11. The perceived images of the original image and the distorted image are recorded as RGB respectively Po , RGB Pd ;

[0071] 12. Use SSIM to evaluate the distorted image perceived by the human eye, and the calculated value is used as the quality evaluation score of the distorted image.

[0072] The image quality evaluation method is used in combination with the application method of the visual model in IQA. The application method of the visual model in IQA includes the human eye contrast sensitivity visual model and the method of combining CSF in IQA. Since the final recipient of the image is the human eye, the ultimate effect goal of IQA is to meet visual perception. The contrast perception characteristic is one of the four major HVS characteristics, so the contrast sensitivity function CSF (Contrast Sensitivity Function) is generally used to quantitatively describe it.

[11] , which often reflects the corresponding relationship between the HVS's perception threshold and angular frequency of the target. Through a large number of related experiments in psychophysics, many CSF models have been reported (reference: Yao Juncai. Measurements of human vision contrast sensitivity to opposite colors using a CRT display [J]. Chinese Science Bulletin, 2011, 56 (23): 2425-2432.). Based on the actual effect of CSF and its application computational complexity, the Barten model is adopted in the proposed IQA model (reference: Sasaki Hisashi. A spatio-velocity extension of Barten's CSF model [C]. Proceedings of the International Display Workshops, 2011, vol. 1, p337-340.) CSF L (luminance CSF) and chrominance model CSF rg and CSF by(Proposed by Nadenau et al., reference: Nadenau M. Integration of human color vision models into high quality image compression [D]. PhD thesis, Polytechnique Fédéral de Lausanne, Switzerland, 2000.), and its expressions are as shown in equations (2), (3) and (4).

[0073] ①Human eye brightness contrast sensitivity function

[0074] CSF L (f θ ) = a·f θ ·exp(-b·f θ )[1+0.06·exp(b·f θ )] 0.5 (2)

[0075] Where a = 540·(1+0.7 / L) -0.2 ·{1+12[w·(1+f θ / 3) 2 ] -1}, b = 0.3·(1+100 / L) 0.15 .

[0076] Where: f θ is the angular frequency, w is the spatial viewing angle, and L is the average brightness of the perceived target. CSF L Quantitatively describes the contrast perception threshold of HVS when perceiving brightness targets. Its reciprocal represents the brightness contrast perception sensitivity of the human eye.

[12] .

[0077] ②Human eye contrast sensitivity function of opposing colors

[0078] Red and green CSF rg :

[0079] Blue and yellow CSF by :

[0080] Where: CSF by and CSF rg They represent the chromaticity contrast perception thresholds of the human eye for blue-yellow (by: blue-yellow) and red-green (rg: red-green) opposing color targets respectively.

[0081] In the IQA method combined with CSF, the image is transformed into a transform domain image after DCT, showing a redistribution diagram of energy with different frequencies and intensities. The frequency is generally called spatial frequency (f), which is defined as the number of periodic light and dark fringes per unit length, and the angular frequency f is θ , a variable in the CSF model, is defined as the number of periods of the grating stripes that stimulate the HVS within a unit visual angle, and is expressed as in Equation (5).

[0082]

[0083] In the formula, θ is the observation angle, and N is the number of fringes within the range of θ. Based on the description of the above two frequencies, the basis for combining CSF when constructing the IQA model is to find the relationship between the two frequencies, which is as follows.

[0084] After DCT transformation, the coordinates of the points on the image transform domain spectrum represent the spatial frequency, which is denoted as (f x , f y The DCT transformation effect of the image shows that: for any point on the spectrum graph, its inverse transform (IDCT) presents a grating, the stripes on it are vertical stripes or horizontal stripes or stripes interlaced with each other, the size of the grating is consistent with the size of the original image, and the number of stripe cycles on it is (f x 2 +f y 2 ) 1 / 2 . This shows that when HVS views an image, it perceives each point, which is actually the stripe grating, and perceives the entire image, which is the effect of the staggered combination of stripe gratings presented by all points. Then, the viewing angle θ is calculated based on the range corresponding to the observed target, the pixels on the observed target are W, and the observation distance is D. Using formula (5), the relationship between the two frequencies can be described by formula (6). Where PPI is the resolution of the display.

[0085]

[0086] According to this description, using equations (2) to (4) and (6), the contrast perception threshold and sensitivity threshold of the HVS for brightness and chromaticity at any point on the image can be calculated when the HVS perceives the image. Based on this, the CSF characteristics and their models can be combined in IQA.

[0087] IQA must ultimately conform to the subjective perception of HVS, and HVS's perception of images mainly comes from the stimulation of brightness, chromaticity, and contrast, and brightness is more sensitive than chromaticity. Therefore, HVS's judgment of image quality mainly depends on the measurement of the perceived brightness, chromaticity, and contrast information of the image. Based on this, the basic idea of ​​the proposed IQA method is: Combined with HVS's perception characteristics of image brightness and chromaticity and contrast sensitivity characteristics, the image is preprocessed, and the result is used as the brightness, chromaticity, and contrast information of the real image perceived by HVS. This perception result is used as the image perceived by HVS, and SSIM is used to evaluate the quality of the perceived image, and the evaluation score is used as the IQA result. The conceptual plan process is as follows: Figure 1 shown.

[0088] Step 2 YC r C b The color space diagram is a diagram of brightness (L), red-green (rg) and blue-yellow (by) components, and the step 3 divides the brightness (L), red-green (rg) and blue-yellow (by) components into sub-blocks respectively.

[0089] The perception function of the HVS in step 4 for the intensity I is the combination of formula (7).

[0090] P L =K L log 10 I L , P rg =K rg log 10 I rg , P by =K by log 10 I by (7)

[0091] Since the human eye perceives the target information based on the spatial contrast of the target, and the sensitivity is based on the contrast of the target, and when the information contrast of the target is less than the contrast threshold of the human eye's perception of the target, the HVS cannot perceive the difference of the target. Based on this, the contrast perception threshold is first used to filter the perception results of image brightness and chromaticity, and then the normalized contrast sensitivity value is used as the weight factor of the filtering result to weight it, and the weighted sum is used as the perceived image of the real image perceived by the human eye.

[0092] Step 6: According to the relationship between the two frequencies when the HVS observes the image, that is, equation (6), the angular frequency of each point in the sub-block transform domain is calculated, and it is substituted into the luminance and chrominance CSF expressions (equations (2), (3) and (4)) to calculate the contrast perception threshold C of each point on the sub-block. T-L and C T-rg , CT-by , and use this threshold as the boundary to implement filtering processing (Filter) on the image.

[0093] CSF L (f θ ) = a·f θ ·exp(-b·f θ )[1+0.06·exp(b·f θ )] 0.5 (2)

[0094] Where a = 540·(1+0.7 / L) -0.2 ·{1+12[w·(1+f θ / 3) 2 ] -1}, b = 0.3·(1+100 / L) 0.15 .

[0095] Where: f θ is the angular frequency, w is the spatial viewing angle, and L is the average brightness of the perceived target. CSF L It quantitatively describes the contrast perception threshold of HVS when perceiving brightness targets, and its reciprocal represents the brightness contrast perception sensitivity of the human eye.

[0096] Red and green CSF rg :

[0097] Blue and yellow CSF by :

[0098] Where: CSF by and CSF rg They represent the chromaticity contrast perception thresholds of the human eye for blue-yellow (by: blue-yellow) and red-green (rg: red-green) opposing color targets respectively.

[0099]

[0100] The filtering process uses contrast perception threshold to perform image brightness and chromaticity perception to eliminate visual redundancy.

[0101] In step 7 and step 8, the normalized contrast sensitivity value is used as a weight factor of the filtering processing result for weighting, and the weighted sum is used as the perceived image of the real image perceived by the human eye.

[0102] The calculation formulas in step 10 are equations (8) and (9).

[0103]

[0104] Step 12: Convert the images x and y in SSIM (Formula (1)) into RGB Po and RGB Pd Substitute, and record this evaluation method as SSIMcsf, and its calculation method is as shown in formula (10).

[0105]

[0106] The international open source databases TID2013 (reference: Ponomarenko N, Jin L, Ieremeiev O, et al. Tampere Image Database 2013 TID2013, version 1.0 [OL]. Available: http: / / www.ponomarenko.info / tid2013.htm.), CSIQ (reference: Larson EC, Chandler DM. The CSIQ image database [OL]. Available: http: / / vision.okstate.edu / ?loc=csiq.) and LIVE (reference: Sheikh HR, Wang Z, Bovik ACC. LIVE Image and Video Quality Assessment The simulation experiments were conducted with the original and distorted images from the database [OL]. Available: http: / / live.ece.utexas.edu / research / quality.); 3000, 866 and 323 distorted images (4189 in total) were selected from the database, as well as 25, 30 and 12 corresponding reference images. To better illustrate the performance of SSIMcsf, the experiments were conducted in the following three ways:

[0107] (1) Evaluate and simulate all distorted images in each database to illustrate the overall evaluation effect of the model;

[0108] (2) Simulate each distortion type of image in the database to explore the evaluation effect of the model on each distortion type;

[0109] (3) Simulations are performed on the contrast change and change of colorsaturation distorted images with very poor SSIM evaluation results to illustrate the improvement effect of the proposed model.

[0110] 1) Simulation of all images in 3 databases

[0111] The proposed IQA model is used to simulate all images in each of the three databases, TID2013, CSIQ and LIVE, to obtain objective quality evaluation scores. Combined with the IQA subjective scores MOS / DMOS in their open source libraries, the consistency parameters between the subjective and objective IQA scores are calculated, including PLCC (Pearson Linear Correlation Coefficient), SROCC (Spearman Rank Order Correlation Coefficient), RMSE (Root Mean Squared Error) and OR (Outlier Ratio)

[0112] (References: Zhai Guangtao, Min Xiongkuo. Perceptual image quality assessment: a survey [J]. Science China. Information Sciences, 2020, 63 (11): 211301. and Hangwei Chen, Xiongli Chai, Feng Shao, et al. Perceptual quality assessment of cartoon images [J]. IEEE Transactions on Multimedia, 2023, 25: 140-153.). At the same time, the SSIM model was used to simulate all images in each of the three databases, and the four consistency parameters were also obtained. Finally, based on the scatter plot between the subjective and objective quality evaluation scores and the four consistency parameters, the results of the two were compared. The results are as follows: Figure 2 shown.

[0113] In order to better illustrate the accuracy of the proposed model, four other common IQA models, namely PSNRHVS, VSNR, FSIMc and GMSD, are used to evaluate all images in each of the three databases. Based on the four correlation parameters PLCC, SROCC, RMSE and OR, the IQA results of the proposed IQA model are compared with those of the four typical models, as shown in Table 1.

[0114] Table 1 Performance comparison of the proposed model with five models based on four parameters of distorted image evaluation results in TID2013, CSIQ and LIVE libraries

[0115]

[0116] Combination Figure 2Table 1 shows the experimental results of the proposed model and five existing IQA models. The results show that: ① From the analysis of the discrete degree and correlation parameter value of the scatter plot, the evaluation effect of SSIMcsf is significantly better than that of SSIM for the overall evaluation of the images in the three databases, indicating that the model performance can be greatly improved by introducing CSF to preprocess the images and then using SSIM for evaluation. The weighted average of the improvement in the three databases is 10.4341%; ② From the evaluation results of the three databases, the PLCC values ​​of SSIMcsf accuracy are all above 0.85, indicating that the proposed model has good generalization performance; ③ The comparison with the experimental results of the four existing models shows that the accuracy of the proposed IQA model is better than that of the four models. The comparative analysis shows that using CSF to preprocess the images is helpful to improve the evaluation performance of SSIM.

[0117] 2) Simulation of all types of distortion images

[0118] The proposed IQA model is used to simulate each type of distorted image in the three databases, and the objective quality evaluation score of each type of distorted image is predicted. Combined with the subjective quality MOS score provided in the open source database, the consistency parameter between the subjective and objective evaluation scores of each type of distorted image is calculated. Based on the correlation parameter PLCC value, the results are compared with the experimental results of 5 common IQA models. The results are as follows: Figure 3 , Figure 4 and Figure 5 shown.

[0119] Through comparative analysis Figures 3 to 5 The results show that: ① For images of all different distortion types in the three databases, the volatility of the PLCC value of the SSIMcsf evaluation results is significantly smaller than that of the SSIM, indicating that the SSIMcsf model has better stability; ② For each distortion type in the three databases, the PLCC value of the SSIMcsf model evaluation results can reach the highest or second highest among the five methods, indicating that the proposed model has higher accuracy and better generalization performance; ③ For each distortion type, the PLCC value of the SSIMcsf model IQA result is basically higher than that of the SSIM, indicating that preprocessing the image using the HVS brightness perception model and the CSF model and its features can effectively improve the evaluation performance of the SSIM model.

[0120] (3) Simulation of Contrast Change and Change of Color Saturation Distorted Images

[0121] In order to better illustrate the effect of the proposed method on SSIM improvement, we selected two types of distorted images with the worst SSIM evaluation in TID2013 and CSIQ databases, namely Contrast change (241 images in both databases) and Change of color saturation (125 images in TID2013 database), and evaluated them using the proposed model. In combination with the subjective IQA score MOS in the international open source database, we calculated the values ​​of four correlation parameters. At the same time, we used SSIM to evaluate the 366 distorted images, and compared the results with those of the proposed model. The correlation analysis was shown in the scatter plot. Figure 6 , Figure 7 and Table 2.

[0122] Table 2 Comparison of the performance of the proposed model and SSIM based on the four parameter values ​​of the evaluation results of CC and CCS distorted images

[0123]

[0124] pass Figures 6-7 Compared with the results in Table 2, the following results are obtained: ① For the CCS distorted images in TID2013, the evaluation accuracy of SSIMcsf is more than double that of SSIM, which shows that the improvement effect of using CSF on SSIM is very obvious; ② For the CC distorted images in the two databases, the evaluation accuracy is also improved greatly, and the PLCC value is increased by 13.4462% on average, indicating that the proposed model SSIMcsf has good generalization performance. ③ After weighting the three groups of data in the two databases, the PLCC value of SSIMcsf is increased by 21.8845% compared with SSIM. Comprehensively speaking, the proposed model is very effective in improving SSIM.

[0125] At the same time, in order to better illustrate the advantages of the proposed model, combined with model accuracy and generalization performance, the CC images in the two open source databases were simulated and tested using four other common IQA models to obtain their objective quality evaluation scores, and four correlation parameters were calculated based on the MOS / DMOS values ​​in the database. The performance of the proposed IQA model was compared with the four models based on the four parameters, and the results are shown in Table 3.

[0126] Table 3 Comparison of the performance of the proposed model with four common models based on four correlation parameters of the evaluation results of CC distorted images in TID2013 and CSIQ libraries

[0127]

[0128] Comparative analysis of the results in Table 3 shows that: ① For CC images in the TID2013 database, the proposed model is better than the four existing models in terms of accuracy; ② For CC images in the CSIQ database, the IQA accuracy of the proposed model is close to that of FSIMc and GMSD, but the IQA accuracy of the two for CC images in TID2013 is relatively low, indicating that the generalization performance of the proposed model is better than that of the FSIMc and GMSD models. The overall benefits of comprehensive accuracy and generalization performance show that the performance of the proposed model is better than the four existing IQA models.

[0129] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An image quality evaluation method based on human visual characteristics, characterized in that: The steps of the image quality evaluation method are as follows: 1) Import the original image and the distorted image; 2) Perform color space transformation and brightness and chromaticity separation on the RGB image to obtain YC r C b Color space diagram; 3) Take 8 pixels × 8 pixels as the target size and adjust the YC r C b The color space diagram is divided into sub-blocks; 4) Through the perception function of HVS for intensity I, each pixel on each sub-block is processed to obtain the subjective perception result P of HVS for the brightness and chromaticity intensity of the image L and P rg , P by ; 5) Perform DCT transformation on each sub-block of the subjective perception value P of image brightness and chromaticity, and perform frequency shift so that the central spectrum is zero, and obtain the spatial frequency f of each point; 6) Calculate the angular frequency of each point on the sub-block transform domain, calculate the contrast perception threshold of each point on the sub-block, and filter the image; 7) Based on the calculated contrast perception threshold, the contrast sensitivity value S of each point on the three-component image is calculated. L and S rg , S by , and normalize it; 8) Then, the contrast sensitivity value is used as the weight factor corresponding to each point on the sub-block after filtering, and the product is calculated; 9) Perform IDCT transformation on the multiplied value to obtain YC r C b The sub-blocks of the image that the human eye perceives in color space; 10) Sort and synthesize all perceptual sub-blocks to obtain YC r C b The human eye perceives the image in the color space, and performs color space transformation to obtain the image perceived by the human eye in the RGB color space, recorded as RGB P ; 11) The perceived images of the original image and the distorted image are recorded as RGB respectively Po , RGB Pd ; 12) SSIM is used to evaluate the distorted image perceived by the human eye, and the calculated value is used as the quality evaluation score of the distorted image.

2. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: The image quality evaluation method is used in combination with the application method of the visual model in IQA. The application method of the visual model in IQA includes a human eye contrast sensitivity visual model and a method combining CSF in IQA.

3. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: In the method of combining CSF in IQA, after the image is processed by DCT, a transform domain image is obtained, which presents an energy redistribution diagram of different frequencies and different intensities. The frequency is generally called spatial frequency f, which is defined as the number of periodic light and dark stripes per unit length, and the angular frequency f θ , a variable in the CSF model, is defined as the number of periods of the grating stripes that stimulate the HVS within a unit visual angle, and is expressed as in formula (5). Where θ is the viewing angle and N is the number of periods of fringes within θ.

4. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: Step 2 of YC r C b The color space diagram is a diagram of brightness, red-green and blue-yellow components, and the step 3 divides the brightness, red-green and blue-yellow components into sub-blocks respectively.

5. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: The perception function of the HVS for the intensity I in step 4 is the combination formula (7). P L =K L log 10 I L ,P rg =K rg log 10 I rg ,P by =K by log 10 I by (7) 6. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: The step 6 calculates the angular frequency of each point in the sub-block transform domain according to the relationship between the two frequencies when the HVS observes the image, i.e., equation (6), and substitutes it into the luminance and chrominance CSF expressions (equations (2), (3) and (4)) to calculate the contrast perception threshold C of each point in the sub-block. T-L and C T-rg , C T-by , and use this threshold as the boundary to achieve filtering processing of the image. CSF L (f θ )=a·f θ ·exp(-b·f θ )[1+0.06·exp(b·f θ )] 0.5 (2) Where a = 540·(1+0.7 / L) -0.2 ·{1+12[w·(1+f θ / 3) 2 ] -1 }, b = 0.3·(1+100 / L) 0.15 . Where: f θ is the angular frequency, w is the spatial viewing angle, and L is the average brightness of the perceived target. CSF L It quantitatively describes the contrast perception threshold of HVS when perceiving brightness targets, and its reciprocal represents the brightness contrast perception sensitivity of the human eye. Red and green CSF rg : Blue and yellow CSF by : Where: CSF by and CSF rg They represent the chromaticity contrast perception thresholds of the human eye for blue-yellow and red-green opposing color targets respectively.

7. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: The filtering process uses a contrast perception threshold to perform the perception results of image brightness and chromaticity to eliminate visual redundancy.

8. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: In step 7 and step 8, the normalized contrast sensitivity value is used as a weight factor of the filtering processing result for weighting, and the weighted sum is used as the perceived image of the real image perceived by the human eye.

9. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: The calculation formulas in step 10 are formulas (8) and (9).

10. The image quality evaluation method based on human visual characteristics according to claim 1, characterized in that: In step 12, the images x and y in SSIM (formula (1)) are respectively converted into RGB Po and RGB Pd Substitute, and record this evaluation method as SSIMcsf, and its calculation method is as shown in formula (10).

Images