Image quality evaluation method and device, equipment, storage medium and computer program product

Abstract

The embodiment of the invention provides an image quality evaluation method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a to-be-evaluated target image and a to-be-evaluated reference image; taking the reference image as a benchmark, and respectively calculating a single-dimensional evaluation result of the target image under each evaluation dimension in the plurality of evaluation dimensions; the plurality of evaluation dimensions comprise at least two of the following items: a pixel difference dimension, a structure difference dimension and a visual information difference dimension; and fusing each single-dimensional evaluation result of the target image to obtain a quality evaluation result of the target image. According to the embodiment of the invention, the quality evaluation result of the target image is closer to the visual perception condition of human eyes, and meanwhile, the accuracy and objectivity of the quality evaluation result of the target image are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of image quality assessment, and in particular, to an image quality assessment method, apparatus, device, storage medium, and computer program product. Background Art

[0002] In the field of digital image processing, image quality evaluation has always been the focus of research. With the rapid development of multimedia technology, image quality evaluation is crucial for applications such as image compression, enhancement, and restoration.

[0003] Although traditional image quality evaluation methods are simple in calculation, they often cannot well reflect the human eye's visual perception of image quality. Summary of the invention

[0004] In view of this, embodiments of the present application provide an image quality assessment method, apparatus, device, storage medium, and computer program product to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of an embodiment of the present application, there is provided an image quality evaluation method, comprising:

[0006] Obtaining a target image to be evaluated and a reference image;

[0007] Taking the reference image as a reference, calculating the single-dimensional evaluation result of the target image in each evaluation dimension of multiple evaluation dimensions respectively; the multiple evaluation dimensions include at least two of the following: a pixel difference dimension, a structure difference dimension, and a visual information difference dimension;

[0008] The single-dimensional evaluation results of the target image are integrated to obtain a quality evaluation result of the target image.

[0009] In some embodiments, the single-dimensional evaluation result corresponding to the pixel difference dimension is the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension is the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension is the visual information fidelity;

[0010] The fusing of the single-dimensional evaluation results of the target image to obtain the quality evaluation result of the target image includes:

[0011] Normalizing the peak signal-to-noise ratio to obtain a normalized peak signal-to-noise ratio; normalizing the visual information fidelity to obtain a normalized visual information fidelity;

[0012] According to a preset weight ratio, the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused to obtain a quality evaluation result of the target image.

[0013] In some embodiments, the process of calculating the fidelity of the visual information of the target image includes:

[0014] Determine a first variance of the reference image based on the grayscale value of the reference image, determine a second variance of the target image based on the grayscale value of the target image, and determine a covariance of the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image;

[0015] Adjusting the target image based on the first variance and the covariance between the reference image and the target image to obtain an adjusted target image; determining a third variance of the adjusted target image based on the second variance and the covariance between the reference image and the target image;

[0016] The fidelity of visual information of the target image is determined based on the first variance and the third variance.

[0017] In some embodiments, determining a first variance of the reference image based on the grayscale value of the reference image, determining a second variance of the target image based on the grayscale value of the target image, and determining a covariance between the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image comprises:

[0018] Performing Gaussian blur processing on the reference image to obtain a Gaussian reference image; performing Gaussian blur processing on the target image to obtain a Gaussian target image;

[0019] The first variance is determined based on the grayscale value of the Gaussian reference image, the second variance is determined based on the grayscale value of the Gaussian target image, and the covariance of the reference image and the target image is determined based on the grayscale value of the Gaussian target image and the grayscale value of the Gaussian reference image.

[0020] In some embodiments, the process of calculating the fidelity of the visual information of the target image includes:

[0021] Performing multi-scale decomposition on the reference image and the target image to obtain the reference image and the target image at multiple scales;

[0022] Determining the sub-scale visual information fidelity of the target image based on the grayscale value of the reference image and the grayscale value of the target image at each scale;

[0023] The average of the sub-scale visual information fidelity of the target image at each scale is taken as the visual information fidelity of the target image.

[0024] In some embodiments, the step of calculating the single-dimensional evaluation result of the target image in each evaluation dimension of the multiple evaluation dimensions based on the reference image includes:

[0025] Inputting the target image and the reference image into a pre-trained evaluation model, and obtaining target image features of the target image and reference image features corresponding to the reference image through a feature extraction model in the evaluation model;

[0026] The peak signal-to-noise ratio calculation module in the evaluation model is used to calculate the peak signal-to-noise ratio of the target image based on the target image features and the reference image features; the structural similarity calculation module in the evaluation model is used to calculate the structural similarity of the target image based on the target image features and the reference image features; the visual information fidelity module in the evaluation model is used to calculate the visual information fidelity of the target image based on the target image features and the reference image features;

[0027] The peak signal-to-noise ratio is normalized through the fusion module in the evaluation model to obtain a normalized peak signal-to-noise ratio; the visual information fidelity is normalized to obtain a normalized visual information fidelity; and the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused according to a preset weight ratio to obtain a quality evaluation result of the target image.

[0028] According to a second aspect of an embodiment of the present application, there is provided an image quality assessment device, comprising:

[0029] An acquisition module, used for acquiring a target image and a reference image to be evaluated;

[0030] A calculation module, used to calculate the single-dimensional evaluation results of the target image under each evaluation dimension of multiple evaluation dimensions based on the reference image; the multiple evaluation dimensions include at least two of the following: pixel difference dimension, structure difference dimension, and visual information difference dimension;

[0031] The evaluation module is used to fuse the single-dimensional evaluation results of the target image to obtain a quality evaluation result of the target image.

[0032] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including:

[0033] A memory for storing executable instructions;

[0034] The processor is used to implement the image quality assessment method as described in the first aspect above when executing the executable instructions stored in the memory.

[0035] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, storing executable instructions for implementing the image quality assessment method as described in the first aspect above when executed by a processor.

[0036] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program or instructions, which, when executed by a processor, implements the image quality assessment method as described in the first aspect above.

[0037] In an embodiment of the present application, a target image to be evaluated and a reference image are obtained; based on the reference image, single-dimensional evaluation results of the target image in each of multiple evaluation dimensions are calculated respectively; the multiple evaluation dimensions include at least two of the following: pixel difference dimension, structure difference dimension, and visual information difference dimension; and the single-dimensional evaluation results of the target image are fused to obtain a quality evaluation result of the target image.

[0038] In the embodiment of the present application, the single-dimensional evaluation results under the pixel difference dimension, the structural difference dimension and the visual information difference dimension are integrated to obtain the quality evaluation result of the target image. On the one hand, the integration of the visual information difference dimension can make the quality evaluation result of the target image closer to the visual perception of the human eye; on the other hand, the integration of multiple evaluation dimensions can improve the accuracy and objectivity of the quality evaluation result of the target image.

[0039] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 1 ;

[0042] Figure 2 A schematic diagram of parameter relationships for calculating evaluation results of each single dimension applicable to an embodiment of the present application;

[0043] Figure 3 A schematic diagram of the implementation flow of a method for determining the fidelity of visual information applicable to an embodiment of the present application;

[0044] Figure 4Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 2 ;

[0045] Figure 5 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 3 ;

[0046] Figure 6 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 4 ;

[0047] Figure 7 A schematic diagram of the structure of an image quality evaluation device applicable to an embodiment of the present application;

[0048] Figure 8 A schematic diagram of a hardware entity of an electronic device applicable to an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0050] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in the embodiments of the present application refers to and includes any or all possible combinations of one or more associated listed items.

[0051] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0052] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0053] In addition, the term "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0054] To facilitate understanding of the technical solutions of the embodiments of the present application, the relevant technologies of the embodiments of the present application are described below. The following related technologies can be arbitrarily combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0055] The specific implementation of the embodiment of the present application is described below in conjunction with the accompanying drawings of the embodiment of the present application.

[0056] See also Figure 1 , Figure 1 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 1 The image quality evaluation method of the embodiment of the present application comprises the following steps:

[0057] Step S101: Acquire a target image and a reference image to be evaluated.

[0058] Specifically, the target image may be an image obtained by performing image distortion processing on a reference image. For example, the reference image may be subjected to geometric transformations such as scaling, rotation, and translation, or the reference image may be blurred by a Gaussian filter to obtain the target image.

[0059] Exemplarily, a target image to be evaluated and a reference image may be acquired to perform image quality evaluation on the target image based on the reference image.

[0060] Optionally, in some of the embodiments, the acquired target image and reference image may be converted to grayscale, so as to facilitate subsequent calculations based on the converted target image and reference image, thereby improving calculation efficiency.

[0061] Step S102: Taking the reference image as a benchmark, respectively calculate the single-dimensional evaluation result of the target image in each of the multiple evaluation dimensions; the multiple evaluation dimensions include at least two of the following: pixel difference dimension, structure difference dimension, and visual information difference dimension.

[0062] Specifically, the pixel difference dimension refers to the difference between the reference image and the target image at the pixel level, and the single-dimensional evaluation result corresponding to the pixel difference dimension can be the peak signal-to-noise ratio, etc. The structural difference dimension refers to the difference in structural features between the reference image and the target image, and the single-dimensional evaluation result corresponding to the structural difference dimension can be structural similarity or multi-scale structural similarity, etc. The visual information difference dimension refers to the difference in visual perception between the reference image and the target image, and the single-dimensional evaluation result corresponding to the visual information difference dimension can be the visual information fidelity.

[0063] Exemplarily, the reference image can be used as a benchmark to calculate the single-dimensional evaluation results of the target image in the evaluation dimensions of pixel difference dimension, structural difference dimension, and visual information difference dimension. Taking the single-dimensional evaluation result corresponding to the pixel difference dimension as the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension as the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension as the visual information fidelity as an example, the reference image can be used as a benchmark to calculate the peak signal-to-noise ratio, structural similarity, and visual information fidelity of the target image.

[0064] Step S103: Fusing the single-dimensional evaluation results of the target image to obtain a quality evaluation result of the target image.

[0065] Exemplarily, the single-dimensional evaluation results of the target image under evaluation dimensions such as pixel difference dimension, structural difference dimension and visual information difference dimension can be fused to obtain the quality evaluation result of the target image. Taking the single-dimensional evaluation result corresponding to the pixel difference dimension as the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension as the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension as the visual information fidelity as an example, the peak signal-to-noise ratio, structural similarity and visual information fidelity of the target image can be fused to obtain the quality evaluation result of the target image.

[0066] In the embodiment of the present application, the single-dimensional evaluation results under the pixel difference dimension, the structural difference dimension and the visual information difference dimension are integrated to obtain the quality evaluation result of the target image. On the one hand, the integration of the visual information difference dimension can make the quality evaluation result of the target image closer to the visual perception of the human eye; on the other hand, the integration of multiple evaluation dimensions can improve the accuracy and objectivity of the quality evaluation result of the target image.

[0067] Optionally, in some embodiments, the single-dimensional evaluation result corresponding to the pixel difference dimension is the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension is the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension is the visual information fidelity. The single-dimensional evaluation results of the target image are fused to obtain the quality evaluation result of the target image, including: normalizing the peak signal-to-noise ratio to obtain a normalized peak signal-to-noise ratio; normalizing the visual information fidelity to obtain a normalized visual information fidelity; and weighted fusion of the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity according to a preset weight ratio to obtain the quality evaluation result of the target image.

[0068] Specifically, see Figure 2 , the peak signal-to-noise ratio of the target image can be calculated according to the mean square error of the target image. After obtaining the peak signal-to-noise ratio of the target image, the peak signal-to-noise ratio of the target image can be normalized to obtain the normalized peak signal-to-noise ratio of the target image. The calculation process of the peak signal-to-noise ratio of the target image is as follows: the mean square error of the target image can be determined based on the size value and pixel value of the reference image and the pixel value of the target image. For the specific calculation process, please refer to formula (1). The peak signal-to-noise ratio of the target image is determined based on the pixel value and mean square error of the target image. For the specific calculation process, please refer to formula (2):

[0069]

[0070] Among them, MSE represents the mean square error of the target image, m and n represent the number of rows and columns of the target image respectively, I(i,j) represents the pixel value of the reference image at position (i,j), and K(i,j) represents the pixel value of the target image at position (i,j).

[0071]

[0072] Among them, PSNR represents the peak signal-to-noise ratio of the target image, MSE represents the mean square error of the target image, and MAX I Indicates the maximum pixel value of the target image.

[0073] See also Figure 2, the brightness parameter (i.e., the brightness similarity between the target image and the reference image) can be calculated based on the grayscale mean of the reference image and the grayscale mean of the target image, the contrast parameter (i.e., the contrast similarity between the target image and the reference image) can be calculated based on the grayscale variance or grayscale standard deviation of the reference image and the grayscale variance or grayscale standard deviation of the target image, the structure parameter (i.e., the reference structure similarity between the target image and the reference image) can be calculated based on the grayscale covariance between the reference image and the target image, and the structure similarity of the target image can be calculated based on the brightness parameter, contrast parameter, and structure parameter. Specifically, the calculation process of the structural similarity of the target image is as follows: the brightness similarity, contrast similarity and reference structural similarity between the target image and the reference image can be determined based on the grayscale value of the reference image and the grayscale value of the target image; the structural similarity between the target image and the reference image is determined based on the brightness similarity, contrast similarity and reference structural similarity between the target image and the reference image. Specifically, the calculation process of brightness similarity is shown in formula (3), the calculation process of contrast similarity is shown in formula (4), the calculation process of reference structural similarity is shown in formula (5), and the calculation process of structural similarity between the target image and the reference image is shown in formula (6):

[0074]

[0075] Among them, l(I,J) represents the brightness similarity between the target image I and the reference image J, μ I and μ J They represent the grayscale mean of the target image I and the reference image J respectively, and C1 is a constant.

[0076]

[0077] Where c(I,J) represents the contrast similarity between the target image I and the reference image J, σ I and σ J They represent the grayscale standard deviation of the target image I and the reference image J respectively, and C2 is a constant.

[0078]

[0079] Among them, s(I,J) represents the reference structure similarity between the target image I and the reference image J, σ I σ J It represents the grayscale covariance between the target image I and the reference image J, and C3 is a constant.

[0080] SSIM(I,J)=l(I,J) α c(I,J) β ·s(I,J) γ (6);

[0081] Among them, SSIM(I,J) represents the structural similarity between the target image I and the reference image J, l(I,J) represents the brightness similarity between the target image I and the reference image J, c(I,J) represents the contrast similarity between the target image I and the reference image J, s(I,J) represents the reference structure similarity between the target image I and the reference image J, α, β, γ respectively represent the proportion of brightness similarity, contrast similarity, and reference structure similarity in the structural similarity measurement. In some embodiments, α, β, γ can all be taken as 1. The values ​​of α, β, γ can also be set according to actual needs, and the embodiments of the present application do not specifically limit this.

[0082] Optionally, in some of the embodiments, the single-dimensional evaluation result corresponding to the structural difference dimension may also include a multi-scale structural similarity (MS-SSIM). Figure 2 , the target image and the reference image can be decomposed at multiple scales to obtain the target image and the reference image at multiple scales. The brightness parameter (i.e., the brightness similarity between the target image and the reference image) is determined based on the grayscale mean of the reference image and the grayscale mean of the target image at the finest scale. At each scale, the contrast parameters of the target image and the reference image at each scale are determined based on the grayscale variance of the reference image and the grayscale variance of the target image (i.e., the contrast similarity between the target image and the reference image). At each scale, the structural parameters of the target image and the reference image at each scale are determined based on the grayscale covariance of the reference image and the target image (i.e., the reference structural similarity between the target image and the reference image). The multi-scale structural similarity of the target image is calculated based on the brightness parameter, the contrast parameter at each scale, and the structural parameter.

[0083] Specifically, the process of calculating the multi-scale structural similarity is as follows: at each scale, the contrast similarity and reference structural similarity between the target image and the reference image are determined based on the grayscale value of the reference image and the grayscale value of the target image, and the brightness similarity between the target image and the reference image is determined based on the grayscale value of the reference image and the grayscale value of the target image at the finest scale. The structural similarity between the target image and the reference image at each scale is calculated based on the contrast similarity and reference structural similarity at each scale, and the brightness similarity at the finest scale, and the structural similarity between the target image and the reference image at each scale is weighted averaged to obtain the multi-scale structural similarity between the target image and the reference image. Multi-scale structural similarity can evaluate the structure of an image at different scales, and can more accurately reflect the details and structural changes of the image.

[0084] Exemplarily, after calculating the peak signal-to-noise ratio, visual information fidelity and structural similarity of the target image, the peak signal-to-noise ratio and the visual information fidelity can be normalized respectively to facilitate subsequent weighted fusion. After obtaining the normalized peak signal-to-noise ratio and the normalized visual information fidelity, the normalized peak signal-to-noise ratio, the normalized visual information fidelity, the structural similarity and the multi-scale structural similarity can be weighted fused according to a preset weight ratio to obtain the quality evaluation result of the target image.

[0085] Among them, different weight ratios can be set for the single-dimensional evaluation results corresponding to different evaluation dimensions, so that the sum of the weight ratios corresponding to each single-dimensional evaluation result is 100%. For example, the weight ratio of the normalized peak signal-to-noise ratio can be set between 20% and 25%, the weight ratio of the normalized visual information fidelity can be set between 20% and 30%, the weight ratio of the structural similarity can be set between 25% and 35%, and the weight ratio of the multi-scale structural similarity can be set between 35% and 40%. The weight ratio corresponding to each single-dimensional evaluation result can also be set according to actual conditions, and the embodiment of the present application does not specifically limit this.

[0086] In the embodiment of the present application, on the one hand, the visual information fidelity is integrated to obtain the quality evaluation result of the target image. The integration of visual information fidelity can make the quality evaluation result of the target image closer to the visual perception of the human eye; on the other hand, according to the preset weight ratio, the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weighted and integrated to obtain the quality evaluation result of the target image. Among them, the peak signal-to-noise ratio can more accurately measure the pixel-level error of the image, the visual information fidelity can more accurately measure the loss of high-frequency information in the image, and the structural similarity can more accurately reflect the changes in the structure, contrast and brightness of the image. The evaluation results of multiple evaluation dimensions are integrated to improve the accuracy and objectivity of the quality evaluation results of the target image.

[0087] Optionally, in some embodiments, the calculation process of the visual information fidelity of the target image includes: determining a first variance of the reference image based on the grayscale value of the reference image, determining a second variance of the target image based on the grayscale value of the target image, and determining a covariance between the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image; adjusting the target image based on the first variance and the covariance between the reference image and the target image to obtain an adjusted target image; determining a third variance of the adjusted target image based on the second variance and the covariance between the reference image and the target image; and determining the visual information fidelity of the target image based on the first variance and the third variance.

[0088] Specifically, the first variance of the reference image is the grayscale variance of the reference image, the second variance of the target image is the grayscale variance of the target image, and the covariance between the reference image and the target image is the grayscale covariance between the reference image and the target image. Optionally, in some embodiments, when the first variance of the reference image is less than 0, the first variance of the reference image is assigned to 0; when the second variance of the target image is less than 0, the second variance of the target image is assigned to 0, ensuring that each variance is non-negative, avoiding computational problems caused by negative variances, and improving the stability and reliability of the algorithm.

[0089] Exemplarily, the target image can be adjusted based on the first variance of the reference image and the covariance between the reference image and the target image to obtain the adjusted target image. Specifically, the correlation coefficient can be determined based on the first variance of the reference image and the covariance between the reference image and the target image, and the target image can be adjusted based on the correlation coefficient to obtain the adjusted target image. The calculation process of the correlation coefficient g is shown in formula (7):

[0090]

[0091] Where g represents the correlation coefficient, Cov(K,I) represents the grayscale covariance between the reference image K and the target image I, represents the first variance of the reference image, G1 represents a constant, and in some embodiments, G1 can be set to 10 -8 , can also be set according to actual needs, so that The result of is greater than 0. The embodiment of the present application does not specifically limit the value of G1.

[0092] The third variance of the adjusted target image can be determined based on the second variance of the target image and the covariance between the reference image and the target image. For the specific calculation process, please refer to formula (8):

[0093]

[0094] in, represents the third difference of the adjusted target image, represents the second variance of the target image, Cov(K,I) represents the grayscale covariance between the reference image K and the target image I, and g represents the correlation coefficient. In some embodiments, when the third variance of the adjusted target image is less than 0, the third variance of the adjusted target image can be assigned a value of 0.

[0095] After determining the third variance of the adjusted target image, the visual information fidelity of the target image can be determined based on the first variance of the reference image and the third variance of the adjusted target image. Specifically, the numerator of the visual information fidelity can be calculated based on the first variance of the reference image, the third variance of the adjusted target image, and the correlation coefficient, and the denominator of the visual information fidelity can be calculated based on the first variance of the reference image, so that the visual information fidelity is determined according to the numerator of the visual information fidelity and the denominator of the visual information fidelity. The calculation process of the numerator of the visual information fidelity is shown in formula (9), the calculation process of the denominator of the visual information fidelity is shown in formula (10), and the calculation process of the visual information fidelity is shown in formula (11):

[0096]

[0097] Among them, num represents the numerator of visual information fidelity, g represents the correlation coefficient, represents the first variance of the reference image, represents the third difference of the adjusted target image, G2 represents a constant. In some embodiments, G2 can be taken as 0.3, or it can be taken according to actual needs. The embodiment of the present application does not specifically limit the value of G2.

[0098]

[0099] Where den represents the denominator of the fidelity of visual information, represents the first variance of the reference image, G2 represents a constant. In some embodiments, G2 can be taken as 0.3, or it can be taken according to actual needs. The embodiment of the present application does not specifically limit the value of G2.

[0100]

[0101] Among them, VIF represents the fidelity of visual information, num represents the numerator of the fidelity of visual information, and den represents the denominator of the fidelity of visual information.

[0102] In the embodiment of the present application, the visual information fidelity of the target image is determined based on the grayscale value of the reference image and the grayscale value of the target image, which simplifies the calculation process of the visual information fidelity and improves the calculation efficiency of the visual information fidelity.

[0103] Optionally, in some embodiments, a first variance of the reference image is determined based on the grayscale value of the reference image, a second variance of the target image is determined based on the grayscale value of the target image, and a covariance between the reference image and the target image is determined based on the grayscale value of the target image and the grayscale value of the reference image, including: performing Gaussian blur processing on the reference image to obtain a Gaussian reference image; performing Gaussian blur processing on the target image to obtain a Gaussian target image; determining a first variance based on the grayscale value of the Gaussian reference image, determining a second variance based on the grayscale value of the Gaussian target image, and determining the covariance between the reference image and the target image based on the grayscale value of the Gaussian target image and the grayscale value of the Gaussian reference image.

[0104] Specifically, the Gaussian blur processing process may include performing Gaussian blur processing on an image, performing scaling processing on the image obtained after the Gaussian blur processing, performing Gaussian blur processing again on the scaled image, and so on.

[0105] Exemplarily, in the process of calculating the fidelity of visual information of a target image, Gaussian blur processing can be performed on the reference image and the target image respectively, and a first variance of the reference image is determined based on the grayscale value of the Gaussian reference image after the Gaussian blur processing, a second variance of the target image is determined based on the grayscale value of the Gaussian target image after the Gaussian blur processing, and a covariance between the reference image and the target image is determined based on the grayscale value of the Gaussian reference image after the Gaussian blur processing and the grayscale value of the Gaussian target image.

[0106] In an embodiment of the present application, the first variance of the reference image and the second variance of the target image are calculated based on the Gaussian reference image and the Gaussian target image after Gaussian blur processing, which can reduce the influence of noise in the reference image and the target image and improve the calculation efficiency and accuracy of the first variance and the second variance.

[0107] Optionally, in some of the embodiments, the calculation process of the visual information fidelity of the target image includes: performing multi-scale decomposition on the reference image and the target image to obtain the reference image and the target image at multiple scales; determining the sub-scale visual information fidelity of the target image based on the grayscale value of the reference image and the grayscale value of the target image at each scale; and taking the average of the sub-scale visual information fidelity of the target image at each scale as the visual information fidelity of the target image.

[0108] Specifically, performing multi-scale decomposition on an image refers to decomposing the image into multiple versions at different scales or resolutions.

[0109] Exemplarily, taking the decomposition of the reference image and the target image into versions at four different scales as an example, at the first scale, the sub-scale visual information fidelity of the target image at the first scale can be determined based on the grayscale value of the reference image at the first scale and the grayscale value of the target image at the first scale; at the second scale, the sub-scale visual information fidelity of the target image at the second scale can be determined based on the grayscale value of the reference image at the second scale and the grayscale value of the target image at the second scale; at the third scale, the sub-scale visual information fidelity of the target image at the third scale can be determined based on the grayscale value of the reference image at the third scale and the grayscale value of the target image at the third scale; at the fourth scale, the sub-scale visual information fidelity of the target image at the fourth scale can be determined based on the grayscale value of the reference image at the fourth scale and the grayscale value of the target image at the fourth scale. The average of the sub-scale visual information fidelity of the target image at the first scale, the sub-scale visual information fidelity of the target image at the second scale, the sub-scale visual information fidelity of the target image at the third scale, and the sub-scale visual information fidelity of the target image at the fourth scale is taken as the visual information fidelity of the target image.

[0110] In the embodiment of the present application, multi-scale decomposition can be used to directly calculate the fidelity of visual information in the spatial domain without the need for frequency domain conversion, thereby improving the calculation efficiency of the fidelity of visual information.

[0111] See also Figure 3 , Figure 3 The following is a flowchart of a method for determining the fidelity of visual information applicable to an embodiment of the present application. The method for determining the fidelity of visual information in an embodiment of the present application comprises the following steps:

[0112] Step S301: modify the image types of the reference image and the target image to double-precision image types.

[0113] Optionally, in some of the embodiments, the image types of the reference image and the target image can be modified to double-precision image types, so that the reference image and the target image of the modified double-precision image type can more finely represent the detail information in the image, which can improve the calculation accuracy of subsequent calculation processes, thereby improving the accuracy of the calculation results.

[0114] Step S302: performing multi-scale decomposition on the reference image and the target image to obtain reference images and target images at multiple scales.

[0115] Step S303: Perform Gaussian blur processing on the reference image at each scale to obtain a Gaussian reference image, and perform Gaussian blur processing on the target image to obtain a Gaussian target image.

[0116] like Figure 2As shown, taking the decomposition of the reference image and the target image into four scales corresponding to scales 1 to 4 as an example, Gaussian blur processing can be performed on the reference image and the target image at scales 1, 2, 3 and 4 respectively. Specifically, the Gaussian blur processing process may include performing Gaussian blur processing on the image, scaling the image obtained after the Gaussian blur processing, performing Gaussian blur processing on the scaled image again, and so on.

[0117] Step S304: Obtain the grayscale value of the Gaussian reference image and the grayscale value of the Gaussian target image at each scale.

[0118] Step S305 : Calculate the first variance of the reference image, the second variance of the target image, and the covariance between the reference image and the target image based on the grayscale value of the Gaussian reference image and the grayscale value of the Gaussian target image at each scale.

[0119] At each scale, the first variance of the reference image is calculated based on the grayscale value of the Gaussian reference image, the second variance of the target image is calculated based on the grayscale value of the Gaussian target image, and the covariance of the reference image and the target image is determined based on the grayscale value of the Gaussian target image and the grayscale value of the Gaussian reference image.

[0120] Optionally, in some embodiments, when the first variance of the reference image is less than 0, the first variance of the reference image is assigned to 0; when the second variance of the target image is less than 0, the second variance of the target image is assigned to 0, thereby ensuring that each variance is non-negative, avoiding computational problems caused by negative variances, and improving the stability and reliability of the algorithm.

[0121] Step S306: Calculate the numerator of the sub-scale visual information fidelity and the denominator of the sub-scale visual information fidelity at each scale.

[0122] like Figure 2 As shown, the sub-scale visual information fidelity at scale 1, scale 2, scale 3 and scale 4 can be calculated respectively, so as to calculate the visual information fidelity based on the sub-scale visual information fidelity at each scale. Specifically, the numerator of the sub-scale visual information fidelity and the denominator of the sub-scale visual information fidelity at each scale can be calculated respectively.

[0123] Exemplarily, the target image can be adjusted at each scale based on the first variance of the reference image and the covariance between the reference image and the target image to obtain the adjusted target image; the third variance of the adjusted target image is determined based on the second variance of the target image and the covariance between the reference image and the target image. For the specific calculation process of the third variance, please refer to formula (7) and formula (8) in the above embodiment. The numerator of the sub-scale visual information fidelity and the denominator of the sub-scale visual information fidelity of the target image are determined based on the first variance of the reference image and the third variance of the adjusted target image. For the calculation process of the numerator of the sub-scale visual information fidelity, please refer to formula (9) in the above embodiment, and for the calculation process of the denominator of the sub-scale visual information fidelity, please refer to formula (10) in the above embodiment.

[0124] Step S307: taking the mean of the numerators of the sub-scale visual information fidelity at each scale as the numerator of the visual information fidelity, and taking the mean of the denominators of the sub-scale visual information fidelity at each scale as the denominator of the visual information fidelity.

[0125] Step S308: Calculate the visual information fidelity based on the numerator of the visual information fidelity and the denominator of the visual information fidelity.

[0126] The visual information fidelity is calculated based on the numerator of the visual information fidelity and the denominator of the visual information fidelity. For the calculation process of the visual information fidelity, please refer to formula (11) in the above embodiment.

[0127] In the embodiments of the present application, on the one hand, by performing multi-scale decomposition on the reference image and the target image, the fidelity of visual information can be directly calculated in the spatial domain without the need for frequency domain conversion, thereby improving the calculation efficiency of the fidelity of visual information; on the other hand, by calculating the first variance of the reference image and the second variance of the target image based on the Gaussian reference image and the Gaussian target image after Gaussian blur processing, the influence of noise in the reference image and the target image can be reduced, thereby improving the calculation efficiency and accuracy of the first variance and the second variance; on still another hand, the calculation process of the fidelity of visual information is simplified, thereby improving the calculation efficiency of the fidelity of visual information.

[0128] See also Figure 4 , Figure 4 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 2 The image quality evaluation method of the embodiment of the present application comprises the following steps:

[0129] Step S401: input a target image and a reference image into a pre-trained evaluation model, and obtain target image features of the target image and reference image features corresponding to the reference image through a feature extraction model in the evaluation model.

[0130] Specifically, the evaluation model is a machine learning model for implementing image quality evaluation. The target image features may include pixel-level features, structural features, and visual information features of the target image, and the reference image features may include pixel-level features, structural features, and visual information features of the reference image. Among them, the pixel-level features may include grayscale values, color component values, etc., the structural features may include edge information, corner point information, etc., and the visual information features may include brightness information, chromaticity information, etc.

[0131] Exemplarily, the evaluation model can be trained based on the image features of the pre-trained image and the quality evaluation results by the determination method of the quality evaluation results in the above embodiment to obtain a pre-trained evaluation model. The target image features of the target image and the reference image features corresponding to the reference image are extracted by the feature extraction model for extracting image features in the evaluation model.

[0132] Step S402, using the peak signal-to-noise ratio calculation module in the evaluation model, based on the target image features and the reference image features, calculate the peak signal-to-noise ratio of the target image; using the structural similarity calculation module in the evaluation model, based on the target image features and the reference image features, calculate the structural similarity of the target image; using the visual information fidelity module in the evaluation model, based on the target image features and the reference image features, calculate the visual information fidelity of the target image.

[0133] Exemplarily, the peak signal-to-noise ratio calculation module in the evaluation model can be trained based on the peak signal-to-noise ratio calculation process in the above embodiment, the structural similarity calculation module in the evaluation model can be trained based on the structural similarity calculation process in the above embodiment, and the visual information fidelity module in the evaluation model can be trained based on the visual information fidelity calculation process in the above embodiment. The peak signal-to-noise ratio calculation module in the evaluation model that has been trained calculates the peak signal-to-noise ratio of the target image based on the target image features and the reference image features; the structural similarity calculation module in the evaluation model calculates the structural similarity of the target image based on the target image features and the reference image features; the visual information fidelity module in the evaluation model calculates the visual information fidelity of the target image based on the target image features and the reference image features.

[0134] Step S403, through the fusion module in the evaluation model, the peak signal-to-noise ratio is normalized to obtain a normalized peak signal-to-noise ratio; the visual information fidelity is normalized to obtain a normalized visual information fidelity; according to a preset weight ratio, the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused to obtain a quality evaluation result of the target image.

[0135] Exemplarily, the fusion module in the evaluation model can be trained based on the process of fusing the evaluation results of each single dimension of the target image in the above embodiment. The peak signal-to-noise ratio is normalized by the fusion module in the evaluation model completed through training to obtain the normalized peak signal-to-noise ratio; the visual information fidelity is normalized to obtain the normalized visual information fidelity; and the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused according to the preset weight ratio to obtain the quality evaluation result of the target image.

[0136] Optionally, in some of the embodiments, the quality evaluation result of the target image can be verified based on the image features and quality evaluation results of multiple pre-trained images and the target image features and quality evaluation results of the target image to determine the accuracy of the quality evaluation result output by the evaluation model. In the case where the accuracy of the quality evaluation result output by the evaluation model is less than a preset indicator, the evaluation model is retrained based on the image features and quality evaluation results of multiple pre-trained images to improve the accuracy of the evaluation model.

[0137] In the embodiment of the present application, the quality evaluation result of the target image is generated based on the target image features and the reference image features through the evaluation model, which can improve the efficiency of determining the quality evaluation result. And the fusion module in the evaluation model integrates the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity to obtain the quality evaluation result of the target image. On the one hand, the fusion of the normalized visual information fidelity can make the quality evaluation result of the target image closer to the visual perception of the human eye; on the other hand, the fusion of the evaluation results of multiple evaluation dimensions can improve the accuracy and objectivity of the quality evaluation result of the target image.

[0138] See also Figure 5 , Figure 5 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 3 .

[0139] Optionally, in some of the embodiments, the image quality evaluation method of the embodiment of the present application may include the following steps:

[0140] Step S501: Acquire a target image and a reference image to be evaluated.

[0141] Step S502: performing denoising and contrast enhancement processing on the target image and the reference image.

[0142] Optionally, in some of the embodiments, image preprocessing operations such as denoising and contrast enhancement can be performed on the target image and the reference image, so that the target image and the reference image after the denoising and contrast enhancement processing can more clearly present the structural features and visual information features in the image.

[0143] Step S503: extracting target image features of the target image and reference image features of the reference image.

[0144] Specifically, the target image features may include pixel-level features, structural features, and visual information features of the target image, and the reference image features may include pixel-level features, structural features, and visual information features of the reference image. Among them, the pixel-level features may include grayscale values, color component values, etc., the structural features may include edge information, corner point information, etc., and the visual information features may include brightness information, chromaticity information, etc.

[0145] Step S504: Calculate the peak signal-to-noise ratio, visual information fidelity, structural similarity and multi-scale structural similarity of the target image based on the target image features and the reference image features.

[0146] Specifically, based on the target image features and the reference image features, the peak signal-to-noise ratio, visual information fidelity, structural similarity and multi-scale structural similarity of the target image can be calculated respectively through the calculation process of the peak signal-to-noise ratio, visual information fidelity, structural similarity and multi-scale structural similarity of the target image in the above embodiment.

[0147] Step S505: perform weighted fusion on the peak signal-to-noise ratio, visual information fidelity, structural similarity and multi-scale structural similarity of the target image to obtain a quality evaluation result of the target image.

[0148] Specifically, the peak signal-to-noise ratio can be normalized to obtain a normalized peak signal-to-noise ratio; the visual information fidelity can be normalized to obtain a normalized visual information fidelity; and the normalized peak signal-to-noise ratio, normalized visual information fidelity, structural similarity and multi-scale structural similarity of the target image can be weighted fused according to a preset weight ratio to obtain a quality evaluation result of the target image.

[0149] Step S506: output the image quality rating of the target image.

[0150] Exemplarily, the image quality rating of the target image can be determined based on the quality evaluation result obtained by weighted fusion of the normalized peak signal-to-noise ratio, normalized visual information fidelity, structural similarity and multi-scale structural similarity of the target image. In some embodiments of the present application, the image quality rating corresponding to the quality evaluation result between 0.9 and 1 is A-level, the image quality rating corresponding to the quality evaluation result between 0.7 and 0.9 is B-level, the image quality rating corresponding to the quality evaluation result between 0.5 and 0.7 is C-level, and the image quality rating corresponding to the quality evaluation result between 0 and 0.5 is D-level. In some embodiments, the numerical range of the quality evaluation results corresponding to each image quality rating can be determined according to actual needs, and the present application does not specifically limit this.

[0151] In the embodiment of the present application, the quality evaluation result of the target image is obtained by weighted fusion of peak signal-to-noise ratio, visual information fidelity, structural similarity and multi-scale structural similarity. On the one hand, the fusion of normalized visual information fidelity can make the quality evaluation result of the target image closer to the visual perception of the human eye; on the other hand, the fusion of the evaluation results of multiple evaluation dimensions can improve the accuracy and objectivity of the quality evaluation result of the target image.

[0152] See also Figure 6 , Figure 6 Schematic diagram of the implementation process of the image quality evaluation method applicable to the embodiment of the present application Figure 4 .

[0153] Optionally, in some of the embodiments, the image quality evaluation method of the embodiment of the present application may include the following steps:

[0154] Step S601: Collect a plurality of pre-trained images corresponding to different image quality ratings and quality evaluation results of the pre-trained images.

[0155] Specifically, the pre-training images are images used to train the evaluation model.

[0156] Exemplarily, the quality evaluation results of multiple pre-trained images and the corresponding image quality ratings can be determined by the image quality evaluation method in the above embodiment. Corresponding to each image quality rating, multiple pre-trained images and the quality evaluation results corresponding to the pre-trained images are collected respectively.

[0157] Step S602: extract pixel-level features, structural features, and visual information features from each pre-trained image.

[0158] Specifically, pixel-level features may include grayscale values, color component values, etc., structural features may include edge information, corner point information, etc., and visual information features may include brightness information, chromaticity information, etc.

[0159] Step S603: training an evaluation model based on the quality evaluation results of each pre-trained image and the pixel-level features, structural features and visual information features.

[0160] Step S604: verifying the evaluation model based on the quality evaluation results of multiple pre-trained images and pixel-level features, structural features, and visual information features.

[0161] Optionally, in some embodiments, part of the pre-trained images can be used as a validation set, and the validation set can be input into the evaluation model to obtain the quality evaluation results corresponding to each pre-trained image in the validation set output by the evaluation model. The quality evaluation results output by the evaluation model are compared with the quality evaluation results of each pre-trained image in the validation set to determine the accuracy of the quality evaluation results output by the evaluation model.

[0162] Step S605: Adjust the parameters of the evaluation model according to the verification result of the evaluation model.

[0163] Exemplarily, the hyperparameters and model structure parameters of the evaluation model can be adjusted according to the verification results of the evaluation model, and the adjusted evaluation model can be re-verified based on the verification set until the accuracy of the quality evaluation results output by the evaluation model reaches the expected value.

[0164] Step S606: deploy the adjusted evaluation model to the image quality evaluation system.

[0165] Step S607: Receive the target image through the image quality evaluation system, extract the pixel-level features, structural features and visual information features of the target image, and output the quality evaluation result of the target image through the evaluation model.

[0166] Step S608: display the quality evaluation result of the target image through the image quality evaluation system.

[0167] Optionally, in some of the embodiments, the image quality evaluation system may obtain user feedback on the quality evaluation result of the target image, so as to adjust the evaluation model according to the user feedback.

[0168] In the embodiment of the present application, an evaluation model is trained based on the quality evaluation results of each pre-trained image and the pixel-level features, structural features and visual information features to output the quality evaluation results of the target image through the evaluation model. On the one hand, by outputting the quality evaluation results through the evaluation model, the efficiency of determining the quality evaluation results can be improved; on the other hand, the fusion of the visual information feature dimensions can make the quality evaluation results of the target image closer to the visual perception of the human eye; on the other hand, the fusion of the evaluation results of multiple evaluation dimensions such as pixel-level features, structural features and visual information features can improve the accuracy and objectivity of the quality evaluation results of the target image.

[0169] See also Figure 7 , Figure 7 The image quality evaluation device of the embodiment of the present application is a schematic diagram of the structure of the image quality evaluation device. The image quality evaluation device of the embodiment of the present application includes the following modules:

[0170] An acquisition module 701 is used to acquire a target image and a reference image to be evaluated;

[0171] The calculation module 702 is used to calculate the single-dimensional evaluation results of the target image in each evaluation dimension of the multiple evaluation dimensions based on the reference image; the multiple evaluation dimensions include at least two of the following: pixel difference dimension, structure difference dimension, and visual information difference dimension;

[0172] The evaluation module 703 is used to fuse the single-dimensional evaluation results of the target image to obtain the quality evaluation result of the target image.

[0173] Optionally, in some embodiments, the single-dimensional evaluation result corresponding to the pixel difference dimension is the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension is the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension is the visual information fidelity; the evaluation module 703 is specifically used to: normalize the peak signal-to-noise ratio to obtain a normalized peak signal-to-noise ratio; normalize the visual information fidelity to obtain a normalized visual information fidelity; and weightedly fuse the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity according to a preset weight ratio to obtain a quality evaluation result of the target image.

[0174] Optionally, in some embodiments, the calculation module 702 is specifically used to: determine a first variance of the reference image based on the grayscale value of the reference image, determine a second variance of the target image based on the grayscale value of the target image, and determine a covariance between the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image; adjust the target image based on the first variance and the covariance between the reference image and the target image to obtain an adjusted target image; determine a third variance of the adjusted target image based on the second variance and the covariance between the reference image and the target image; and determine the fidelity of the visual information of the target image based on the first variance and the third variance.

[0175] Optionally, in some embodiments, the calculation module 702 is specifically used to: perform Gaussian blur processing on the reference image to obtain a Gaussian reference image; perform Gaussian blur processing on the target image to obtain a Gaussian target image; determine a first variance based on the grayscale value of the Gaussian reference image, determine a second variance based on the grayscale value of the Gaussian target image, and determine the covariance of the reference image and the target image based on the grayscale value of the Gaussian target image and the grayscale value of the Gaussian reference image.

[0176] Optionally, in some of the embodiments, the computing module 702 is specifically used to: perform multi-scale decomposition on the reference image and the target image to obtain the reference image and the target image at multiple scales; determine the sub-scale visual information fidelity of the target image based on the grayscale value of the reference image and the grayscale value of the target image at each scale; and take the average of the sub-scale visual information fidelity of the target image at each scale as the visual information fidelity of the target image.

[0177] Optionally, in some of the embodiments, the calculation module 702 is specifically used to: input the target image and the reference image into a pre-trained evaluation model, and obtain the target image features of the target image and the reference image features corresponding to the reference image through the feature extraction model in the evaluation model; calculate the peak signal-to-noise ratio of the target image based on the target image features and the reference image features through the peak signal-to-noise ratio calculation module in the evaluation model; calculate the structural similarity of the target image based on the target image features and the reference image features through the structural similarity calculation module in the evaluation model; calculate the visual information fidelity of the target image based on the target image features and the reference image features through the visual information fidelity module in the evaluation model; normalize the peak signal-to-noise ratio through the fusion module in the evaluation model to obtain a normalized peak signal-to-noise ratio; normalize the visual information fidelity to obtain a normalized visual information fidelity; and weightedly fuse the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity according to a preset weight ratio to obtain a quality evaluation result of the target image.

[0178] The image quality evaluation device of this embodiment is used to implement the corresponding image quality evaluation method in the aforementioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the image quality evaluation device of this embodiment can refer to the description of the corresponding part in the aforementioned method embodiment, which will not be described in detail here.

[0179] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0180] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0181] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above computer program and computer program product embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the computer program and computer program product embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0182] Figure 8 A schematic diagram of a hardware entity of an electronic device applicable to an embodiment of the present application is shown in FIG. Figure 8 As shown, the hardware entity of the electronic device 800 includes: a processor 801 and a memory 802, wherein the memory 802 stores a computer program that can be run on the processor 801, and the processor 801 implements the steps in the method of any of the above embodiments when executing the program.

[0183] The memory 802 stores computer programs that can be run on the processor. The memory 802 is configured to store instructions and applications executable by the processor 801. It can also cache data to be processed or processed by the processor 801 and various modules in the electronic device 800 (for example, image data, audio data, voice communication data, and video communication data). This can be achieved through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0184] When the processor 801 executes the program, the steps of any of the above-mentioned image quality assessment methods are implemented. The processor 801 generally controls the overall operation of the electronic device 800.

[0185] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the image quality assessment method of any of the above embodiments.

[0186] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0187] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0188] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0189] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similarities can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0190] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0191] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0192] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0193] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0194] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks or optical disks.

[0195] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0196] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0197] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0198] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0199] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for evaluating image quality, characterized in that: include: Obtaining a target image to be evaluated and a reference image; Taking the reference image as a reference, calculating the single-dimensional evaluation result of the target image in each evaluation dimension of multiple evaluation dimensions respectively; the multiple evaluation dimensions include at least two of the following: a pixel difference dimension, a structure difference dimension, and a visual information difference dimension; The single-dimensional evaluation results of the target image are integrated to obtain a quality evaluation result of the target image.

2. The method according to claim 1, characterized in that The single-dimensional evaluation result corresponding to the pixel difference dimension is the peak signal-to-noise ratio, the single-dimensional evaluation result corresponding to the structural difference dimension is the structural similarity, and the single-dimensional evaluation result corresponding to the visual information difference dimension is the visual information fidelity; The fusing of the single-dimensional evaluation results of the target image to obtain the quality evaluation result of the target image includes: Normalizing the peak signal-to-noise ratio to obtain a normalized peak signal-to-noise ratio; normalizing the visual information fidelity to obtain a normalized visual information fidelity; According to a preset weight ratio, the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused to obtain a quality evaluation result of the target image.

3. The method according to claim 2, characterized in that The process of calculating the fidelity of the visual information of the target image includes: Determine a first variance of the reference image based on the grayscale value of the reference image, determine a second variance of the target image based on the grayscale value of the target image, and determine a covariance of the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image; Adjusting the target image based on the first variance and the covariance between the reference image and the target image to obtain an adjusted target image; determining a third variance of the adjusted target image based on the second variance and the covariance between the reference image and the target image; The fidelity of visual information of the target image is determined based on the first variance and the third variance.

4. The method according to claim 3, characterized in that The determining of a first variance of the reference image based on the grayscale value of the reference image, determining a second variance of the target image based on the grayscale value of the target image, and determining a covariance between the reference image and the target image based on the grayscale value of the target image and the grayscale value of the reference image comprises: Performing Gaussian blur processing on the reference image to obtain a Gaussian reference image; performing Gaussian blur processing on the target image to obtain a Gaussian target image; The first variance is determined based on the grayscale value of the Gaussian reference image, the second variance is determined based on the grayscale value of the Gaussian target image, and the covariance of the reference image and the target image is determined based on the grayscale value of the Gaussian target image and the grayscale value of the Gaussian reference image.

5. The method according to claim 2, characterized in that: The process of calculating the fidelity of the visual information of the target image includes: Performing multi-scale decomposition on the reference image and the target image to obtain the reference image and the target image at multiple scales; Determining the sub-scale visual information fidelity of the target image based on the grayscale value of the reference image and the grayscale value of the target image at each scale; The average of the sub-scale visual information fidelity of the target image at each scale is taken as the visual information fidelity of the target image.

6. The method according to any one of claims 1 to 5, characterized in that: The step of calculating the single-dimensional evaluation result of the target image in each evaluation dimension of the multiple evaluation dimensions based on the reference image comprises: Inputting the target image and the reference image into a pre-trained evaluation model, and obtaining target image features of the target image and reference image features corresponding to the reference image through a feature extraction model in the evaluation model; The peak signal-to-noise ratio calculation module in the evaluation model is used to calculate the peak signal-to-noise ratio of the target image based on the target image features and the reference image features; the structural similarity calculation module in the evaluation model is used to calculate the structural similarity of the target image based on the target image features and the reference image features; the visual information fidelity module in the evaluation model is used to calculate the visual information fidelity of the target image based on the target image features and the reference image features; The peak signal-to-noise ratio is normalized through the fusion module in the evaluation model to obtain a normalized peak signal-to-noise ratio; the visual information fidelity is normalized to obtain a normalized visual information fidelity; and the normalized peak signal-to-noise ratio, the normalized visual information fidelity and the structural similarity are weightedly fused according to a preset weight ratio to obtain a quality evaluation result of the target image.

7. An image quality evaluation device, characterized in that: include: An acquisition module, used for acquiring a target image and a reference image to be evaluated; A calculation module, used to calculate the single-dimensional evaluation results of the target image under each evaluation dimension of multiple evaluation dimensions based on the reference image; the multiple evaluation dimensions include at least two of the following: pixel difference dimension, structure difference dimension, and visual information difference dimension; The evaluation module is used to fuse the single-dimensional evaluation results of the target image to obtain a quality evaluation result of the target image.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the image quality assessment method according to any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to implement the image quality assessment method described in any one of claims 1 to 6 when executed by a processor.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the image quality assessment method according to any one of claims 1 to 6 is implemented.

Images