Full-reference image quality assessment method based on splitting structural distortion and signal error

By splitting the full reference image quality evaluation method of structural distortion and signal error, the Prewitt operator calculates the image gradient and divides the characteristic areas, the difficulty in evaluating structural distortion and pixel-level signal distortion in the prior art is solved, and the more Accurate image quality evaluation and low computational complexity.

CN113971675BActive Publication Date: 2024-11-22NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111191503.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-13
Publication Date
2024-11-22
Estimated Expiration
2041-10-13

AI Technical Summary

Technical Problem

Existing image quality evaluation methods are difficult to accurately evaluate structural distortion and pixel-level signal distortion in distorted images, especially when human vision systems are sensitive to both, and the computational complexity is high, which cannot meet the real-time image processing requirements.

Method used

The full reference image quality evaluation method based on split structure distortion and signal error is adopted. The image gradient is calculated by the Prewitt operator, the structure-keeping pixels, structural change pixels, flat area pixels and texture areas are divided, and the gradient similarity standard deviation and The average signal error energy is fused to the final mass fraction.

Benefits of technology

It more accurately expresses the perception mechanism of distortion of different characteristics of human visual system, the calculation complexity is only slightly higher than the mean square error, lower than the structural similarity measurement, and the evaluation results are consistent with the human eye evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113971675B_ABST
    Figure CN113971675B_ABST
Patent Text Reader

Abstract

The present invention discloses a full-reference image quality evaluation method based on splitting structural distortion and signal error, comprising: reading in a reference image and a distorted image to be measured; calculating the gradient amplitude at each pixel position and the gradient mean of the reference image; calculating two gradient thresholds for distinguishing structural distortion from flat or textured areas according to the gradient mean of the reference image; for an area with structural distortion characteristics, calculating the standard deviation of gradient amplitude similarity as a structural distortion score, and for an area with signal error characteristics, calculating the average signal error energy as a signal distortion score; fusing the structural distortion score and the signal distortion score as a final quality score of the distorted image; the method adopts appropriate methods to measure distortion respectively and then fuses them into a final overall image quality score, thereby making better use of the perception characteristics of the human visual system to different types of distortion, and the obtained quality evaluation result is more consistent with the evaluation of the human eye.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a full-reference image quality evaluation method based on splitting structural distortion and signal error, and belongs to the technical field of image quality evaluation. Background Art

[0002] Image quality assessment has important applications in many areas of image processing, such as image restoration, image coding, image communication, etc., and is currently a hot research topic in the field of image processing. Due to the fact that different types and degrees of distortion are often introduced at different stages of image processing, as well as the diversity of image content and its mutual masking relationship with distortion, it is difficult to evaluate the quality of distorted images. At present, the most accurate image quality assessment method is still the subjective evaluation method based on manual scoring, but the subjective evaluation method is time-consuming and labor-intensive and cannot be applied to real-time image processing systems. Therefore, an objective evaluation method is proposed to automatically evaluate the quality of distorted images.

[0003] At present, the most widely used in image processing systems are still the classic mean square error (MSE) or peak signal-to-noise ratio (PSNR), which are mainly based on pixel-level error for quality measurement. The advantages are simple calculation, clear physical meaning, and good evaluation effect on pixel-level distortion (such as noise distortion), but the disadvantage is that the evaluation effect on structural distortion (such as JPEG compression) is not good. Considering that the human visual system (HVS) is very sensitive to the structural information in the image, the structural similarity (SSIM) measurement has been proposed and widely used. The structural similarity mainly measures the change of the structural information of the distorted image compared with the reference image, so the evaluation effect on pixel-level distortion is average, and the calculation amount is much higher than that of the mean square error.

[0004] Considering that structural distortion and pixel-level signal distortion often exist simultaneously in distorted images, and the human visual system is very sensitive to both structural distortion and signal distortion, but with different perceptual characteristics, we propose to spatially split the distorted image into structural distortion area and signal distortion area, measure the distortion separately and then fuse them for quality evaluation, which achieves better quality evaluation effect. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a full-reference image quality evaluation method based on splitting structural distortion and signal error.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a full-reference image quality assessment method based on splitting structural distortion and signal error, comprising:

[0008] Read in the reference image and the distorted image to be measured;

[0009] The Prewitt operator is used to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then the gradient amplitude of each pixel position and the gradient mean of the reference image are calculated;

[0010] Two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image;

[0011] According to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels;

[0012] The structure-preserving pixels and the structure-changing pixel regions are combined into a structure-distortion characteristic region, and the flat region pixels and the texture region pixels are combined into a signal-error characteristic region;

[0013] For the structural distortion characteristic region, the standard deviation of the gradient amplitude similarity is calculated as the structural distortion score, and for the signal error characteristic region, the average signal error energy is calculated as the signal distortion score;

[0014] The structural distortion score and the signal distortion score are fused as the final quality score of the distorted image.

[0015] Furthermore, the use of the Prewitt operator to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then calculating the gradient amplitude of each pixel position and the gradient mean of the reference image, includes:

[0016] The Prewitt horizontal and vertical gradient operators are used to perform template operations on the pixel areas at the corresponding positions of the two images to calculate the horizontal and vertical gradients of each pixel position of the two images. The Prewitt horizontal and vertical gradient operators (gx, gy) are as follows:

[0017]

[0018] The horizontal and vertical gradients calculated at the pixel coordinate (i, j) of the reference image R are and The horizontal and vertical gradients of the distorted image D calculated at the pixel coordinate (i, j) are and The gradient amplitude of the reference image R and the distorted image D at the position (i, j) is calculated according to the following formula ( and ):

[0019]

[0020] The gradient mean of the reference image R is calculated as follows:

[0021]

[0022] Wherein, M and N are the width and height of the reference image R respectively.

[0023] Furthermore, the two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image, including:

[0024] According to the gradient mean of the reference image R The threshold T1 for distinguishing structural areas and the threshold T2 for distinguishing flat or textured areas are calculated using the following formula:

[0025]

[0026] Among them, α and β are empirical constants, and their values ​​are 1 and 0.6 respectively according to experiments.

[0027] Furthermore, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels according to the relationship between the gradient amplitudes of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, including:

[0028] if and This means that the larger gradient amplitude at position (i, j) of the reference image R is maintained in the distorted image D, and the pixel (i, j) is defined as a structure-preserving pixel;

[0029] if and or and It means that the larger gradient amplitude at position (i, j) of the reference image R becomes smaller in the distorted image D due to distortion, or the smaller gradient amplitude becomes larger in the distorted image D due to distortion. The pixel (i, j) is defined as a structural change pixel.

[0030] if and This means that the smaller gradient amplitude at position (i, j) of the reference image R is still smaller in the distorted image D, and the pixel (i, j) is defined as a flat area pixel;

[0031] Except for the above three cases, it means that the gradient amplitude of the reference image R and the distorted image D at the position (i, j) is at a medium level, and the pixel (i, j) is defined as a texture area pixel.

[0032] Furthermore, the principle of merging the structure-preserving pixels and the structure-changing pixel areas into a structure-distortion characteristic area and merging the flat area pixels and the texture area pixels into a signal error characteristic area is based on the different perception mechanisms of the human visual system for different types of distortion.

[0033] Furthermore, the step of calculating the standard deviation of the gradient amplitude similarity as the structural distortion score for the structural distortion characteristic region includes:

[0034] For the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity of each pixel in the reference image R and the distorted image D in this area is calculated as the structural distortion score. The calculation formula is as follows:

[0035]

[0036] SD score =std(SDGS i,j )

[0037] in, and They represent the gradient amplitude of the reference image R and the distorted image D at the position (i, j) in the structural distortion characteristic region, T is an empirical constant, which is 170 according to the experiment, SDGS i,j is the gradient amplitude similarity at position (i, j), SD score is the standard deviation of the gradient amplitude similarity, which is defined as the structural distortion score of the structural distortion characteristic area. The larger its value is, the greater the structural change distortion is and the lower the image quality is.

[0038] Furthermore, calculating the average signal error energy as the signal distortion score for the signal error characteristic region includes:

[0039] For the signal error characteristic area, the average signal error energy of each pixel in the reference image R and the distorted image D in this area is calculated as the signal distortion score. The calculation formula is as follows:

[0040]

[0041] Among them, N se is the number of pixels in the signal error characteristic area, R k With D k are the k-th pixel brightness values ​​of the reference image R and the distorted image D in this area, SE score is the average signal error energy, defined as the signal distortion score in this area. The larger the value, the greater the signal error and the lower the image quality.

[0042] Furthermore, the fusing the structural distortion score and the signal distortion score as a final quality score of the distorted image includes:

[0043] The following formula is defined to fuse the structural distortion score and the signal distortion score as the final quality score of the distorted image:

[0044] SDSE score =SD score +γ×log(SE score +ε)

[0045] Among them, ε is an empirical small constant to maintain the stability of logarithmic operation, and its value is 0.001. γ is the balance factor between the structural distortion score and the signal distortion score, and its value is 0.05 according to the experiment. SDSE score is the final quality score of the distorted image.

[0046] In a second aspect, the present invention provides a full-reference image quality assessment device based on splitting structural distortion and signal error, comprising:

[0047] An input unit, used for reading in a reference image and a distorted image to be measured, and extracting brightness components of the reference image and the distorted image to be measured for quality evaluation;

[0048] A first calculation unit, used for calculating the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured by using a Prewitt operator, and then calculating the gradient amplitude of each pixel position and the gradient mean of the reference image;

[0049] A second calculation unit calculates two gradient thresholds for distinguishing structural distortion from flat or textured areas according to the gradient mean of the reference image;

[0050] A division unit, used for dividing the distorted image into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels according to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds;

[0051] A merging unit, used to merge the structure-preserving pixels and the structure-changing pixel area into a structure-distortion characteristic area, and merge the flat area pixels and the texture area pixels into a signal error characteristic area;

[0052] A third calculation unit is used to calculate the standard deviation of the gradient amplitude similarity as the structural distortion score for the structural distortion characteristic region, and calculate the average signal error energy as the signal distortion score for the signal error characteristic region;

[0053] A fusion unit is used to fuse the structure distortion score and the signal distortion score as a final quality score of the distorted image.

[0054] In a third aspect, the present invention provides a full-reference image quality assessment device based on splitting structural distortion and signal error, comprising a processor and a storage medium;

[0055] The storage medium is used to store instructions;

[0056] The processor is used to operate according to the instructions to execute the steps of any of the methods described above.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. When evaluating the quality of a distorted image, this method spatially splits it into a structural distortion characteristic region and a signal error characteristic region, and performs distortion measurement on each region, which better expresses the perception mechanism of the human visual system for distortions of different characteristics.

[0059] 2. For the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity is used as the structural distortion score, and for the signal error characteristic area, the average signal error energy is used as the signal distortion score, which more accurately calculates the distortion scores of different characteristic areas; the final evaluation result of the fusion of the structural distortion score and the signal distortion score is more consistent with the human eye evaluation, and the calculation complexity is only slightly higher than the mean square error and lower than the structural similarity measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is a flow chart of a full-reference image quality evaluation method based on splitting structural distortion and signal error provided by an embodiment of the present invention;

[0061] Figure 2 It is a schematic diagram of test results of a differential subjective evaluation score (DMOS) and a provided algorithm quality prediction score (SDSE) on the most commonly used LIVE image quality evaluation dataset provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0063] Example 1

[0064] This embodiment introduces a full-reference image quality assessment method based on splitting structural distortion and signal error, including:

[0065] Read in the reference image and the distorted image to be measured;

[0066] The Prewitt operator is used to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then the gradient amplitude of each pixel position and the gradient mean of the reference image are calculated;

[0067] Two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image;

[0068] According to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels;

[0069] The structure-preserving pixels and the structure-changing pixel regions are combined into a structure-distortion characteristic region, and the flat region pixels and the texture region pixels are combined into a signal-error characteristic region;

[0070] For the structural distortion characteristic region, the standard deviation of the gradient amplitude similarity is calculated as the structural distortion score, and for the signal error characteristic region, the average signal error energy is calculated as the signal distortion score;

[0071] The structural distortion score and the signal distortion score are fused as the final quality score of the distorted image.

[0072] The full reference image quality evaluation method based on splitting structural distortion and signal error provided in this embodiment is as follows: Figure 1 As shown, the application process specifically involves the following steps:

[0073] Step 1) read in a reference image and a distorted image to be tested, and extract the brightness components of the two images for quality evaluation.

[0074] Step 2) Use the Prewitt operator to calculate the horizontal and vertical gradients of each pixel position in the two images, and then calculate the gradient amplitude of each pixel position and the gradient mean of the reference image.

[0075] In the implementation of the present invention, the Prewitt horizontal and vertical gradient operators are used to perform template operations on the pixel areas at corresponding positions of the two images to calculate the horizontal and vertical gradients of each pixel position of the two images. x ,g y )as follows:

[0076]

[0077] The horizontal and vertical gradients calculated at the pixel coordinate (i, j) of the reference image R are and The horizontal and vertical gradients of the distorted image D calculated at the pixel coordinate (i, j) are and The gradient amplitude of the reference image R and the distorted image D at the position (i, j) is calculated according to the following formula ( and ):

[0078]

[0079] The gradient mean of the reference image R is calculated as follows:

[0080]

[0081] Wherein, M and N are the width and height of the reference image R respectively.

[0082] Step 3) Two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image, and the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels, and texture area pixels in units of pixels based on the relationship between the gradient amplitude at each pixel position of the two images and the two gradient thresholds.

[0083] In the implementation of the present invention, according to the gradient mean of the reference image R The threshold T1 for distinguishing structural areas and the threshold T2 for distinguishing flat or textured areas are calculated using the following formula:

[0084]

[0085] Among them, α and β are empirical constants, and their values ​​are 1 and 0.6 respectively according to experiments;

[0086] According to the relationship between the gradient amplitude of each pixel position of the two images and the two gradient thresholds, the rule for dividing the distorted image into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels is defined as follows:

[0087] 1) If and This means that the larger gradient amplitude at position (i, j) of the reference image R is maintained in the distorted image D, and the pixel (i, j) is defined as a structure-preserving pixel;

[0088] 2) If and or and It means that the larger gradient amplitude at position (i, j) of the reference image R becomes smaller in the distorted image D due to distortion, or the smaller gradient amplitude becomes larger in the distorted image D due to distortion. The pixel (i, j) is defined as a structural change pixel.

[0089] 3) If and This means that the smaller gradient amplitude at position (i, j) of the reference image R is still smaller in the distorted image D, and the pixel (i, j) is defined as a flat area pixel;

[0090] 4) Except for the above three cases, it means that the gradient amplitude of the reference image R and the distorted image D at the position (i, j) is at a medium level, and the pixel (i, j) is defined as a texture area pixel.

[0091] The improvements in the implementation of the present invention are as follows:

[0092] Two gradient thresholds for distinguishing structural distortion and flat or textured areas are calculated based on the gradient mean of the reference image. The distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels on a pixel-by-pixel basis according to the relationship between the gradient amplitude at each pixel position of the two images and the two gradient thresholds. The division results of areas with different distortion characteristics are more stable.

[0093] Step 4) The structure-preserving pixels and the structure-changing pixel regions are merged into the structural distortion characteristic region, and the flat region pixels and the texture region pixels are merged into the signal error characteristic region, which is more in line with the perception mechanism of the human visual system for different characteristic distortions.

[0094] Step 5) For the structural distortion characteristic region, the standard deviation of the gradient amplitude similarity is calculated as the structural distortion score, and for the signal error characteristic region, the average signal error energy is calculated as the signal distortion score.

[0095] In the implementation of the present invention, for the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity of each pixel in the reference image R and the distorted image D in this area is calculated as the structural distortion score, and the calculation formula is as follows:

[0096]

[0097] SD score =std(SDGS i,j )

[0098] in, and They represent the gradient amplitude of the reference image R and the distorted image D at the position (i, j) in the structural distortion characteristic region, T is an empirical constant, which is 170 according to the experiment, SDGS i,j is the gradient amplitude similarity at position (i, j), SD score is the standard deviation of the gradient amplitude similarity, which is defined as the structural distortion score of the structural distortion characteristic area. The larger its value, the greater the structural change distortion and the lower the image quality;

[0099] For the signal error characteristic area, the average signal error energy of each pixel in the reference image R and the distorted image D in this area is calculated as the signal distortion score. The calculation formula is as follows:

[0100]

[0101] Among them, N se is the number of pixels in the signal error characteristic area, R k With D k are the k-th pixel brightness values ​​of the reference image R and the distorted image D in this area, SE score is the average signal error energy, defined as the signal distortion score in this area. The larger the value, the greater the signal error and the lower the image quality.

[0102] The improvements in the implementation of the present invention are as follows:

[0103] For the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity is used as the structural distortion score, and for the signal error characteristic area, the average signal error energy is used as the signal distortion score, which more accurately calculates the distortion scores of different characteristic areas.

[0104] Step 6) The structural distortion score and the signal distortion score are fused as the final quality score of the distorted image.

[0105] In the implementation of the present invention, the following formula is defined to fuse the structural distortion score and the signal distortion score as the final quality score of the distorted image:

[0106] SDSE score =SD score +γ×log(SE score +ε)

[0107] Among them, ε is an empirical small constant to maintain the stability of logarithmic operation, and its value is 0.001. γ is the balance factor between the structural distortion score and the signal distortion score, and its value is 0.05 according to the experiment. SDSE score is the final quality score of the distorted image.

[0108] Table 1 shows the SROCC, KROCC, PLCC and RMSE indicators of the full-reference image quality assessment method based on splitting structural distortion and signal error proposed in the present invention tested on different types of distorted images and all distorted images on the LIVE image quality assessment dataset.

[0109] Table 1

[0110] index JP2K JPEG WN Gblur FF All SROCC 0.9854 0.9799 0.9870 0.9668 0.9587 0.9733 KROCC 0.9007 0.8976 0.9073 0.8457 0.8363 0.8673 PLCC 0.9822 0.9838 0.9854 0.9701 0.9496 0.9669 RMSE 5.4360 6.3199 5.6408 5.8208 9.8151 7.9845

[0111] Table 1 shows the overall performance of the method of the present invention on the LIVE image quality evaluation dataset. The results show that the method of the present invention has achieved good quality evaluation results for different types of distorted images, and has good consistency with the evaluation results of human eyes.

[0112] Example 2

[0113] This embodiment provides a full-reference image quality assessment device based on splitting structural distortion and signal error, including:

[0114] An input unit, used for reading in a reference image and a distorted image to be measured, and extracting brightness components of the reference image and the distorted image to be measured for quality evaluation;

[0115] A first calculation unit, used for calculating the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured by using a Prewitt operator, and then calculating the gradient amplitude of each pixel position and the gradient mean of the reference image;

[0116] A second calculation unit calculates two gradient thresholds for distinguishing structural distortion from flat or textured areas according to the gradient mean of the reference image;

[0117] A division unit, used for dividing the distorted image into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels according to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds;

[0118] A merging unit, used to merge the structure-preserving pixels and the structure-changing pixel area into a structure-distortion characteristic area, and merge the flat area pixels and the texture area pixels into a signal error characteristic area;

[0119] A third calculation unit is used to calculate the standard deviation of the gradient amplitude similarity as the structural distortion score for the structural distortion characteristic region, and calculate the average signal error energy as the signal distortion score for the signal error characteristic region;

[0120] A fusion unit is used to fuse the structure distortion score and the signal distortion score as a final quality score of the distorted image.

[0121] Example 3

[0122] This embodiment provides a full-reference image quality assessment device based on splitting structural distortion and signal error, including a processor and a storage medium;

[0123] The storage medium is used to store instructions;

[0124] The processor is configured to operate according to the instructions to perform the steps of any of the following methods:

[0125] Reading in a reference image and a distorted image to be measured, and extracting brightness components of the reference image and the distorted image to be measured for quality evaluation;

[0126] The Prewitt operator is used to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then the gradient amplitude of each pixel position and the gradient mean of the reference image are calculated;

[0127] Two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image;

[0128] According to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels;

[0129] The structure-preserving pixels and the structure-changing pixel regions are combined into a structure-distortion characteristic region, and the flat region pixels and the texture region pixels are combined into a signal-error characteristic region;

[0130] For the structural distortion characteristic region, the standard deviation of the gradient amplitude similarity is calculated as the structural distortion score, and for the signal error characteristic region, the average signal error energy is calculated as the signal distortion score;

[0131] The structural distortion score and the signal distortion score are fused as the final quality score of the distorted image.

[0132] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0136] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A full-reference image quality assessment method based on splitting structural distortion and signal error, It is characterized in that include: Read in the reference image and the distorted image to be measured; The Prewitt operator is used to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then the gradient amplitude of each pixel position and the gradient mean of the reference image are calculated; Two gradient thresholds for distinguishing structural distortion from flat or textured areas are calculated based on the gradient mean of the reference image; According to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels; The structure-preserving pixels and the structure-changing pixel regions are combined into a structure-distortion characteristic region, and the flat region pixels and the texture region pixels are combined into a signal-error characteristic region; For the structural distortion characteristic region, the standard deviation of the gradient amplitude similarity is calculated as the structural distortion score, and for the signal error characteristic region, the average signal error energy is calculated as the signal distortion score; fusing the structural distortion score and the signal distortion score as a final quality score of the distorted image; The step of calculating the standard deviation of the gradient amplitude similarity as the structural distortion score for the structural distortion characteristic region includes: For the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity of each pixel in the reference image R and the distorted image D in this area is calculated as the structural distortion score. The calculation formula is as follows: ; ; in, and They represent the gradient amplitude of the reference image R and the distorted image D at the position (i, j) in the structural distortion characteristic region, T is an empirical constant, which is 170 according to the experiment, SDGS i,j is the gradient amplitude similarity at position (i, j), SD score is the standard deviation of the gradient amplitude similarity, which is defined as the structural distortion score of the structural distortion characteristic area. The larger its value, the greater the structural change distortion and the lower the image quality; The step of calculating the average signal error energy as a signal distortion score for the signal error characteristic region includes: For the signal error characteristic area, the average signal error energy of each pixel in the reference image R and the distorted image D in this area is calculated as the signal distortion score. The calculation formula is as follows: ; Among them, N se is the number of pixels in the signal error characteristic area, R k With D k are the k-th pixel brightness values ​​of the reference image R and the distorted image D in this area, SE score is the average signal error energy, defined as the signal distortion score in this area. The larger the value, the greater the signal error and the lower the image quality.

2. The full-reference image quality assessment method based on splitting structural distortion and signal error according to claim 1, Features: The use of the Prewitt operator to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured, and then calculating the gradient amplitude of each pixel position and the gradient mean of the reference image, includes: The Prewitt horizontal and vertical gradient operators are used to perform template operations on the pixel areas at the corresponding positions of the two images to calculate the horizontal and vertical gradients of each pixel position of the two images. The Prewitt horizontal and vertical gradient operators (gx, gy) are as follows: ; The horizontal and vertical gradients calculated at the pixel coordinate (i, j) of the reference image R are and , the horizontal and vertical gradients of the distorted image D calculated at the pixel coordinate (i, j) are and , then the gradient amplitude of the reference image R and the distorted image D at the position (i, j) is calculated according to the following formula ( and ): ; The gradient mean of the reference image R is calculated as follows: : ; Wherein, M and N are the width and height of the reference image R respectively.

3. The full-reference image quality assessment method based on splitting structural distortion and signal error according to claim 1, Features: The calculating two gradient thresholds for distinguishing structural distortion from flat or textured areas according to the gradient mean of the reference image comprises: According to the gradient mean of the reference image R Calculate the threshold T to distinguish structural areas 1 and the threshold T for distinguishing flat or textured areas 2 , the calculation formula is as follows: , ; Among them, α and β are empirical constants, and their values ​​are 1 and 0.6 respectively according to experiments.

4. The full-reference image quality assessment method based on splitting structural distortion and signal error according to claim 1, Features: According to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds, the distorted image is divided into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels, including: if and , it means that the larger gradient amplitude of the reference image R at position (i, j) is maintained in the distorted image D, and the pixel (i, j) is defined as the structure-preserving pixel; if and ,or and , it means that the larger gradient amplitude at position (i, j) of the reference image R becomes smaller in the distorted image D due to distortion, or the smaller gradient amplitude becomes larger in the distorted image D due to distortion. The pixel (i, j) is defined as a structural change pixel; if and , it means that the smaller gradient amplitude of the reference image R at position (i, j) is still smaller in the distorted image D, and the pixel (i, j) is defined as a flat area pixel; Except for the above three cases, it means that the gradient amplitude of the reference image R and the distorted image D at the position (i, j) is at a medium level, and the pixel (i, j) is defined as a texture area pixel.

5. The full-reference image quality assessment method based on splitting structural distortion and signal error according to claim 1, Features: The principle of merging the structure-preserving pixels and the structure-changing pixel regions into a structure-distortion characteristic region and merging the flat region pixels and the texture region pixels into a signal-error characteristic region is based on the different perception mechanisms of the human visual system for different types of distortion.

6. The full-reference image quality assessment method based on splitting structural distortion and signal error according to claim 1, Features: The fusing the structural distortion score and the signal distortion score as the final quality score of the distorted image includes: The following formula is defined to fuse the structural distortion score and the signal distortion score as the final quality score of the distorted image: ; Among them, ε is an empirical small constant to maintain the stability of logarithmic operation, and its value is 0.

001. γ is the balance factor between the structural distortion score and the signal distortion score, and its value is 0.05 according to the experiment. SDSE score is the final quality score of the distorted image.

7. A full-reference image quality assessment device based on splitting structural distortion and signal error, It is characterized in that include: An input unit, used for reading in a reference image and a distorted image to be measured, and extracting brightness components of the reference image and the distorted image to be measured for quality evaluation; A first calculation unit, configured to calculate the horizontal and vertical gradients of each pixel position of the reference image and the distorted image to be measured by using a Prewitt operator, and then calculate the gradient amplitude of each pixel position and the gradient mean of the reference image; A second calculation unit calculates two gradient thresholds for distinguishing structural distortion from flat or textured areas according to the gradient mean of the reference image; A division unit, used for dividing the distorted image into structure-preserving pixels, structure-changing pixels, flat area pixels and texture area pixels in units of pixels according to the relationship between the gradient amplitude of each pixel position of the reference image and the distorted image to be measured and two gradient thresholds; A merging unit, used to merge the structure-preserving pixels and the structure-changing pixel area into a structure-distortion characteristic area, and merge the flat area pixels and the texture area pixels into a signal error characteristic area; A third calculation unit is used to calculate the standard deviation of the gradient amplitude similarity as the structural distortion score for the structural distortion characteristic region, and calculate the average signal error energy as the signal distortion score for the signal error characteristic region; A fusion unit, used for fusing the structural distortion score and the signal distortion score as a final quality score of the distorted image; In the third calculation unit, for the structural distortion characteristic region, calculating the standard deviation of the gradient amplitude similarity as the structural distortion score includes: For the structural distortion characteristic area, the standard deviation of the gradient amplitude similarity of each pixel in the reference image R and the distorted image D in this area is calculated as the structural distortion score. The calculation formula is as follows: ; ; in, and They represent the gradient amplitude of the reference image R and the distorted image D at the position (i, j) in the structural distortion characteristic region, T is an empirical constant, which is 170 according to the experiment, SDGS i,j is the gradient amplitude similarity at position (i, j), SD score is the standard deviation of the gradient amplitude similarity, which is defined as the structural distortion score of the structural distortion characteristic area. The larger its value, the greater the structural change distortion and the lower the image quality; In the third calculation unit, for the signal error characteristic region, calculating the average signal error energy as the signal distortion score includes: For the signal error characteristic area, the average signal error energy of each pixel in the reference image R and the distorted image D in this area is calculated as the signal distortion score. The calculation formula is as follows: ; Among them, N se is the number of pixels in the signal error characteristic area, R k With D k are the k-th pixel brightness values ​​of the reference image R and the distorted image D in this area, SE score is the average signal error energy, defined as the signal distortion score in this area. The larger the value, the greater the signal error and the lower the image quality.

8. A full-reference image quality assessment device based on splitting structural distortion and signal error, Features: including processor and storage medium; The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 6.