A no-reference image blur quality assessment method combining salient edge characteristics and global characteristics

Abstract

The present invention discloses a reference-free image fuzzy quality assessment method that combines significant edge characteristics and global characteristics, comprising the following steps: performing saliency detection on an input image to determine significant region image blocks and non-significant region image blocks; performing edge detection on significant region image blocks to determine non-edge blocks and edge blocks; distinguishing non-edge blocks based on texture features to determine smooth blocks and texture blocks; performing local quality assessment on edge blocks and texture blocks based on edge fuzzy width features to obtain significant local quality scores; performing global quality assessment on image blocks to obtain global quality scores; and fusing the significant local quality scores with the global quality scores. This method adopts an overall framework that combines significant local characteristics with global characteristics, calculates significant local quality scores and global quality scores separately, and fuses them to obtain the quality score of the entire input image, making more effective use of input image information and making the assessment results more comprehensive and reliable.

Description

Technical Field

[0001] The invention relates to a reference-free image fuzzy quality evaluation method combining significant edge characteristics and global characteristics, and belongs to the field of image quality evaluation. Background Art

[0002] With the rapid development of various digital image processing technologies, image quality assessment plays a vital role in applications such as digital image processing, computer graphics, and computer vision. However, image quality can be distorted and degraded throughout the acquisition, processing, and transmission processes, and these defects have a significant impact on the consumer experience. Due to the complexity of applying subjective quality assessment techniques, the development of reliable objective quality metrics has become a crucial research topic.

[0003] The goal of no-reference fuzzy image quality assessment is to develop an objective evaluation method that is highly consistent with human subjective evaluation. No-reference fuzzy image quality assessment methods can be roughly divided into traditional methods and deep learning methods.

[0004] In its early development, traditional no-reference blurred image quality assessment methods studied the different characteristics of image distortion in different variation domains in detail, calculating the image quality score by calculating the relevant eigenvalues ​​to characterize the degree of image blur. In addition, scholars have also conducted extensive research on the human visual system. These methods are effective when used to evaluate the blur level of images with the same content, but when used to evaluate the blur of images with different contents, these evaluation metrics are not consistent with the subjective quality of the image. With the emergence and gradual maturity of deep learning methods, a large number of researchers are currently using neural networks to train designed evaluation models with samples to obtain no-reference blurred image quality scores. However, problems such as the small number of samples and computational complexity have also limited their development. To this end, we propose a no-reference image blur quality assessment method that combines significant edge characteristics and global characteristics. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a reference-free image blur quality evaluation method that combines significant edge characteristics and global characteristics.

[0006] To solve the above technical problems, the present invention provides a reference-free image blur quality assessment method that combines significant local characteristics and global characteristics, comprising the following steps:

[0007] Perform saliency detection on the input image to determine salient area image blocks and non-salient area image blocks;

[0008] Perform edge detection on image blocks in the salient area to determine non-edge blocks and edge blocks;

[0009] Based on texture features, non-edge blocks are identified to determine smooth blocks and texture blocks;

[0010] Based on the edge blur width feature, the local quality of edge blocks and texture blocks is evaluated to obtain a significant local quality score;

[0011] Perform global quality evaluation on the image blocks to obtain a global quality score;

[0012] The significant local quality score and the global quality score are fused to obtain the quality score of the input image, and the overall quality of the input image is evaluated by the quality score of the input image.

[0013] Furthermore, saliency detection of the input image includes the following steps:

[0014] The saliency map is obtained by using the saliency detection method based on wavelet domain;

[0015] Divide the saliency map into blocks and calculate the average saliency value of each block, and determine whether it is a saliency region or a non-saliency region based on the average saliency value;

[0016] According to the mapping relationship between the saliency map and the spatial positions of the blocks of the input image, the salient area image blocks and the non-salient area image blocks of the input image are determined.

[0017] Furthermore, performing edge detection on the salient region image block includes the following steps:

[0018] Perform edge detection on each pixel in the salient area image block of the input image in the vertical direction and calculate the absolute value of the difference;

[0019] Classify edge points and non-edge points. If the absolute value of the difference is not less than the threshold, it is an edge point, otherwise it is a non-edge point.

[0020] Scan the input image line by line, and regard the local brightness extreme value closest to the edge point as the start and end position of the edge point. The width of the edge point is the difference between the end position and the start position, combined with the corresponding gradient angle to determine the width of each edge point;

[0021] The number of edge points in each pixel block is determined, and the edge block or non-edge block is determined based on the ratio of the number of edge points to the number of pixels in the pixel block.

[0022] Furthermore, the non-edge blocks are identified based on texture features, including:

[0023] The non-edge blocks are divided into relatively smooth smooth blocks and blurred texture blocks, and the texture blocks and the smooth blocks are judged based on whether the changes in their texture feature values ​​after high-frequency emphasis filtering and histogram equalization are obvious, including the following steps:

[0024] Calculate the non-edge blocks before processing separately , processed non-edge blocks Gray-level co-occurrence matrix , which is defined as Direction, distance A pair of pixels have grayscale values and The probability of occurrence is used to extract the corresponding texture features; the selected features are: angular second moment (energy) , inverse difference moment (local stationarity) ,entropy :

[0025]

[0026]

[0027]

[0028] The three eigenvalues ​​are combined to obtain the total texture eigenvalue :

[0029]

[0030] Using relative changes Indicates the changes in texture features:

[0031]

[0032] in, and are the non-edge blocks before processing and processed non-edge blocks The total texture feature value of =0.5 to determine its type; if It can be considered as a texture block, otherwise it is a smooth block.

[0033] Furthermore, local quality evaluation of edge blocks and texture blocks based on edge blur width features includes:

[0034] Calculate the just perceptible blur edge width for each edge point in the edge block :

[0035]

[0036] in is the contrast, which is the maximum pixel value in an edge block With the minimum pixel value The difference is:

[0037]

[0038] The fuzzy edge width eigenvalue is obtained using the following formula: :

[0039]

[0040] in, is the edge point, is the edge point width, is the width of the just perceptible blur edge; define For different fuzzy edge width eigenvalues The ratio of the number of corresponding edge points to the total number of pixels is used to obtain the local quality score of the edge block. :

[0041]

[0042] in, For the time When , substitute the local quality score calculation formula of the edge block to obtain the fuzzy edge width feature value; The larger it is, the better the local quality of the salient edge block is, and vice versa.

[0043] Local quality score of the texture block :

[0044]

[0045] in, For the time When , substitute the fuzzy edge width feature value obtained by improving the fuzzy edge width feature formula; The larger the value is, the better the local quality of the salient texture block is, and vice versa.

[0046] Furthermore, the significant local quality score is obtained by the local quality score of the edge block and local quality scores of texture blocks in non-edge blocks Fusion yields:

[0047]

[0048] in, is the significant local mass fraction, is the weight of the edge block, obtained through experimental fitting ; The larger the value is, the better the significant local quality of the image is, and vice versa.

[0049] Furthermore, the global quality evaluation of the image block includes:

[0050] The input image is divided into blocks of 32×32 pixels, and the global quality score of each block is represented by the maximum gradient and the gradient change value;

[0051] Calculate the maximum gradient in each block as , and calculate the gradient change value of each block ,in Represent the width and length of a block respectively; the maximum gradient value and gradient change value of each block are fused to obtain the global quality score of each block :

[0052]

[0053] in, is the weight coefficient of the maximum gradient, obtained by experimental fitting .

[0054] Furthermore, the global quality evaluation of the image block also includes:

[0055] Calculate the mean quality score of all blocks of the image and then calculate the standard deviation as the global quality score, including;

[0056] Calculate the mean of the quality scores of all blocks :

[0057]

[0058] in, is the number of blocks of the input image;

[0059] Then use the standard deviation formula to express the global quality score of this image:

[0060]

[0061] That is, the global quality score, The larger it is, the better the global quality of the image is, and vice versa.

[0062] Furthermore, the fusion of significant local quality scores and global quality scores includes:

[0063] For significant local quality fraction and global quality score Fusion is performed to obtain the quality score of the entire input image:

[0064]

[0065] That is, the quality score of the input image, which is determined by experiments The experience value is ; The larger the value, the better the overall quality of the image, and vice versa.

[0066] The beneficial effects achieved by the present invention are:

[0067] First, this method adopts an overall framework that combines significant local and global features. It calculates significant local quality scores and global quality scores separately, and finally fuses them to obtain the quality score of the entire input image. This makes full use of the information of the input image and makes the evaluation result more comprehensive and reliable.

[0068] Second, when evaluating significant local quality, we categorized non-edge areas into smooth and textured areas based on changes in texture features. We also evaluated the textured areas using local quality characteristics similar to those of edge areas, making the experimental results more consistent with human visual perception.

[0069] 3. When evaluating significant local quality, we made some improvements based on the original fuzzy edge width characteristic formula, making the experimental data more consistent with the actual results, simplifying the calculation, and greatly reducing the program running time;

[0070] Fourth, when conducting global quality evaluation, the standard deviation formula is used to characterize the global quality of the input image by the discrete degree of the image gradient value, making the experimental results more representative. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A flow chart of a method for evaluating blur quality of a reference-free image by combining significant edge characteristics and global characteristics is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0073] Example

[0074] A no-reference image blur quality assessment method combining salient edge characteristics and global characteristics, such as Figure 1 As shown, it includes the following six steps:

[0075] Step 1) Perform saliency detection on the input image and determine the salient regions.

[0076] In the implementation of the present invention, a saliency detection method based on the wavelet domain is used to obtain a saliency map. Specifically, the saliency map is divided into 32×32 blocks. The average saliency value of each block is calculated. If the average saliency value is greater than 0.3, it is recorded as a salient region, otherwise it is a non-salient region. The saliency map is mapped to the spatial position of the blocks in the input image to obtain the salient regions of the input image, namely multiple 32×32 pixel blocks.

[0077] Step 2) Perform edge detection on the image blocks in the salient area to determine whether they are edge blocks.

[0078] Perform edge detection on each pixel in the salient area of ​​the input image in the vertical direction, calculate the absolute value of the difference, set the threshold to 2 to classify edge points and non-edge points, if the absolute value of the difference is greater than or equal to 2, it is an edge point, otherwise it is a non-edge point; scan the input image line by line, for each edge point, the local brightness extreme value closest to the edge point is regarded as the start and end position of the edge point, that is, the width of the edge point is the difference between the end position and the start position, combined with the corresponding gradient angle to determine the width of each edge point ; Then, the number of edge points in a 32×32 pixel block is determined. If the number of edge points accounts for no less than 0.2% of the total number of pixels in the pixel block, it is determined to be an edge block, otherwise it is a non-edge block.

[0079] Step 3) Classify smooth blocks and texture blocks according to texture features.

[0080] The non-edge blocks are divided into relatively smooth smooth blocks and blurred texture blocks; the texture blocks and smooth blocks are judged based on whether the changes in their texture feature values ​​after high-frequency emphasis filtering and histogram equalization are obvious;

[0081] Calculate the non-edge blocks before processing separately , processed non-edge blocks Gray-level co-occurrence matrix , which is defined as Direction, distance A pair of pixels have grayscale values and The probability of occurrence is used to extract the corresponding texture features; the selected features are: angular second moment (energy) , inverse difference moment (local stationarity) ,entropy :

[0082]

[0083]

[0084]

[0085] The three eigenvalues ​​are combined to obtain the total texture eigenvalue :

[0086]

[0087] Using relative changes Indicates the changes in texture features:

[0088]

[0089] in, and are the non-edge blocks before processing and processed non-edge blocks The total texture feature value of

[0090] Setting thresholds =0.5 to determine its type; if It can be considered as a texture block, otherwise it is a smooth block.

[0091] The improvements in the implementation of the present invention are as follows:

[0092] When performing significant local quality evaluation, the present invention classifies non-edge areas into smooth areas and texture areas according to changes in texture features, and performs a local characteristic quality evaluation on the texture area similar to that of the edge area, so that the experimental results are more consistent with the visual perception of the human eye.

[0093] Step 4) Calculate the significant local quality score based on the edge blur width feature.

[0094] Calculate the just perceptible blur edge width for each edge point in the edge block :

[0095]

[0096] in is the contrast, which is the maximum pixel value in an edge block With the minimum pixel value The difference is:

[0097]

[0098] The fuzzy edge width eigenvalue is obtained using the following formula: :

[0099]

[0100] in, is the edge point, is the edge point width, is the width of the just perceptible blur edge; define For different fuzzy edge width eigenvalues The ratio of the number of corresponding edge points to the total number of pixels is used to obtain the local quality score of the edge block. :

[0101] ;

[0102] in, For the time When , substitute the fuzzy edge width eigenvalue obtained by the above formula;

[0103] The larger it is, the better the local quality of the salient edge block is, and vice versa.

[0104] Local quality score of the texture block :

[0105] ;

[0106] in, For the time When , substitute the fuzzy edge width feature value obtained by improving the fuzzy edge width feature formula;

[0107] The larger it is, the better the local quality of the significant texture block is, and vice versa.

[0108] Local quality score for edge blocks and local quality scores of texture blocks in non-edge blocks Fusion is performed to obtain significant local quality scores :

[0109]

[0110] in, is the weight of the edge block, obtained through experimental fitting ;

[0111] The larger the value is, the better the significant local quality of the image is, and vice versa.

[0112] The improvements in the implementation of the present invention are as follows:

[0113] The present invention makes certain improvements on the basis of the original fuzzy edge width characteristic formula, so that the experimental data is more consistent with the actual results, and simplifies the calculation, so that the program running time is greatly reduced.

[0114] Step 5) Calculate the global quality score for the input image blocks.

[0115] The input image is divided into blocks of 32×32 pixels, and the global quality score of each block is represented by the maximum gradient and the gradient change value;

[0116] Calculate the maximum gradient in each block as , and calculate the gradient change value of each block ,in Represent the width and length of a block respectively; the maximum gradient value and gradient change value of each block are fused to obtain the global quality score of each block :

[0117]

[0118] in, is the weight coefficient of the maximum gradient, obtained by experimental fitting ;

[0119] Calculate the mean quality score of all blocks in the image, and then calculate the standard deviation as the global quality score;

[0120] Calculate the mean of the quality scores of all blocks :

[0121]

[0122] in, is the number of blocks of the input image;

[0123] Then use the standard deviation formula to express the global quality score of this image :

[0124]

[0125] The larger it is, the better the global quality of the image is, and vice versa.

[0126] The improvements in the implementation of the present invention are as follows:

[0127] When performing global quality evaluation, the present invention adopts a standard deviation formula to characterize the global quality of the input image by the discrete degree of the image gradient value, so that the experimental results are more representative.

[0128] Step 6) Fuse the local and global quality scores to obtain the quality score of the entire input image.

[0129] For significant local quality fraction and global quality score Fusion is performed to obtain the quality score of the entire input image :

[0130]

[0131] Determined by experiment The experience value is ;

[0132] The larger the value, the better the overall quality of the image, and vice versa.

[0133] This method adopts an overall framework that combines significant local characteristics with global characteristics. It calculates the significant local quality score and the global quality score respectively in combination with the saliency map, and finally fuses them to obtain the quality score of the entire input image, making more full use of the information of the input image and making the evaluation results more comprehensive and reliable. When performing significant local quality evaluation, we classify the non-edge area into smooth areas and texture areas according to the changes in texture characteristics, and perform a local characteristic quality evaluation of the texture area similar to that of the edge area, so that the experimental results are more consistent with the visual perception of the human eye. When performing significant local quality evaluation, we make certain improvements on the basis of the original fuzzy edge width feature formula, so that the experimental data is more consistent with the actual results, and the calculation is simplified, which greatly reduces the running time of the program. When performing global quality evaluation, we use the standard deviation formula to characterize the global quality of the input image by the discrete degree of the image gradient value, so that the experimental results are more representative.

[0134] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products; therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects; moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application; it should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as the combination of the processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions; these computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A no-reference image blur quality assessment method combining salient edge characteristics and global characteristics, characterized in that: The following steps are involved: Perform saliency detection on the input image to determine salient area image blocks and non-salient area image blocks; Perform edge detection on image blocks in the salient area to determine non-edge blocks and edge blocks; Based on texture features, non-edge blocks are identified to determine smooth blocks and texture blocks; Based on the edge blur width feature, the local quality of edge blocks and texture blocks is evaluated to obtain a significant local quality score; Perform global quality evaluation on the image blocks to obtain a global quality score; The quality score of the input image is obtained by fusing the significant local quality score with the global quality score, and the overall quality of the input image is evaluated by the quality score of the input image; The discrimination of non-edge blocks based on texture features includes: The non-edge blocks are divided into relatively smooth smooth blocks and blurred texture blocks, and the texture blocks and the smooth blocks are judged based on whether the changes in their texture feature values ​​after high-frequency emphasis filtering and histogram equalization are obvious, including the following steps: Calculate the gray level co-occurrence matrix of the non-edge block S before processing and the non-edge block S' after processing respectively , which is defined as In the direction, a pair of pixels separated by a distance d have the probability of having grayscale values ​​i and j respectively, so as to extract the corresponding texture features; the selected features are: angular second-order moment C1, inverse difference moment C2, entropy C3: ； ； ； The three eigenvalues ​​are combined to obtain the total texture eigenvalue C: ； The relative change u is used to represent the change of texture features: ； Among them, C and C' are the total texture feature values ​​of the non-edge block S before processing and the non-edge block S' after processing, respectively; the threshold t=0.5 is set to distinguish its type; if u≥t, it can be considered as a texture block, otherwise it is a smooth block.

2. The method for evaluating blur quality of a non-reference image by combining salient edge characteristics and global characteristics according to claim 1, wherein: Saliency detection of an input image consists of the following steps: The saliency map is obtained by using the saliency detection method based on wavelet domain; Divide the saliency map into blocks and calculate the average saliency value of each block, and determine whether it is a saliency region or a non-saliency region based on the average saliency value; According to the mapping relationship between the saliency map and the spatial positions of the blocks of the input image, the salient area image blocks and the non-salient area image blocks of the input image are determined.

3. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 1, characterized in that: Edge detection of salient area image blocks includes the following steps: Perform edge detection on each pixel in the salient area image block of the input image in the vertical direction and calculate the absolute value of the difference; Classify edge points and non-edge points. If the absolute value of the difference is not less than the threshold, it is an edge point, otherwise it is a non-edge point. Scan the input image line by line, and regard the local brightness extreme value closest to the edge point as the start and end position of the edge point. The width of the edge point is the difference between the end position and the start position, combined with the corresponding gradient angle to determine the width of each edge point; The number of edge points in each pixel block is determined, and the edge block or non-edge block is determined based on the ratio of the number of edge points to the number of pixels in the pixel block.

4. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 1, characterized in that: The local quality evaluation of edge blocks and texture blocks based on edge blur width features includes: Calculate the just perceptible blur edge width ω of each edge point in the edge block JNB (e i ): ； Among them C i is the contrast, which is the maximum pixel value I in an edge block max With the minimum pixel value I min The difference is: ； The fuzzy edge width eigenvalue is obtained using the following formula: : ； Among them, e i is the edge point, is the edge point width, is the width of the just perceptible blur edge; define For different fuzzy edge width eigenvalues The ratio of the number of corresponding edge points to the total number of pixels is used to obtain the local quality score q1 of the edge block: ； in, Substitute the fuzzy edge width feature value into the local quality score calculation formula of the edge block; the larger q1 is, the better the local quality of the significant edge block is, and vice versa; The local quality score q2 of the texture block: ； in, Substitute the fuzzy edge width feature value obtained by the fuzzy edge width feature improvement formula; the larger q2 is, the better the local quality of the significant texture block is, and vice versa.

5. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 4, characterized in that: The significant local quality score is obtained by fusing the local quality score q1 of the edge block and the local quality score q2 of the texture block in the non-edge block: ； Among them, Q1 is the salient local quality score, η is the weight of the edge block, and η = 7.2 is obtained through experimental fitting. The larger Q1 is, the better the salient local quality of the image, and vice versa.

6. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 5, characterized in that: The global quality evaluation of image blocks includes: The input image is divided into 32×32 pixel units, and the global quality score of each block is represented by the maximum gradient and gradient change value; Calculate the maximum gradient in each block as , and calculate the gradient change value of each block , where W and H represent the width and length of a block respectively; the maximum gradient value and gradient change value of each block are fused to obtain the global quality score of each block : ； in, is the weight coefficient of the maximum gradient, obtained by experimental fitting = 0.

69.

7. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 6, characterized in that: The global quality evaluation of the image block also includes: Calculate the mean quality score of all blocks of the image and then calculate the standard deviation as the global quality score, including; Calculate the mean of the quality scores of all blocks : ； Where N is the number of input image blocks; Then use the standard deviation formula to express the global quality score of this image: ； Q2 is the global quality score. The larger the Q2, the better the global quality of the image, and vice versa.

8. The method for no-reference image blur quality assessment combining salient edge characteristics and global characteristics according to claim 7, characterized in that: Fusion of significant local quality scores and global quality scores includes: The significant local quality score Q1 and the global quality score Q2 are fused to obtain the quality score of the entire input image: ； Q is the quality score of the input image, which is determined by experiments. The experience value is 1.17, =0.014; the larger the Q, the better the overall quality of the image, and vice versa.

Images