Image quality assessment method, system, electronic device and storage medium

Abstract

The present invention provides an image quality assessment method, system, electronic device, and storage medium. The method includes: denoising an image to be assessed based on multiple convolution kernels, calculating a gradient map of the denoised image to be assessed, and standardizing the gradient map; binarizing the standardized gradient map, and extracting a region of interest in the image to be assessed based on the binarized gradient map; calculating a quality score for each denoised image to be assessed based on the gradient matrix and the area of ​​the region of interest, and calculating a final quality score for the image to be assessed. The present invention filters an image based on multiple convolution kernels and extracts a region of interest in the image based on the gradient map, effectively removing image noise, and evaluating the image quality through a combined score, further reducing the impact of noise on the gradient and improving the accuracy of the assessment.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and more particularly to an image quality assessment method, system, electronic device and storage medium. Background Art

[0002] Image quality assessment can be categorized as either subjective or objective. Subjective assessment is unstable, time-consuming, and labor-intensive, making it impractical for widespread use in production. Objective assessment uses image processing algorithms to perform qualitative and quantitative analysis of image quality. If these algorithms could be used to automatically and accurately predict results that are consistent with subjective assessments, the shortcomings of subjective assessment could be overcome and applied in production practice.

[0003] Image quality assessment occurs in two scenarios: with a reference (reference) and without a reference (no reference or blind). The former can be achieved by comparing the correlation or similarity with a reference image at the low-level semantics (pixel level) or high-level semantics (multiple convolutions). Research results in this area are relatively mature. The latter, on the other hand, is used in a wide range of applications where a reference image is unavailable. In this scenario, the ever-changing nature of images poses significant challenges for no-reference image quality assessment algorithms. This is particularly true for images acquired using scanning electron microscopes (SEMs) under suboptimal lighting conditions or with moving objects. These images are characterized by high noise levels and irregular noise distribution, blurred or insufficient edge information, and inconsistencies in the objects within the image. Previous image processing algorithms, due to their poor noise immunity, were unable to achieve results consistent with subjective assessments. Summary of the Invention

[0004] In response to the technical problems existing in the prior art, the present invention provides an image quality assessment method, system, electronic device and storage medium. The method has the characteristics of anti-interference and anti-noise, and can also work normally in scenes with insufficient edge information and drifting objects in the image.

[0005] According to a first aspect of the present invention, there is provided an image quality assessment method, comprising:

[0006] Denoising the image to be evaluated based on multiple different convolution kernels, calculating the gradient map of the denoised image to be evaluated, and normalizing the gradient map;

[0007] Binarizing the normalized gradient map, and extracting a region of interest in the image to be evaluated based on the binarized gradient map;

[0008] Calculating a quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest;

[0009] Based on the quality score of the image to be evaluated after denoising by each convolution kernel, the final quality score of the image to be evaluated is calculated.

[0010] On the basis of the above technical solution, the present invention can also make the following improvements.

[0011] Optionally, binarizing the normalized gradient map and extracting a region of interest in the image to be evaluated based on the binarized gradient map includes:

[0012] The gradient map is binarized using a clustering algorithm, the image to be evaluated is divided into two categories based on the binarized gradient map, and a region consisting of pixels of a category with a large gradient value is extracted as a region of interest in the image to be evaluated.

[0013] Optionally, the region of interest includes multiple sub-regions of interest, each sub-region of interest includes at least one pixel point, the normalized gradient map is binarized, and the region of interest in the image to be evaluated is extracted based on the binarized gradient map, and then the method further includes:

[0014] Eliminate abnormal sub-regions of interest in the region of interest;

[0015] The step of calculating the quality score of each image to be evaluated after denoising by the convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest includes:

[0016] Based on the gradient matrix and area of ​​the region of interest after removal, the quality score of the image to be evaluated after each convolution kernel denoising is calculated.

[0017] Optionally, removing abnormal sub-regions of interest in the region of interest includes:

[0018] Calculating the gradient average of each sub-region of interest;

[0019] The sub-regions of interest whose average gradient value is greater than or equal to the preset gradient threshold are eliminated.

[0020] Optionally, calculating the quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest includes:

[0021] Calculate the gain coefficient corresponding to each convolution kernel based on the maximum gradient in the gradient matrix of the region of interest, the standard deviation of the gradient matrix and the area of ​​the region of interest;

[0022] Based on the gain coefficient, the gradient matrix of the region of interest, the binary mask of the gradient matrix of the region of interest, and the area of ​​the region of interest, the quality score of the image to be evaluated after each convolution kernel denoising is calculated.

[0023] Optionally, the step of calculating the gain coefficient corresponding to each convolution kernel based on the maximum gradient value in the gradient matrix of the region of interest, the standard deviation of the gradient matrix, and the area of ​​the region of interest includes:

[0024] F w =MAX(GN(x,y))×[STD(G(x,y))] 2 / A roi ;

[0025] Among them, F w is the gain coefficient corresponding to the w-th convolution kernel, MAX(GN(x, y)) is the maximum gradient in the gradient matrix GN(x, y) after standardization of the region of interest, STD(G(x, y)) is the standard deviation of the gradient matrix, A roi is the area of ​​the region of interest;

[0026] The step of calculating the quality score of the image to be evaluated after each convolution kernel denoising based on the gain coefficient, the gradient matrix of the region of interest, the binary mask of the gradient matrix of the region of interest, and the area of ​​the region of interest includes:

[0027]

[0028] Among them, M ROI is the binary mask of the gradient matrix, which is obtained by clustering algorithm, ∑|GN(x, y)∧M ROI | is the gradient value in the normalized gradient matrix GN(x, y) and the binary mask M ROI The sum of the absolute values ​​after logical AND, A roi is the area of ​​the region of interest, S w is the quality score of the image to be evaluated after denoising by the w-th convolution kernel.

[0029] Optionally, the calculating of a final quality score of the image to be evaluated based on the quality score of each image to be evaluated after denoising by the convolution kernel includes:

[0030]

[0031] Among them, n is the number of convolution kernels, w∈[1,n],k w is the w-th convolution kernel size, S w is the quality score of the image to be evaluated after denoising by the w-th convolution kernel, and Score is the final quality score of the image to be evaluated.

[0032] According to a second aspect of the present invention, there is provided an image quality assessment system, comprising:

[0033] A first computing module denoises the image to be evaluated based on multiple convolution kernels, calculates a gradient map of the denoised image to be evaluated, and standardizes the gradient map;

[0034] an extraction module, configured to binarize the normalized gradient map and extract a region of interest in the image to be evaluated based on the binarized gradient map;

[0035] A second calculation module is used to calculate the quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest;

[0036] The third calculation module is used to calculate the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel.

[0037] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein the processor is configured to implement the steps of the image quality assessment method when executing a computer management program stored in the memory.

[0038] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of the image quality assessment method are implemented.

[0039] The present invention provides an image quality assessment method, system, electronic device, and storage medium. These methods filter images based on multiple convolution kernels and extract regions of interest in images based on gradient maps, effectively removing image noise. Furthermore, the combined scoring method is used to evaluate image quality, further reducing the impact of noise on gradients and improving assessment accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flow chart of an image quality assessment method provided by an embodiment of the present invention;

[0041] Figure 2 This is an overall flow chart of an image quality assessment method according to an embodiment of the present invention;

[0042] FIG3 is a schematic diagram of images corresponding to different image processing processes according to an embodiment of the present invention, wherein:

[0043] Figure 3(a) is a schematic diagram of an ideal image, Figure 3(b) is a schematic diagram of an actual image, Figure 3(c) is a schematic diagram of a blurred image, Figure 3(d) is a gradient map corresponding to Figure 3(a), Figure 3(e) is a gradient map corresponding to Figure 3(b), Figure 3(f) is a gradient map corresponding to Figure 3(c), Figure 3(g) is a schematic diagram of a region of interest extracted based on Figure 3(d), Figure 3(h) is a schematic diagram of a region of interest extracted based on Figure 3(e), and Figure 3(i) is a schematic diagram of a region of interest extracted based on Figure 3(f);

[0044] Figure 4 is a schematic diagram of the final quality scores of 10 images obtained in one embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of final quality scores of 10 images obtained in another embodiment of the present invention;

[0046] Figure 6 A schematic structural diagram of an image quality assessment system provided by an embodiment of the present invention;

[0047] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention;

[0048] Figure 8 A schematic diagram of the hardware structure of a computer-readable storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0050] Example 1

[0051] An image quality assessment method mainly comprises: denoising an image to be assessed based on multiple different convolution kernels, calculating a gradient map of the denoised image to be assessed, and standardizing the gradient map; binarizing the standardized gradient map, and extracting a region of interest in the image to be assessed based on the binarized gradient map; calculating a quality score of the image to be assessed after denoising each convolution kernel based on a gradient matrix of the region of interest and the area of ​​the region of interest; and calculating a final quality score of the image to be assessed based on the quality score of the image to be assessed after denoising each convolution kernel.

[0052] Understandably, image quality assessment (IQA) based on pixel correlation or gradients cannot achieve ideal results in scenes with high noise levels and insufficient edge information. Deep learning, with its nonlinear learning capabilities, enables end-to-end feature extraction of both low-level and high-level pixel semantics. However, this approach requires massive data collection and significant manual annotation costs, making it uneconomical in practical applications.

[0053] Based on this, an embodiment of the present invention discloses a non-deep learning image quality assessment method, which can be used to accurately assess the quality of images with uncertain noise interference and sparse edge images. The method uses a variety of convolution kernels to filter and denoise the image, and then calculates the gradient of each pixel in the image to form a gradient map of the image, and then dynamically binarizes the gradient map to remove noise interference in non-edge areas, and then introduces a gain coefficient to further optimize the score, and finally obtains the final image quality assessment score by weighting the scores of different filters. The embodiment of the present invention has very good generalization and anti-interference effects in the no reference (No Reference or Blind) mode, and can be widely used in automatic optical inspection (AOI), scanning electron beam microscope (SEM) inspection, auto focus (Auto Focus) and other fields. It is a highly reliable image quality assessment method.

[0054] Example 2

[0055] A method for image quality assessment, see Figure 1 , the method comprises the following steps:

[0056] S1, denoise the image to be evaluated based on multiple convolution kernels, calculate the gradient map of the denoised image to be evaluated, and standardize the gradient map.

[0057] It's understandable that noise can affect the calculated pixel changes. Therefore, the image needs to be filtered using a convolution kernel to remove noise. However, because the magnitude of the noise is uncertain, it's impossible to select a fixed, suitable convolution kernel. A convolution kernel that's too small won't effectively remove noise; a convolution kernel that's larger than the edge will blur the edge, losing a crucial basis for determining image clarity. Therefore, in step S1, multiple different convolution kernels are used to denoise the image, yielding the image to be evaluated after filtering and denoising using each convolution kernel.

[0058] Exemplarily, for each image to be evaluated after denoising by a convolution kernel, calculating the gradient map of the denoised image to be evaluated includes:

[0059] G(x, y) = |Gx |+|G y | (1);

[0060] Where (x, y) is the pixel point in the image to be evaluated, G(x, y) is the gradient map (gradient matrix) of the image to be evaluated, which is composed of the gradient values ​​of each pixel point in the image to be evaluated. x and G y They are the gradient components calculated based on the horizontal edge detection operator H and the vertical edge detection operator V, respectively. For the convenience of calculation, the sum of absolute values ​​is used for approximation.

[0061] The horizontal edge detection operator H and the vertical edge detection operator V are both n×n edge detection operators, where n is a positive integer. For example, H and V can be:

[0062]

[0063]

[0064] In equations (2) and (3), H is an example of a horizontal edge detection operator with a size of 3×3, and V is an example of a vertical edge detection operator with a size of 3×3. Similarly, an edge detection operator with a size of n×n can be set.

[0065]

[0066]

[0067] Among them, I is the image to be evaluated, G x and G y I is convolved with an odd-sized kernel respectively, which reflects the speed at which the pixels in the image change in the horizontal and vertical directions. That is, the gradient component calculated based on the horizontal edge detection operator H and the vertical edge detection operator V can be used to calculate the approximate gradient of the image grayscale, and the larger the gradient, the higher the edge clarity.

[0068] It should be noted that after calculating the gradient matrix of the image to be evaluated, the gradient matrix is ​​normalized, and the expression is:

[0069]

[0070] Where G(x, y) is the gradient matrix, L is the height of the gradient matrix, W is the width of the gradient matrix, ∑(G(x, y)) / (L×W) is the average value in the gradient matrix, STD(G(x, y)) is the standard deviation of the gradient matrix, and GN(x, y) is the normalized gradient matrix. After normalization, it can be better used to compare the quality of images with different content and textures.

[0071] S2, binarizing the normalized gradient map, and extracting the region of interest in the image to be evaluated based on the binarized gradient map.

[0072] As an embodiment, the binarization of the gradient map and the extraction of the region of interest in the image to be evaluated based on the binarized gradient map include: binarizing the standardized gradient map using a clustering algorithm, dividing the image to be evaluated into two categories based on the binarized gradient map, and extracting a region composed of pixels of a category with large gradient values ​​as the region of interest in the image to be evaluated.

[0073] It can be understood that after the gradient map of the image to be evaluated is calculated in the above step S1, the gradient map of the image to be evaluated is binarized in this step. For example, the gradient map can be binarized based on a clustering algorithm (for example, Otsu's method). The element values ​​in the binarized gradient map are 0 and 1. Based on the binarized gradient map, the image to be evaluated is divided into two categories, and the pixels corresponding to the element value of 1 are divided into the region of interest.

[0074] Edge information is the key to judging the clarity of an image. Therefore, in step S2, the gradient matrix of the image is binarized, and the areas above the threshold are segmented out as the regions of interest of the image to be evaluated. In the subsequent processing, only the gradient values ​​of these regions of interest are calculated. On the one hand, the gradient values ​​generated by noise in small areas can be excluded, and these gradient values ​​will affect the final score to a certain extent. On the other hand, it can effectively widen the difference in scores, which is conducive to distinguishing the quality of images with little difference. In step S2, the category with large gradient is selected as the region of interest through the binarized gradient map, and the gradient mean of the region is calculated. This attention mechanism can focus more on the areas that truly distinguish image quality, ultimately improving the accuracy of the score.

[0075] It should be noted that although the region of interest is extracted from the image to be evaluated through gradient threshold segmentation, the region of interest therein includes multiple sub-regions of interest, each of which includes at least one pixel point, wherein each sub-region of interest can be a connected region or a non-connected region. Exemplarily, the sub-region of interest includes only a single pixel point; exemplarily, the sub-region of interest includes multiple pixels and forms a connected region, but is not limited thereto. However, due to certain special circumstances, such as when the edge features of only a small part of the local area are too obvious, it is very likely caused by abnormal noise. Therefore, the embodiment of the present invention performs an abnormal elimination process on the extracted region of interest, that is, sets a threshold to exclude abnormal sub-regions of interest in the region of interest, which can improve the anti-interference ability, especially the anti-noise interference ability.

[0076] As an embodiment, removing abnormal sub-regions of interest in a region of interest includes: calculating the gradient average of any sub-region of interest in the multiple sub-regions of interest:

[0077] TH j =∑G roi / A cd (7);

[0078] Among them, TH j is the average gradient of the jth sub-region of interest, G roi is the gradient value of each pixel in the jth subregion of interest, ∑G roi is the sum of the gradient values ​​of all pixels in the j-th subregion of interest, A cd is the area of ​​the j-th sub-region of interest, that is, the number of pixels; the sub-regions of interest whose average gradient value is greater than or equal to the preset gradient threshold are eliminated.

[0079] Specifically, the above steps extract multiple sub-regions of interest from the image to be evaluated, and for each sub-region of interest, calculate its gradient average value TH j , set the filtering rule threshold of the abnormal area to determine whether to exclude the abnormal sub-region of interest. The abnormal sub-region of interest often has A cd The characteristics of very small and extremely large gradients are that these connected regions with values ​​above the filter rule threshold need to be removed. This can eliminate abnormal sub-regions of interest and retain the remaining sub-regions of interest. Subsequent image quality assessment is performed based on the sub-regions of interest after abnormal elimination. The filter rule threshold can be set to a corresponding value according to different applications.

[0080] S3: Calculate the quality score of each image to be evaluated after denoising by the convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest.

[0081] As an embodiment, the quality score of the image to be evaluated after denoising by each convolution kernel is calculated based on the gradient matrix of the region of interest and the area of ​​the region of interest, including: calculating the gain coefficient corresponding to each convolution kernel based on the maximum gradient value in the gradient matrix of the region of interest, the standard deviation of the gradient matrix and the area of ​​the region of interest; calculating the quality score of the image to be evaluated after denoising by each convolution kernel based on the gain coefficient, the gradient matrix of the region of interest, the binary mask of the gradient matrix of the region of interest and the area of ​​the region of interest.

[0082] It can be understood that subsequent operations are all performed based on the region of interest after the abnormal area is eliminated. In the process of calculating the quality score of the image to be evaluated after each convolution kernel denoising, the gain coefficient of each convolution kernel is first calculated, and then the corresponding quality score is calculated based on the gain coefficient.

[0083] Specifically, to select an appropriate gain coefficient, it is necessary to ensure that the objects in the image are consistent and aligned. Therefore, the application of focusing is selected as an example. During the focusing process, as the image clarity increases, the maximum value of the gradient in the region of interest tends to be larger, and the variance of the gradient tends to be larger. However, the number of pixels (approximate area) in the ROI region of interest decreases, and vice versa. Therefore, in step S3, several features with a high correlation with image quality are selected as gain coefficients, and the gradient mean score is linearly multiplied. The score result will become more sensitive and overall widen the score of images of different quality, thereby better reflecting the subtle differences in image quality.

[0084] Since image quality is strongly correlated with the maximum gradient, the standard deviation of the gradient, and the area of ​​the region of interest, they are combined as a gain coefficient, and then this gain coefficient is used to widen the score gap.

[0085] The specific gain coefficient calculation formula is:

[0086] F w =MAX(GN(x,y))×[STD(G(x,y))] 2 / A roi (8);

[0087] Among them, F w is the gain coefficient corresponding to the w-th convolution kernel, MAX(GN(x, y)) is the maximum gradient in the gradient matrix GN(x, y) after standardization of the region of interest, STD(G(x, y)) is the standard deviation of the gradient matrix, A roi is the area of ​​the region of interest.

[0088] The quality score of the image to be evaluated after filtering and denoising by each convolution kernel is calculated according to the gain coefficient of each convolution kernel. The calculation formula is:

[0089]

[0090] Among them, M ROI is the binary mask of the gradient matrix, which is obtained by clustering algorithm, ∑|GN(x, y)∧M ROI | is the gradient value in the normalized gradient matrix GN(x, y) and the binary mask M ROI The sum of the absolute values ​​after logical AND is used to combine GN(x, y) with the binary mask MROI The purpose of performing logical AND is to obtain the above-mentioned region of interest, A roi is the area of ​​the region of interest, S w The quality score of the image to be evaluated after denoising with the wth convolution kernel. A mask is used to focus on the region of interest to avoid noise interference. The dimensionless score is obtained by dividing the average normalized gradient by the area of ​​the region of interest. Finally, the gain coefficient is used to increase the difference in the score.

[0091] S4, calculating a final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel.

[0092] It is understood that the above step S3 calculates the quality score of each image to be evaluated after filtering and denoising with the convolution kernel. This step performs a weighted summation of the quality scores of multiple images to be evaluated after filtering and denoising with the convolution kernel to obtain the final quality score of the image to be evaluated. The specific expression for obtaining the final quality score of the image to be evaluated by weighted summation is:

[0093]

[0094] Among them, n is the number of convolution kernels, w∈[1,n],k w is the w-th convolution kernel size, S w is the quality score of the image to be evaluated after denoising by the w-th convolution kernel, and Score is the final quality score of the image to be evaluated.

[0095] A higher score indicates better image quality. This multi-feature fusion plays a crucial role in the final image quality score. This strategy eliminates various noise interferences and captures more image features, making this embodiment of the present invention highly resistant to interference and generalizable, significantly improving the accuracy of the score.

[0096] Based on the denoised image, the embodiment of the present invention obtains the gradient matrices of the image to be evaluated after denoising using multiple different convolution kernel filters in step S2, and obtains different scores in subsequent steps. Because the degree of denoising and the gradient value are approximately proportional, formula (10) is used to perform a weighted summation, which integrates the information of the gradient features of the multiple denoised images. This multi-feature fusion strategy can effectively reduce the impact of certain types of noise on the gradient, and therefore, the embodiment of the present invention has strong noise resistance.

[0097] Example 3

[0098] A method for image quality assessment, see Figure 2First, the image is blurred by Gaussian Blur or Median Blur using a combination of multiple convolution kernels. The purpose of this is to prevent the gradient generated by noise from affecting the final score. Then, the gradient of each pixel in the image in the X and Y directions is calculated using the Sobel operator or the Scharr operator to obtain a gradient map, as shown in formulas (1) to (6). In order to eliminate noise interference in non-edge areas, the attention mechanism is introduced. First, the maximum inter-class variance (OTSU) of the gradient map is found and binarized, and then the area below the threshold is removed to extract the region of interest in the image. Then, the extremely abnormal areas are excluded by filtering rules, and the extremely small number of edges or noise are excluded, as shown in formula (7). Then, the maximum value, standard deviation and area of ​​the remaining gradient are calculated, and the gain coefficient of each convolution kernel is obtained using formula (8). Then, the gradient of the ROI area is multiplied by the coefficient using formula (9) to obtain the score under a single filter. Finally, the image quality assessment score is finally obtained by weighted summing the scores obtained by different filters, as shown in formula (10).

[0099] Figure 3 shows a schematic diagram of an image used in an embodiment of the present invention to extract different features using various convolution kernels. An ideal image for evaluating image quality using gradients should have low noise and rich edge information, as shown in Figure 3(a). However, images captured in the real world can be noisy, with blurred edges and unstable proportions due to factors such as lighting conditions, application areas, and equipment materials, as shown in Figure 3(b). Gradients reflect the rate of edge change, thus enabling us to determine image clarity. Figure 3(b) also contains a large number of noisy gradients, which appear both at the edge and in a wide range of other regions. The presence of a large amount of noise accumulates these gradients, which are not of concern. When calculating the gradient statistics, these gradients are also included, affecting the final result. Therefore, the image needs to be denoised, as shown in Figure 3(c). Due to the different noise levels and edge shapes and characteristics, the features obtained using the same filter are not rich enough. Therefore, different kernel sizes are used to obtain different gradient maps, as shown in Figures 3(d), 3(e), and 3(f), and the corresponding regions of interest, as shown in Figures 3(g), 3(h), and 3(i). In this example, (f) captures the most accurate edge regions. By filtering out non-edge regions, the gradient score is less susceptible to noise. In practice, a filtered feature will be more effective, significantly increasing its contribution to the score. This combination of multiple feature fusion strategies yields image quality scores with greater generalization and accuracy.

[0100] Figure 4A schematic diagram of a region of interest gradient matrix showing the final quality scores of 10 images obtained in one embodiment of the present invention is shown, and the schematic diagram also shows the distribution of three quantities related to the gain coefficient used in the embodiment of the present invention, namely MAX(GN(x, y)), STD(G(x, y)) and A roi ,exist Figure 4 The data are 10 images sampled during the SEM focusing process. Figure 4 The horizontal axis in represents the number of these 10 images. These images go from unclear to clear and then to unclear. Among them, the score of the 5th image is the highest, which is the clearest image, that is, the closest to the imaging focus. The figure shows the scores of these images, the gradient standard deviation, maximum value and area (number of pixels) of the ROI region. It can be seen from the figure that the clarity of the image is proportional to the gradient standard deviation and maximum value of the ROI region, and inversely proportional to the gradient area of ​​the ROI region. Therefore, a gain coefficient of the image score is designed to multiply the standardized ROI gradient mean, such as formulas (8) and (9). The introduction of the gain coefficient can effectively further improve the anti-interference ability, further separate the score differences between images of different quality, and help improve the accuracy of applications such as autofocus and image quality classification.

[0101] Figure 5 The figure shows an example of another embodiment of the present invention being applied to autofocus. In the autofocus application, by adjusting the focal length, such as adjusting from far to near, the image will go from blurry to clear and then back to blurry. Therefore, it is necessary to give an accurate image quality assessment score for each sampled image. The upper half of the figure shows the score calculated using the method of the embodiment of the present invention for each sampled image, and the lower half of the figure lists the corresponding pictures for each sample, a total of 10 images, and Figure 4 The 10 images collected in the embodiment of the present invention are different. The fifth image (because the first image has a sequence number of 0, the fifth image has a sequence number of 4) can be accurately found to be the clearest.

[0102] Example 4

[0103] An image quality assessment system, see Figure 6 The system includes a first calculation module 601, an extraction module 602, a second calculation module 603 and a third calculation module 604.

[0104] Among them, the first calculation module 601 is used to denoise the image to be evaluated based on multiple convolution kernels, calculate the gradient map of the denoised image to be evaluated, and standardize the gradient map; the extraction module 602 is used to binarize the standardized gradient map and extract the region of interest in the image to be evaluated based on the binarized gradient map; the second calculation module 603 is used to calculate the quality score of the image to be evaluated after denoising by each convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest; the third calculation module 604 is used to calculate the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel.

[0105] It can be understood that the image quality assessment system provided in an embodiment of the present invention corresponds to the image quality assessment method provided in the aforementioned embodiments. The relevant technical features of the image quality assessment system can refer to the relevant technical features of the image quality assessment method, which will not be repeated here.

[0106] Example 5

[0107] See also Figure 7 , Figure 7 Schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 7 As shown, an embodiment of the present invention provides an electronic device 700, including a memory 710, a processor 720, and a computer program 711 stored in the memory 710 and executable on the processor 720. When the processor 720 executes the computer program 711, the following steps are implemented: denoising the image to be evaluated based on multiple convolution kernels, calculating the gradient map of the denoised image to be evaluated, and normalizing the gradient map; binarizing the normalized gradient map, and extracting the region of interest in the image to be evaluated based on the binarized gradient map; calculating the quality score of the image to be evaluated after denoising by each convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest; and calculating the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel.

[0108] As an embodiment, the computer program 711 stored in the memory 710 of the electronic device 700 may implement the method in the first embodiment, the second embodiment or the third embodiment when executed.

[0109] Example 6

[0110] See also Figure 8 , Figure 8 Schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 8As shown, this embodiment provides a computer-readable storage medium 800, on which a computer program 711 is stored. When the computer program 711 is executed by a processor, the following steps are implemented: denoising the image to be evaluated based on multiple convolution kernels, calculating the gradient map of the denoised image to be evaluated, and standardizing the gradient map; binarizing the standardized gradient map, and extracting the region of interest in the image to be evaluated based on the binarized gradient map; calculating the quality score of the image to be evaluated after denoising by each convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest; and calculating the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel.

[0111] As an embodiment, the computer program 711 stored on the computer-readable storage medium 800 may also implement the method in the first embodiment, the second embodiment, or the third embodiment when executed.

[0112] It should be noted that, in the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0113] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0117] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0118] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for image quality assessment, characterized in that: include: Denoising the image to be evaluated based on multiple different convolution kernels, calculating the gradient map of the denoised image to be evaluated, and normalizing the gradient map; Binarizing the normalized gradient map, and extracting a region of interest in the image to be evaluated based on the binarized gradient map; Calculating a quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest; Calculate the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel; Binarizing the normalized gradient map and extracting a region of interest in the image to be evaluated based on the binarized gradient map includes: The gradient map is binarized using a clustering algorithm, the image to be evaluated is divided into two categories based on the binarized gradient map, and a region consisting of pixels of a category with a large gradient value is extracted as a region of interest in the image to be evaluated.

2. The image quality assessment method according to claim 1, wherein: The region of interest includes a plurality of sub-regions of interest, each sub-region of interest includes at least one pixel point, the standardized gradient map is binarized, and the region of interest in the image to be evaluated is extracted based on the binarized gradient map, and then the following steps are further included: Eliminate abnormal sub-regions of interest in the region of interest; The step of calculating the quality score of each image to be evaluated after denoising by the convolution kernel based on the gradient matrix of the region of interest and the area of ​​the region of interest includes: Based on the gradient matrix and area of ​​the region of interest after removal, the quality score of the image to be evaluated after each convolution kernel denoising is calculated.

3. The image quality assessment method according to claim 2, wherein: The step of removing abnormal sub-regions of interest from the region of interest includes: Calculating the gradient average of each sub-region of interest; The sub-regions of interest whose average gradient value is greater than or equal to the preset gradient threshold are eliminated.

4. The image quality assessment method according to claim 1, wherein: The step of calculating the quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest includes: Calculate the gain coefficient corresponding to each convolution kernel based on the maximum gradient in the gradient matrix of the region of interest, the standard deviation of the gradient matrix and the area of ​​the region of interest; Based on the gain coefficient, the gradient matrix of the region of interest, the binary mask of the gradient matrix of the region of interest, and the area of ​​the region of interest, the quality score of the image to be evaluated after each convolution kernel denoising is calculated.

5. The image quality assessment method according to claim 4, wherein: The step of calculating the gain coefficient corresponding to each convolution kernel based on the maximum gradient value in the gradient matrix of the region of interest, the standard deviation of the gradient matrix, and the area of ​​the region of interest includes: F w =MAX(GN(x,y))×[STD(G(x,y))] 2 / A roi ; Among them, F w is the gain coefficient corresponding to the w-th convolution kernel, MAX(GN(x, y)) is the maximum gradient in the gradient matrix GN(x, y) after standardization of the region of interest, STD(G(x, y)) is the standard deviation of the gradient matrix, A roi is the area of ​​the region of interest; The step of calculating the quality score of the image to be evaluated after each convolution kernel denoising based on the gain coefficient, the gradient matrix of the region of interest, the binary mask of the gradient matrix of the region of interest, and the area of ​​the region of interest includes: Among them, M ROI is the binary mask of the gradient matrix, which is obtained by clustering algorithm, ∑|GN(x, y)∧M ROI | is the gradient value in the normalized gradient matrix GN(x, y) and the binary mask M ROI The sum of the absolute values ​​after logical AND, A roi is the area of ​​the region of interest, S w is the quality score of the image to be evaluated after denoising by the w-th convolution kernel.

6. The image quality assessment method according to claim 5, wherein: The method of calculating the final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel includes: Among them, n is the number of convolution kernels, w∈[1,n],k w is the w-th convolution kernel size, S w is the quality score of the image to be evaluated after denoising by the w-th convolution kernel, and Score is the final quality score of the image to be evaluated.

7. An image quality assessment system, characterized in that: include: A first computing module denoises the image to be evaluated based on multiple convolution kernels, calculates a gradient map of the denoised image to be evaluated, and standardizes the gradient map; an extraction module, configured to binarize the normalized gradient map and extract a region of interest in the image to be evaluated based on the binarized gradient map; A second calculation module is used to calculate the quality score of the image to be evaluated after each convolution kernel denoising based on the gradient matrix of the region of interest and the area of ​​the region of interest; A third calculation module is used to calculate a final quality score of the image to be evaluated based on the quality score of the image to be evaluated after denoising by each convolution kernel; The extraction module is used to binarize the normalized gradient map and extract the region of interest in the image to be evaluated based on the binarized gradient map, including: The gradient map is binarized using a clustering algorithm, the image to be evaluated is divided into two categories based on the binarized gradient map, and a region consisting of pixels of a category with a large gradient value is extracted as a region of interest in the image to be evaluated.

8. An electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of the image quality assessment method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the image quality assessment method according to any one of claims 1 to 6 are implemented.

Images