Monitoring image quality evaluation method based on multi-scale structure fidelity

Abstract

The invention discloses a monitoring image quality evaluation method based on multi-scale structure fidelity, and belongs to the field of image quality evaluation. The invention aims to solve the technical problem of providing the monitoring image quality evaluation method based on the multi-scale structure fidelity, and the method measures the distortion degree of an image contour and an edge by calculating and fusing the gradient amplitude fidelity of multiple scales so as to deduce an image quality score. Structural information is extracted from an original image and a distorted image by using a gradient operator with efficient calculation, and the image processing mode of a human vision system is simulated, so that an evaluation result is more in line with the perception habit of human beings. Mutual information is adopted to measure the amount of information extracted when a human visual system senses an external input signal, and a new view angle is provided for image quality evaluation, that is, the image quality is evaluated from the view of information transmission. And the quality score of the image is predicted by fusing the multi-scale gradient amplitude fidelity.

Description

Technical Field

[0001] The present invention belongs to the field of image quality evaluation, and particularly designs a method for evaluating the quality of surveillance images based on multi-scale structure fidelity. Background Art

[0002] Image quality evaluation is a key technology for achieving "precision" and "intelligence" evaluation in the fields of image processing and computer vision. At the front end of the image processing system, i.e., the image information acquisition stage, the image quality directly determines the effects of subsequent analysis, recognition, and applications. Especially in practical applications such as image monitoring, transmission, and processing, the accurate evaluation of image quality is directly related to the quality of the user experience and the performance of the system. Therefore, exploring efficient and accurate image quality evaluation techniques is of great significance for improving the performance of image processing systems and ensuring the high-quality transmission and utilization of image information in all links.

[0003] Traditional image quality evaluation methods have been difficult to meet the increasing demands for accuracy and diversity in modern image processing and computer vision applications. These methods often focus on using single image features (such as gradients and local features) while ignoring the modeling of the human visual perception system. Although these methods can reflect the fidelity of images to a certain extent, when dealing with complex industrial scenarios covering industrial products and equipment, they often cannot comprehensively and efficiently capture multi-feature information in images, which may lead to a decline in the effectiveness of the entire image processing system. These technical deficiencies have limited the depth and breadth of the application of image quality evaluation methods in the fields of image processing and computer vision to a certain extent.

[0004] The method for evaluating the quality of surveillance images based on multi-scale structure fidelity not only fills the application gap of traditional evaluation methods in the field of the human perception system, but also provides a solid theoretical basis and technical support for the application of high-precision and high-requirement image processing systems, which is of great significance for promoting the reform of traditional image quality evaluation methods. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for evaluating the quality of surveillance images based on multi-scale structure fidelity. This method measures the distortion of image contours and edges by calculating and fusing the gradient magnitude fidelity at multiple scales, thereby inferring the image quality score. The model framework is as Figure 1 shown, and its construction includes the following steps:

[0006] The first step: Calculate the gradient magnitude of the image;

[0007] The second step: Calculate the gradient magnitude fidelity of the image;

[0008] The third step: Introduce a multi-scale strategy to evaluate the image quality.

[0009] The innovations and contributions of the present invention are mainly reflected in:

[0010] 1) The present invention constructs the largest current image quality assessment database, which contains 5000 images with rich distortion processing and their subjective quality scores processed by an expert-guided outlier removal technique. The establishment of this database aims to provide a benchmark for evaluating the quality of images under different distortion conditions.

[0011] 2) Inspired by visual neuroscience, this work uses computationally efficient gradient operators to extract structural information from the original image and the distorted image. These operators have been successfully used in the field of image quality assessment. This method simulates the way the human visual system (HVS) processes images, making the evaluation results more in line with human perception habits.

[0012] 3) Using the information theory framework, the fidelity between the gradient magnitudes of the original image and the distorted image is calculated to quantify the quality loss of the image during the distortion process. By using mutual information to measure the amount of information extracted when the HVS perceives an external input signal, this method provides a new perspective for image quality evaluation, that is, evaluating image quality from the perspective of information transmission.

[0013] 4) Introducing a multi-scale strategy to evaluate the quality of images, this study measures the distortion degree of the image contour and edges by calculating the multi-scale gradient magnitude fidelity, and fuses the multi-scale gradient magnitude fidelity to predict the quality score of the image. This method can not only quantify the fidelity of the image contour and edges, but also fuse the information at different scales to obtain a more comprehensive evaluation of the image quality. This cross-scale information fusion can capture the features of the image at different levels, thus providing a more detailed quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a framework diagram of the monitoring image quality evaluation method based on multi-scale structure fidelity designed by the present invention;

[0015] Figure 2 is a flow chart of the gradient magnitude fidelity model designed by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] The embodiments of the present invention are described in detail below. These embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.

[0017] 1) Calculate the gradient magnitude of the image. The present invention respectively applies gradient operators to extract structural information from the input original image A and the image B that takes A as a reference and has been processed by different degrees and types of distortion.

[0018] Specifically, extensive neurophysiological observations of the HVS and limited electrophysiological recording studies have shown that the HVS exhibits the maximum response to horizontal and vertical stimuli in terms of behavioral measurements of acuity and receptive field distribution. This phenomenon may be due to the high adaptation of the HVS to the environment, making it more sensitive to horizontal and vertical stimuli. For example, humans respond faster to horizontal and vertical stimuli (such as straight lines or rectangles) compared to stimuli in oblique directions. Based on the above visual characteristics, this study uses computationally efficient gradient operators to extract structural information from original and distorted images. These operators have been successfully applied in the field of image quality assessment and are particularly good at perceiving local distortions such as noise, blur, and compression.

[0019] The present invention introduces the commonly used Prewitt operator to calculate the gradient magnitude C of the original image and the gradient magnitude D of its corresponding distorted image.

[0020] 2) Calculate the gradient magnitude fidelity of the image. The present invention uses an information - theoretic framework to calculate the fidelity between the gradient magnitudes of the original and distorted images, thereby quantifying the loss of image quality during the distortion process. Previous studies have shown that within the information - theoretic framework, consistent image quality measurement can be achieved by measuring information fidelity. Figure 2 The proposed gradient magnitude fidelity model is shown.

[0021] Specifically, the present invention uses mutual information to measure the information extracted by the HVS when perceiving external input signals. By calculating the changes in the gradient magnitudes of the original image processed by the HVS through non - distorted channels and the changes in the gradient magnitudes of the distorted image processed by the HVS through distorted channels, the information of the original and distorted images is characterized. This framework based on mutual - information measurement is good at perceiving global distortions such as contrast changes. By combining these two types of information, the present invention calculates the gradient magnitude fidelity to evaluate the loss of image quality.

[0022] First, the gradient magnitudes C and D after being processed by the Prewitt filter simulate how humans perceive and interpret images through the HVS. The gradient magnitude fidelity model Figure 2 As shown, the E and F output by the HVS can be represented in a mathematically feasible and reasonable way, and the formula is as follows:

[0023]

[0024] Where N represents static additive white noise, which is used to simulate the uncertain interference generated during the process of C and D forming E and F through the HVS, based on the "data - processing inequality" theorem in information theory.

[0025] Subsequently, by quantifying the HVS, information can be ideally extracted from the gradient magnitude of the original image to obtain I(A; E); by quantifying the HVS, information can be ideally extracted from the gradient magnitude of the distorted image to obtain I(A; F).

[0026] Finally, by combining these two types of information, the gradient magnitude fidelity is calculated to evaluate the loss of image quality. Through the above calculations, we can obtain the result of the gradient magnitude fidelity Z, and the formula is as follows:

[0027]

[0028] As can be seen from the formula, a Z value close to 1 indicates a high-quality distorted image, similar to the original image; while a Z value close to 0 indicates a low-quality distorted image, different from the original image.

[0029] 3) Introduce a multi-scale strategy to evaluate image quality. We introduce a multi-scale strategy to evaluate the quality of images. In this study, the multi-scale gradient magnitude fidelity is calculated to measure the distortion degree of image contours and edges, and the multi-scale gradient magnitude fidelity is fused to predict the quality score of the image. We can infer the final quality score Q from the original image A and its corresponding distorted image B:

[0030]

[0031] where S = 5 is used to determine the number of Gaussian pyramids; w i represents the psychophysical weight at the i-th scale; P(X, i) represents the operation of extracting the image from the i-th scale of the Gaussian pyramid constructed according to X, and X is A and B. Obviously, the larger the Q value, the better the quality of the distorted image.

[0032] The present invention not only fills the application gap of traditional evaluation methods in the field of the human perception system, but also provides a solid theoretical basis and technical support for the application of high-precision and strict-requirement image processing systems, which is of great significance for promoting the reform of traditional image quality evaluation methods.

Claims

1. Calculate the gradient magnitude of the image; The present invention respectively applies gradient operators to extract structural information from the input original image A and the image B which takes A as a reference and has been processed with different degrees and types of distortion, as follows: Extensive neurophysiological observations and limited electrophysiological recording studies of the human visual system (HVS) have shown that the HVS exhibits the maximum response to horizontal and vertical stimuli in terms of behavioral measurements of acuity and receptive field distribution. This phenomenon is due to the high adaptation of the HVS to the environment, making it more sensitive to horizontal and vertical stimuli. Based on this, the present invention introduces the commonly used Prewitt operator to calculate the gradient magnitude C of the original image and the gradient magnitude D of its corresponding distorted image, as shown by the following formula: where '★' represents the convolution operation, Θ h , Θ v represent the horizontal and vertical directions of the Prewitt filter respectively; A and B represent the original image and its corresponding distorted image respectively.

2. Calculate the fidelity of the image gradient magnitude; The present invention uses an information - theoretic framework to calculate the fidelity between the gradient magnitudes of the original and distorted images, thereby quantifying the quality loss of the image during the distortion process. Specifically, first, the gradient magnitudes C and D after being processed by the Prewitt filter simulate how humans perceive and interpret images through the HVS; E and F output by the HVS can be represented in a mathematically feasible and reasonable way, and the formula is as follows: Among them, N represents static additive white noise, which is used to simulate the uncertain interference generated during the process of C and D forming E and F through the HVS, based on the "data - processing inequality" theorem in information theory; Subsequently, by quantifying that the HVS can ideally extract information from the original image gradient magnitude, Ι(A; E) is obtained; by quantifying that the HVS can ideally extract information from the distorted image gradient magnitude, Ι(A; F) is obtained; Finally, by combining these two types of information, the result of calculating the gradient magnitude fidelity Z is obtained, and the formula is as follows:

3. Introduce a multi - scale strategy to evaluate image quality; This study measures the distortion degree of image contours and edges by calculating the multi - scale gradient magnitude fidelity and fuses the multi - scale gradient magnitude fidelity to predict the quality score of the image; Specifically, the final quality score Q is inferred from the original image A and its corresponding distorted image B: where S = 5 is used to determine the number of Gaussian pyramids; w i represents the psychophysical weight at the i-th scale; P(X, i) represents the operation of extracting an image from the i-th scale of the Gaussian pyramid constructed according to X, where X is A and B. Obviously, the larger the Q value, the better the quality of the distorted image.

Images