Optical image quality blind evaluation method and system oriented to motion blur and haze influence

Abstract

The invention discloses an optical image quality blind evaluation method and system oriented to motion blur and haze influence, and the method comprises the steps: obtaining a video stream shot by an optical camera of an unmanned plane, extracting image data from the video stream, and storing the image data in an image library; performing Tenenggrad gradient calculation on each image, performing primary scoring on the image quality, and directly measuring the image fuzziness condition; the method comprises the following steps: performing re-blurring processing on an image stored in an image library by using a Gaussian blurring kernel to obtain a re-blurred image, and calculating the image similarity between the re-blurred image and an original image by using an SSIM similarity calculation method so as to obtain secondary scoring of image quality and measure the image blurring condition through the blurring condition similarity; calculating an image quality comprehensive evaluation coefficient by using the primary scoring and the secondary scoring of the image quality, and carrying out comprehensive measurement and evaluation on the image ambiguity; and performing threshold judgment by using the image quality comprehensive evaluation coefficient. According to the method, the influence degrees of haze and motion blur in an image processing system can be effectively distinguished, and the to-be-processed data can be effectively screened.

Description

Technical Field

[0001] The present invention relates to the field of optical image processing and image quality evaluation, and in particular to a blind evaluation method and system for optical image quality oriented to motion blur and haze effects. Background Art

[0002] When using an optical camera mounted on an unmanned vehicle to observe outdoor scene targets, images with degraded quality may be obtained due to various reasons such as the platform, weather, and transmission. The two main forms that have a greater impact on target observation and subsequent applications are: due to the relative motion between the platform and the observed target, the image quality is degraded due to motion blur; or due to the large amount of water vapor and solid particles suspended in the air, the image quality is degraded due to haze. By objectively evaluating the image quality, the degree to which the image is affected by motion blur and haze can be measured, thereby effectively screening the acquired images to ensure that the processing end can receive images with high image quality.

[0003] Optical image quality assessment is a fundamental problem in the field of image processing and computer vision, which attempts to quantify the degree of image quality degradation. At present, image quality assessment methods are mainly divided into objective quality assessment methods and subjective quality assessment methods. Among them, subjective quality assessment methods are not widely used because they require human participation. This paper mainly involves blind image quality assessment of optical images, so it focuses on the current status of optical image quality assessment without reference.

[0004] Patent 1 [Chu Ying, You Weilin. Haze image quality evaluation method, system, storage medium and electronic device: 201910436390[P][2023-08-31].] A haze image quality evaluation method based on a dark channel prior method is proposed.

[0005] The paper [Xie Fengying, Pan Xiaoxi, Jiang Zhiguo, et al. Reference-free evaluation of haze impact on remote sensing image quality (English) [J]. Chinese Stereology and Image Analysis, 2015(1):6. DOI:CNKI:SUN:ZTSX.0.2015-01-004.] proposed a haze distribution impact index based on dark channel prior and combined with Weber's law.

[0006] Patent 2 [A blind evaluation method for perceptual quality of de-motion blurring of a single image] proposes a blind evaluation method for perceptual quality of de-motion blurring of a single image. This method utilizes a CNN model with a two-stage network training method of classification pre-training + quality score prediction to implement a blind evaluation method for the data set for de-blurring perceptual quality evaluation. Summary of the invention

[0007] The purpose of the present invention is to propose a blind evaluation method and system for optical image quality under the influence of motion blur and haze, so as to solve the problem of image quality evaluation when the optical camera of a drone is affected by motion blur and haze when shooting.

[0008] The technical solution to achieve the purpose of the present invention is: a blind evaluation method for optical image quality oriented to motion blur and haze, comprising the following steps:

[0009] Step 1: Get the video stream captured by the optical camera of the drone, and extract image data from the video stream according to the video frame rate and equal interval extraction method as the subsequent processing application images I1, I2, ... I n , save to the gallery;

[0010] Step 2: Perform Tenengrad gradient calculation on each image I(x,y), extract the gradient values ​​in the horizontal and vertical directions respectively, and obtain the average Tenengrad gradient value of the image;

[0011] Step 3: Use the image average Tenengrad gradient value, to score the image quality and directly measure the image blur;

[0012] Step 4: Use the Gaussian blur kernel to re-blur the image saved in the library to obtain a re-blurred image. Use the SSIM similarity calculation method to calculate the image similarity between the re-blurred image and the original image, so as to obtain a secondary score of the image quality. The image blurriness is measured by the blur similarity;

[0013] Step 5: Calculate the comprehensive evaluation coefficient of image quality by using the primary and secondary scores of image quality, and conduct a comprehensive evaluation of image blur;

[0014] Step 6: Use the comprehensive image quality evaluation coefficient to make a threshold judgment and divide the image into "good", "average", and "poor". Discard the data with the "poor" label, and retain and subsequently process the data with the "good" and "average" labels.

[0015] Further, step 2: perform Tenengrad gradient calculation on each image I(x,y), and use the Sobel operator to extract the gradient values ​​in the horizontal and vertical directions respectively to obtain the average Tenengrad gradient value of the image. x represents the Sobel operator of size 3×3 in the horizontal direction, K y Represents a Sobel operator of size 3×3 in the vertical direction:

[0016]

[0017] First use the image I(x,y) and the template operator Kx Convolution of the horizontal gradient value I x (x,y):

[0018] I x (x,y)=I(x,y)*K x

[0019] Using image I(x,y) and gradient operator K y Convolution of to obtain the vertical gradient value I y (x,y):

[0020] I y (x,y)=I(x,y)*K y

[0021] Finally, the average Tenengrad gradient value tenengrad mean , where M is the image row size and N is the image column size:

[0022]

[0023] Further, step 3: use the image average Tenengrad gradient value and A ref The image quality is scored based on the ratio. ref Set to 7.4, the image quality score obtained is I Q1 for:

[0024]

[0025] Among them, A ref Image quality reference value determined for scene images captured by the drone's optical camera.

[0026] Further, step 4: using the Gaussian blur kernel to re-blur the image stored in the library to obtain the re-blurred image I blurred , using the SSIM similarity calculation method, calculate the heavy blurred image I blurred The image similarity with the original image I, thereby obtaining the image quality score I Q2 , the calculation formula is as follows:

[0027] I Q2 =1-ssim(I,I blurred ).

[0028] Further, step 5: using the first and second image quality scores, calculate the image quality comprehensive evaluation coefficient, and conduct a comprehensive evaluation of image blur, where the image quality comprehensive evaluation coefficient I Q for:

[0029] IQ =α·I Q1 +β·I Q2

[0030] Among them, α and β are two adjustment coefficients for measuring image quality scores. In order to ensure I Q The result is a value between 0 and 1, α and β are both between 0 and 1, and α+β=1. Q2 , setting different α and β coefficients.

[0031]

[0032] Further, step 6: using the comprehensive evaluation coefficient of image quality to perform threshold judgment, the image is divided into "good", "normal", and "bad", the data with the "bad" label is discarded, and the data with the "good" and "normal" labels are retained and subsequently processed, wherein the threshold judgment method is:

[0033]

[0034] A blind evaluation system for optical image quality facing motion blur and haze, which implements the blind evaluation method for optical image quality facing motion blur and haze, realizes blind evaluation of optical image quality facing motion blur and haze, and includes:

[0035] Module 1: Obtain the video stream captured by the optical camera of the drone, and extract image data from the video stream according to the video frame rate and equal interval extraction method as subsequent processing application images I1, I2, ... I n , save to the gallery;

[0036] Module 2: Perform Tenengrad gradient calculation on each image I(x,y), extract the gradient values ​​in the horizontal and vertical directions respectively, and obtain the average Tenengrad gradient value of the image;

[0037] Module 3: Use the average Tenengrad gradient value of the image to score the image quality and directly measure the image blur;

[0038] Module 4: Use the Gaussian blur kernel to re-blur the images saved in the library to obtain the re-blurred image. Use the SSIM similarity calculation method to calculate the image similarity between the re-blurred image and the original image, so as to obtain a secondary score of the image quality. The image blurriness is measured by the blur similarity;

[0039] Module 5: Using the primary and secondary image quality scores, calculate the comprehensive image quality evaluation coefficient and conduct a comprehensive evaluation of image blur;

[0040] Module 6: Use the comprehensive image quality evaluation coefficient to make threshold judgments and classify images into "good", "average", and "poor". Discard the data with the "poor" label, and retain and subsequently process the data with the "good" and "average" labels.

[0041] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for blind evaluation of optical image quality facing motion blur and haze effects is implemented to achieve blind evaluation of optical image quality facing motion blur and haze effects.

[0042] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the blind evaluation method for optical image quality facing motion blur and haze effects, thereby realizing blind evaluation of optical image quality facing motion blur and haze effects.

[0043] Compared with the prior art, the present invention has the following significant advantages: simple calculation, no need for model training, application deployment is not restricted by hardware platform, and it can effectively identify the degree of influence of haze and motion blur in image processing systems and effectively screen the data to be processed. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the image quality evaluation method of the present invention.

[0045] Figure 2 It is a processing flow chart of the image quality evaluation system of the present invention.

[0046] Figure 3 This is a comparison chart of the image quality score of the present invention and the MOS score on the fastfading loss image.

[0047] Figure 4 This is a comparison chart of the image quality score of the present invention and the MOS score on the gblur loss image. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The present invention proposes a blind evaluation method for optical image quality affected by motion blur and haze, which uses the results of gradient feature statistics to evaluate image quality, and combines the image re-blur to perform secondary scoring, and combines the two evaluation results to provide a method for evaluating the quality of images affected by motion blur and haze, and uses the results to screen images to select images with better quality for subsequent processing. The specific steps are as follows:

[0050] Step 1: Get the video stream captured by the drone, extract image data in equal intervals according to the video frame rate, and use the image data extracted from the video stream as the subsequent processing application images I1, I2, ... I n .Save to the gallery. (If the object you observe moves very fast, you can keep a few more frames).

[0051] Step 2: Perform Tenengrad gradient calculation on each image I(x,y) to obtain the average Tenengrad gradient value of the image. The gradient function uses the Sobel operator to extract the gradient values ​​in the horizontal and vertical directions respectively. The template operator is:

[0052]

[0053] First use the image and gradient operator K x Convolution of the horizontal gradient value I x (x,y):

[0054] I x (x,y)=I(x,y)*K x

[0055] Reusing the image and gradient operator K y Convolution of to obtain the vertical gradient value I y (x,y):

[0056] I y (x,y)=I(x,y)*K y

[0057] Finally, the gradient at each pixel position is calculated by averaging to obtain the average Tenengrad gradient value, where M is the total number of rows in the image and N is the total number of columns in the image:

[0058]

[0059] Step 3: Use the image average Tenengrad gradient tenengrad_mean value to take the logarithm with base 2 and the reference value A of the natural scene taken by the drone for this application ref =7.4 to calculate the ratio and get the first score of image quality I Q1 .

[0060]

[0061] Step 4: Use the Gaussian blur kernel with a window size of 11×11 to re-blur the image I(x,y) to be evaluated in the image library in step 1 to obtain the re-blurred image I blurred , using the SSIM similarity calculation method, calculate the heavy blurred image I blurred The image similarity with the original image I, thereby obtaining the second score I of the image quality Q2 .

[0062] I Q2 =1-ssim(I,I blurred )

[0063] Step 5: Calculate the comprehensive image quality evaluation coefficient I Q , a comprehensive evaluation is performed by directly measuring the image blurriness and the similarity measurement results.

[0064] I Q =α·I Q1 +β·I Q2

[0065] Among them, α and β are two adjustment coefficients for measuring image quality scores. In order to ensure I Q The result is a value between 0 and 1, α and β are both between 0 and 1, and α+β=1. Q2 , setting different α and β coefficients.

[0066]

[0067] Through the above evaluation steps, the comprehensive image quality evaluation coefficient of the image obtained in step 1 can be calculated to give a quality evaluation score that is more in line with the subjective perception of the human eye, thereby guiding subsequent applications.

[0068] Step 6: Use the comprehensive image quality evaluation coefficient to make a threshold judgment, so as to classify the image into "good", "normal" and "bad". When the system processing capacity is insufficient, the data with the "bad" label can be discarded, and the data with the "good" and "normal" labels can be retained and subsequently processed, so as to reduce the waste of computing power and storage resources.

[0069]

[0070] The present invention also proposes a blind evaluation system for optical image quality facing motion blur and haze effects, implements the blind evaluation method for optical image quality facing motion blur and haze effects, and realizes blind evaluation of optical image quality facing motion blur and haze effects.

[0071] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for blind evaluation of optical image quality facing motion blur and haze effects is implemented to achieve blind evaluation of optical image quality facing motion blur and haze effects.

[0072] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the blind evaluation method for optical image quality facing motion blur and haze effects, thereby realizing blind evaluation of optical image quality facing motion blur and haze effects.

[0073] MOS and DMOS are manual value annotation methods for image and video quality evaluation. Both are subjective evaluation methods, which are obtained by collecting scores of multiple observers on the quality of images or videos and calculating the average of these scores. DMOS (Differential Mean Opinion Score) measures the quality difference between distorted images and undistorted images. The smaller the DMOS value, the higher the quality. MOS (Mean Opinion Score) is used to evaluate the quality of images or videos. The larger the MOS value, the higher the quality. Among them, MOS = 100-DMOS.

[0074] In order to illustrate the effectiveness of the method of the present invention, images with gblur and fastfading losses were selected from the LIVE-Release2 data with DMOS manually annotated values ​​for computational experiments, and the values ​​of the results of the present method were compared with the manually annotated MOS normalized values.

[0075] MOS normalized =(100-DMOS) / 100

[0076] Experimental results show that the image is evaluated and scored using the present invention, and the score evaluation results are consistent with the subjective judgment of the human eye, and show an upward trend as the image quality increases.

[0077] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0078] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A blind evaluation method for optical image quality under the influence of motion blur and haze, characterized in that: The steps include: Step 1: Get the video stream captured by the optical camera of the drone, and extract image data from the video stream according to the video frame rate and equal interval extraction method as the subsequent processing application images I1, I2, ...I n , save to the gallery; Step 2: Perform Tenengrad gradient calculation on each image I(x, y), extract the gradient values ​​in the horizontal and vertical directions respectively, and obtain the average Tenengrad gradient value of the image; Step 3: Use the average Tenengrad gradient value of the image to score the image quality and directly measure the image blur; Step 4: Use the Gaussian blur kernel to re-blur the image saved in the library to obtain a re-blurred image. Use the SSIM similarity calculation method to calculate the image similarity between the re-blurred image and the original image, so as to obtain a secondary score of the image quality. The image blurriness is measured by the blur similarity; Step 5: Calculate the comprehensive evaluation coefficient of image quality by using the primary and secondary scores of image quality, and conduct a comprehensive evaluation of image blur; Step 6: Use the comprehensive image quality evaluation coefficient to make a threshold judgment and divide the image into "good", "average", and "poor". Discard the data with the "poor" label, and retain and subsequently process the data with the "good" and "average" labels.

2. The blind evaluation method for optical image quality under motion blur and haze effects according to claim 1, characterized in that: Step 2: Perform Tenengrad gradient calculation on each image I(x, y). The gradient function uses the Sobel operator to extract the gradient values ​​in the horizontal and vertical directions respectively to obtain the average Tenengrad gradient value of the image, where K x represents the Sobel operator of size 3×3 in the horizontal direction, K y It represents the Sobel operator of 3×3 size in the vertical direction. Its template operator is: First, use the image I(x, y) and the template operator K x Convolution of the horizontal gradient value I x (x, y): I x (x,y)=I(x,y)*K x Using the image I(x, y) and the gradient operator K y Convolution of to obtain the vertical gradient value I y (x, y): I y (x,y)=I(x,y)*K y Finally, according to the row and column pixel values, the average Tenengrad gradient value tenengrad mean , where M is the image row size and N is the image column size:

3. The blind evaluation method for optical image quality under motion blur and haze according to claim 2 is characterized in that: Step 3: Use the average Tenengrad gradient value of the image to score the image quality. The image quality score I Q1 for: Among them, A ref Image quality reference value determined for scene images captured by the drone's optical camera.

4. The blind evaluation method for optical image quality under motion blur and haze effects according to claim 3 is characterized in that: Step 4: Use the Gaussian blur kernel to re-blur the image saved in the library to obtain the re-blurred image I blurred , using the SSIM similarity calculation method, calculate the heavy blurred image I blurred The image similarity with the original image I, thereby obtaining the image quality score I Q2 , expressed as: I Q2 =1-ssim(I,I blurred )。 5. The blind evaluation method for optical image quality under motion blur and haze effects according to claim 4, characterized in that: Step 5: Calculate the image quality comprehensive evaluation coefficient using the primary and secondary image quality scores to conduct a comprehensive evaluation of image blur, where the image quality comprehensive evaluation coefficient I Q for: I Q =α·I Q1 +β·I Q2 Among them, α and β are two adjustment coefficients for measuring image quality scores. In order to ensure I Q The result is a value between 0 and 1, α and β are both between 0 and 1, and α+β=1.

6. The blind evaluation method for optical image quality under motion blur and haze effects according to claim 5, characterized in that: Step 6: Use the comprehensive image quality evaluation coefficient to make a threshold judgment, and divide the image into "good", "normal", and "poor". Discard the data with the "poor" label, and retain and subsequently process the data with the "good" and "normal" labels. The threshold judgment method is:

7. A blind evaluation system for optical image quality under the influence of motion blur and haze, characterized in that: Implementing the blind evaluation method for optical image quality facing motion blur and haze effects as described in any one of claims 1 to 6 to achieve blind evaluation of optical image quality facing motion blur and haze effects, comprising: Module 1: Obtain the video stream captured by the optical camera of the drone, and extract image data from the video stream according to the video frame rate and equal interval extraction method as subsequent processing application images I1, I2, ... I n , save to the gallery; Module 2: Perform Tenengrad gradient calculation on each image I(x,y), extract the gradient values ​​in the horizontal and vertical directions respectively, and obtain the average Tenengrad gradient value of the image; Module 3: Use the average Tenengrad gradient value of the image to score the image quality and directly measure the image blur; Module 4: Use the Gaussian blur kernel to re-blur the images saved in the library to obtain the re-blurred image. Use the SSIM similarity calculation method to calculate the image similarity between the re-blurred image and the original image, so as to obtain a secondary score of the image quality. The image blurriness is measured by the blur similarity; Module 5: Using the primary and secondary image quality scores, calculate the comprehensive image quality evaluation coefficient and conduct a comprehensive evaluation of image blur; Module 6: Use the comprehensive image quality evaluation coefficient to make threshold judgments and classify images into "good", "average", and "poor". Discard the data with the "poor" label, and retain and subsequently process the data with the "good" and "average" labels.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for blind evaluation of optical image quality facing motion blur and haze effects as described in any one of claims 1 to 6 is implemented to achieve blind evaluation of optical image quality facing motion blur and haze effects.

9. A computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for blind evaluation of optical image quality facing motion blur and haze effects as described in any one of claims 1 to 6 is implemented to achieve blind evaluation of optical image quality facing motion blur and haze effects.

Images