Establishment method of aerial image quality comprehensive evaluation system

Abstract

The invention discloses a method for establishing an aerial image quality comprehensive evaluation system, which is characterized in that after image quality influence factors are fully analyzed, the image quality influence factors are divided into three aspects of information loss, structural similarity and perception gain. The four abstract concepts are defined as latent variables, and the candidate indexes are defined as observation variables. The relation between observation variables and latent variables is analyzed by utilizing a structural equation model, and an image quality evaluation system is finally established by optimizing the system, so that comprehensive and accurate evaluation on the optimized aerial image is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for establishing a comprehensive evaluation system for aerial image quality. Background Art

[0002] Aerial images have attracted much attention from researchers due to their wide applications in remote sensing, agriculture, urban planning and other fields. However, due to the interference of natural factors such as wind, rain, and haze during aerial photography, as well as the jitter of the acquisition equipment itself and the imperfection of the optical system, aerial images often have distortion such as noise and blur. In response to the distortion in the image, scholars have proposed many image optimization algorithms to restore the distorted image to the original image. The performance difference of the optimization algorithm makes the quality of the optimized image different. For the receiver of the image, the optimized image has less information loss, higher structural similarity, and higher perceptual gain than the reference image, and its quality is correspondingly higher. However, the existing image quality evaluation methods have certain limitations in evaluating the optimized image, making it difficult to comprehensively and accurately evaluate the quality of the image, and the evaluation results are quite different from the actual perception results of humans. This will bring certain difficulties to the subsequent image processing and analysis. Therefore, how to improve the comprehensiveness and accuracy of aerial image evaluation has always been a difficult and hot topic in research in this field.

[0003] Structural equation modeling is a method for establishing, estimating, and testing causal relationship models. It is an empirical analysis model method that verifies whether the assumptions of a certain structural relationship or model are reasonable and whether the model is correct by finding the inherent structural relationship between variables. Different from the traditional single indicator evaluation method, the structural equation model clearly analyzes the effect of a single indicator on the overall situation and the relationship between single indicators, and considers the measurement error of the measurement indicator. Therefore, the use of the structural equation model can more fully explore the inherent connection between the evaluation indicators and the latent variables, and comprehensively and accurately evaluate the latent variables.

[0004] The characteristics of the structural equation model provide a solution for evaluating the quality of optimized images. Considering that image quality, information loss, structural similarity, and perceptual gain are all abstract concepts that are difficult to accurately define and measure, these concepts can be defined as latent variables, and the selected indicators can be defined as observed variables. The system can be built and optimized through the structural equation model, and finally the image quality can be comprehensively evaluated in multiple dimensions using a combination of low-redundancy indicators. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a method for establishing a comprehensive evaluation system for aerial image quality. After fully analyzing the factors affecting image quality, the system is divided into three aspects: information loss, structural similarity, and perceptual gain. These four abstract concepts are defined as latent variables, and the selected indicators are defined as observed variables. The structural equation model is used to analyze the relationship between observed variables and latent variables. By optimizing the system, an image quality evaluation system is finally established to achieve a comprehensive and accurate evaluation of the optimized aerial images.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] Step 1: Generation of optimized aerial images;

[0008] Select the aerial image as the reference image R, and generate the distorted image D according to the distorted scene of the image;

[0009] For the image under the interference of noise n, the distorted image is modeled as D = R + n;

[0010] The image denoising algorithm is used to process the distorted image D to obtain the optimized image.

[0011] Step 2: Initialize the indicator system model;

[0012] The concept of image quality is analyzed, and its influencing factors are divided into three aspects: structural similarity, information loss and perceptual gain;

[0013] Define structural similarity, information loss, perceptual gain and image quality as latent variables ξ1, ξ2, ξ3, ξ4 respectively;

[0014] Select from the existing indicators and classify them according to their design characteristics and evaluation objects. Reference indicators are classified into ξ1, ξ2, ξ3, and non-reference indicators are classified into ξ4. The quality score of the selected indicators is recorded as x ji , the weight corresponding to the indicator is w ji , i represents the category to which it belongs, j represents the jth candidate indicator of the category; the number of indicators selected by these four latent variables are p1, p2, p3, and p4 respectively, and the initialized indicator system is obtained;

[0015] Step 3: Calculation of image quality after optimization;

[0016] Based on the processing results of step 1 and step 2, the optimized image is evaluated using the index system of step 2. Calculate the mass and get x ji ;

[0017] Step 4: Index system fitting;

[0018] The partial least squares structural equation model is used to fit the indicator system, which is expressed as:

[0019]

[0020] Among them, β i and ζ represent the path coefficient and the fitting error of the structural model, respectively;

[0021] The measurement model is expressed as:

[0022]

[0023] Among them, ε i represents the latent variable ξ i The model first initializes the weight to 1 and the error to 0, that is, all w ji =1, ε1=ε1=ε1=ζ=0;

[0024] Step 5: Optimize the indicator system;

[0025] Eliminate low-precision evaluation indicators based on weight coefficients: For each latent variable, remove all reference indicators with weights lower than 0.8 and those without reference indicators with weights lower than 0.5 in the fitting model; at the same time, eliminate indicators based on VIF values. VIF indicates the redundancy of indicators. For indicator x ji , its VIF value is calculated by the following formula:

[0026]

[0027] Where q represents the qth sample in the total number of images N, μ ji represents the average quality score calculated from the indicators of N samples, Represents ξ i Divide by x ji The value of the indicator calculated by regression for the qth sample;

[0028] Calculate the VIF value for the remaining indicators after the initial elimination. If there are indicators with VIF>10 corresponding to the latent variable, define these indicators as redundant indicators; if after deleting one of the redundant indicators, the number of indicators that meet the VIF>10 is the smallest among the remaining indicators, delete this indicator; if there are several redundant indicators, and their deletion makes the number of indicators that meet the condition the same, then sum the VIF values ​​of these indicators and delete the indicator with the smallest VIF value sum; then return to step 4 until the number of indicators corresponding to the latent variable is less than or equal to 3, or the VIF values ​​are all less than 10;

[0029] Step 6: Reliability and validity assessment of the indicator system;

[0030] Cronbach's alpha coefficient was used to evaluate the reliability of the index system, and the average variance extracted was used to evaluate the convergent validity of the index system to verify the accuracy and reliability of the index system prediction.

[0031] Preferably, the image denoising algorithm is the FFDNet algorithm.

[0032] Furthermore, the specific steps of the index system fitting in step 4 are:

[0033] ① Calculate the measurement model through formula (2) and get ξ i The value of

[0034] ②Perform partial least squares regression and calculate the path coefficient β i ;

[0035] ③ζ is set to 0, and the calculated β i and i The new

[0036] ④Use and x ji Calculate the correlation through bivariate regression and update w ji ;

[0037] ⑤ General Assign to ξ i ;

[0038] Repeat the above five steps. When the updated value of the weight is less than 10 -7 , indicating that the model has converged, and the index system fitting is completed.

[0039] The beneficial effects of the present invention are as follows:

[0040] The present invention conducts a comprehensive and systematic evaluation of the optimized aerial images. Through in-depth analysis of the optimized images, the images are evaluated from three aspects: information loss, structural similarity and perceptual gain, and a basic system is obtained. Many indicators are incorporated into the construction of the system and simplified by using structural equation modeling. Indicators with low accuracy or poor correlation can be filtered out to form a refined and efficient indicator system. Experimental results verify the reliability and effectiveness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 Initialize the indicator system diagram for the present invention;

[0043] Figure 3 This is a schematic diagram of the index system results of the method of the present invention;

[0044] Figure 4 Scatter plots of subjective scores and predicted scores of denoised images in two databases according to an embodiment of the present invention, in each figure, the upper figure: KADID, the lower figure: PIPAL; (a) NLPD, (b) LPIPS, (c) VIF, (d) ISC. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0046] The present invention aims to provide a method for establishing an image quality evaluation system based on a structural equation model, so as to eliminate the limitations of a single-index evaluation model and the redundancy of a multi-index evaluation model, and to achieve a comprehensive and accurate evaluation of the quality of optimized aerial images.

[0047] The present invention mainly solves the problems of high indicator redundancy, low evaluation accuracy and difficulty in practical application. The purpose of the present invention is to improve the following aspects:

[0048] 1. The multi-indicator evaluation model is redundant, resulting in too many evaluation indicators and difficulty in practical application.

[0049] 2. The single-index evaluation model has limitations, resulting in incomplete and inaccurate image quality evaluation.

[0050] 3. The intrinsic relationship between image quality and information loss, structural similarity and perceptual gain is unclear, which is not conducive to the design of optimization algorithms and the optimization of evaluation indicators.

[0051] 4. Existing evaluation methods rely too much on human subjective ratings, resulting in poor generalization ability of the model.

[0052] The present invention proposes a method for establishing a comprehensive evaluation system for aerial image quality. After fully analyzing the factors affecting image quality, the system is divided into three aspects: information loss, structural similarity, and perceptual gain. These four abstract concepts are defined as latent variables, and the selected indicators are defined as observed variables. The structural equation model is used to analyze the relationship between observed variables and latent variables. By optimizing the system, an image quality evaluation system is finally established to achieve a comprehensive and accurate evaluation of the optimized aerial images. The technical solution is as follows:

[0053] Step 1: Generate optimized aerial images. Select a high-quality aerial image as the reference image R, and generate a distorted image D according to the distorted scene of the image. The present invention is aimed at images under noise interference, so the distorted image is

[0054] The model is D = R + n. The optimization algorithm is used to process the distorted image D to obtain the optimized image

[0055] Step 2: Initialize the index system model. The concept of image quality is analyzed, and its influencing factors are divided into three aspects: structural similarity, information loss, and perceptual gain. Structural similarity, information loss, perceptual gain, and image quality are defined as latent variables ξ1, ξ2, ξ3, ξ4 respectively. Select from the existing indicators and classify them according to the design characteristics and evaluation objects of the indicators. The reference indicators are classified into ξ1, ξ2, ξ3, and the non-reference indicators are classified into ξ4. The quality score of the selected indicators is recorded as x ji (Calculated in step 3), the weight corresponding to the indicator is w ji (calculated in step 4), i represents the category, j represents the jth candidate indicator of the category. The number of indicators selected by these four latent variables are p1, p2, p3, and p4 respectively. In this way, the initialized indicator system is obtained. Figure 2 shown.

[0056] Step 3: Calculate the quality of the optimized image. Based on the processing results of steps 1 and 2, use the indicators of step 2 to calculate the quality of the optimized image. Calculate the mass and get x ji .

[0057] Step 4: Fitting the index system. Here, the partial least squares structural equation model is used to fit the index system. The core of this method is partial least squares regression. Figure 2 The structural model shown is expressed as:

[0058]

[0059] Among them, β i and ζ represent the path coefficient and the fitting error of the structural model respectively. The measurement model is expressed as:

[0060]

[0061] Among them, ε i represents the latent variable ξ i The model first initializes the weight to 1 and the error to 0, that is, all w ji =1, ε1=ε1=ε1=ζ=0. ① Calculate the measurement model through formula (2) and calculate ξ i ② Perform partial least squares regression to calculate the path coefficient β i ③ζ is set to 0, and the calculated β i and i The new ④Use and x ji Calculate the correlation through bivariate regression and update w ji⑤ Assign to ξ i Repeat the above five steps. When the updated value of the weight is less than 10 -7 , indicating that the model has converged, and the index system fitting is completed.

[0062] Step 5: Optimize the indicator system. Eliminate the evaluation indicators with low precision according to the weight coefficient. For each latent variable, delete all the reference indicators with weights less than 0.8 and those without reference indicators with weights less than 0.5 in the fitting model. At the same time, eliminate the indicators according to the VIF value. VIF indicates the redundancy of the indicator. For the indicator x ji For , its VIF value can be calculated by the following formula:

[0063]

[0064] Where q represents the qth sample in the total number of images N. μ ji represents the average quality score calculated from the indicators of N samples, Represents ξ i Divide by x ji The value of the indicator calculated by regression for the qth sample. Calculate the VIF value for the remaining indicators after preliminary elimination. If there are indicators with VIF>10 corresponding to the latent variable, these indicators are defined as redundant indicators. If after deleting one of the redundant indicators, the number of indicators that satisfy VIF>10 is the smallest among the remaining indicators, then delete the indicator. If there are several redundant indicators, and their deletion makes the number of indicators that meet the conditions the same, then sum the VIF values ​​of these indicators and delete the indicator that minimizes the sum of VIF values. Then return to step 4. Until the number of indicators corresponding to the latent variable is less than or equal to 3, or the VIF values ​​are all less than 10.

[0065] Step 6: Reliability and validity evaluation of the indicator system. The reliability of the indicator system was evaluated using Cronbach's alpha coefficient, and the convergent validity of the indicator system was evaluated using the average variance extraction to verify the accuracy and reliability of the indicator system prediction.

[0066] In general, the present invention is a method for establishing an optimized aerial image quality evaluation system. It views the quality evaluation problem from a systematic perspective and realizes the transformation from a directional evaluation of a single indicator to a comprehensive evaluation of an indicator system. It effectively solves the problems of the one-sidedness and limitations of the evaluation process of a single performance indicator in the prior art, and truly, comprehensively and accurately reflects the quality of the optimized aerial images.

[0067] Example:

[0068] Reference Figure 1 , Figure 2 and Figure 3, the implementation steps of the present invention are as follows:

[0069] Step 1: Generate optimized aerial images. Select a high-quality aerial image as the reference image R, and generate a distorted image D based on the distorted scene of the image. The present invention targets images under noise n interference, so the distorted image is modeled as D = R + n. Use the optimization algorithm to process the distorted image D to obtain the optimized image

[0070] Step 2: Initialize the index system model. The concept of image quality is analyzed, and its influencing factors are divided into three aspects: structural similarity, information loss, and perceptual gain. Structural similarity, information loss, perceptual gain, and image quality are defined as latent variables ξ1, ξ2, ξ3, ξ4 respectively. Select from the existing indicators and classify them according to the design characteristics and evaluation objects of the indicators. The reference indicators are classified into ξ1, ξ2, ξ3, and the non-reference indicators are classified into ξ4. The quality score of the selected indicators is recorded as x ji (Calculated in step 3), the weight corresponding to the indicator is w ji (calculated in step 4), i represents the category, j represents the jth candidate indicator of the category. The number of indicators selected by these four latent variables are p1, p2, p3, and p4 respectively. In this way, the initialized indicator system is obtained. Figure 2 shown.

[0071] Step 3: Calculate the quality of the optimized image. Based on the processing results of steps 1 and 2, use the indicators of step 2 to calculate the quality of the optimized image. Calculate the mass and get x ji .

[0072] Step 4: Fitting the index system. Here, the partial least squares structural equation model is used to fit the index system. The core of this method is partial least squares regression. Figure 2 The structural model shown is expressed as:

[0073]

[0074] Among them, β i and ζ represent the path coefficient and the fitting error of the structural model respectively. The measurement model is expressed as:

[0075]

[0076] Among them, ε i represents the latent variable ξ i The model first initializes the weight to 1 and the error to 0, that is, all w ji =1, ε1=ε1=ε1=ζ=0. ① Calculate the measurement model using formula (2) and calculate ξi ② Perform partial least squares regression to calculate the path coefficient β i ③ζ is set to 0, and the calculated β i and i The new ④Use and x ji Calculate the correlation through bivariate regression and update w ji ⑤ Assign to ξ i . Repeat the above five steps. When the updated value of the weight is less than 10 -7 , indicating that the model has converged, and the index system fitting is completed.

[0077] Step 5: Optimize the indicator system. Eliminate the evaluation indicators with low precision according to the weight coefficient. For each latent variable, delete all the reference indicators with weights less than 0.8 and those without reference indicators with weights less than 0.5 in the fitting model. At the same time, eliminate the indicators according to the VIF value. VIF indicates the redundancy of the indicator. For the indicator x ji For , its VIF value can be calculated by the following formula:

[0078]

[0079] Where q represents the qth sample in the total number of images N. μ ji represents the average quality score calculated from the indicators of N samples, Represents ξ i Divide by x ji The value of the indicator calculated by regression for the qth sample. Calculate the VIF value for the remaining indicators after preliminary elimination. If there are indicators with VIF>10 corresponding to the latent variable, these indicators are defined as redundant indicators. If after deleting one of the redundant indicators, the number of indicators that satisfy VIF>10 is the smallest among the remaining indicators, then delete the indicator. If there are several redundant indicators, and their deletion makes the number of indicators that meet the conditions the same, then sum the VIF values ​​of these indicators and delete the indicator that minimizes the sum of VIF values. Then return to step 4. Until the number of indicators corresponding to the latent variable is less than or equal to 3, or the VIF values ​​are all less than 10.

[0080] Step 6: Evaluation of the reliability and validity of the indicator system. The reliability of the indicator system was evaluated using Cronbach's alpha coefficient, and the convergent validity of the indicator system was evaluated using the average variance extraction to verify the accuracy and reliability of the indicator system prediction. The test was conducted on two data sets. The scatter plot of the subjective score and the predicted score of the denoised image is shown in Figure 2. Figure 4 shown.

[0081] The numerical evaluation indicators of the model tested by composite reliability and convergent validity are shown in Table 1. The discriminant validity of the model on grayscale images and color images is shown in Tables 2 and 3, respectively. SS, IL, PG, and IQ represent four constructs: structural similarity SS, information loss IL, perceptual gain PG, and image quality IQ. Table 4 shows the comparison of the two prediction methods on the two datasets, which shows that the two prediction methods have significant consistency. It is worth noting that the assigned weight represents the degree of contribution of the indicator to the construct. This shows that our method can effectively replace the traditional indicator comparison method, thereby minimizing the potential bias of human subjectivity.

[0082] Table 1 Composite reliability and convergence of the measurement model

[0083]

[0084] Table 2 Discriminant validity of measurement model on grayscale image

[0085]

[0086] Table 3 Discriminant validity of measurement models on color images

[0087]

[0088] Table 4 Comparison of two prediction methods

[0089]

Claims

1. A method for establishing a comprehensive evaluation system for aerial image quality, characterized in that: The steps include: Step 1: Generation of optimized aerial images; Select the aerial image as the reference image R, and generate the distorted image D according to the distorted scene of the image; For the image under the interference of noise n, the distorted image is modeled as D = R + n; The image denoising algorithm is used to process the distorted image D to obtain the optimized image. Step 2: Initialize the indicator system model; The concept of image quality is analyzed, and its influencing factors are divided into three aspects: structural similarity, information loss and perceptual gain; Define structural similarity, information loss, perceptual gain and image quality as latent variables ξ1, ξ2, ξ3, ξ4 respectively; Select from the existing indicators and classify them according to their design characteristics and evaluation objects. Reference indicators are classified into ξ1, ξ2, ξ3, and non-reference indicators are classified into ξ4. The quality score of the selected indicators is recorded as x ji , the weight corresponding to the indicator is w ji , i represents the category to which it belongs, j represents the jth candidate indicator of the category; the number of indicators selected by these four latent variables are p1, p2, p3, and p4 respectively, and the initialized indicator system is obtained; Step 3: Calculation of image quality after optimization; Based on the processing results of step 1 and step 2, the optimized image is evaluated using the index system of step 2. Calculate the mass and get x ji ; Step 4: Index system fitting; The partial least squares structural equation model is used to fit the indicator system, which is expressed as: Among them, β i and ζ represent the path coefficient and the fitting error of the structural model, respectively; The measurement model is expressed as: Among them, ε i represents the latent variable ξ i The model first initializes the weight to 1 and the error to 0, that is, all w ji =1, ε1=ε1=ε1=ζ=0; Step 5: Optimize the indicator system; Eliminate low-precision evaluation indicators based on weight coefficients: For each latent variable, remove all reference indicators with weights lower than 0.8 and those without reference indicators with weights lower than 0.5 in the fitting model; at the same time, eliminate indicators based on VIF values. VIF indicates the redundancy of indicators. For indicator x ji , its VIF value is calculated by the following formula: Where q represents the qth sample in the total number of images N, μ ji represents the average quality score calculated from the indicators of N samples, Represents ξ i Divide by x ji The value of the indicator calculated by regression for the qth sample; Calculate the VIF value for the remaining indicators after the initial elimination. If there are indicators with VIF>10 corresponding to the latent variable, define these indicators as redundant indicators; if after deleting one of the redundant indicators, the number of indicators that meet the VIF>10 is the smallest among the remaining indicators, delete this indicator; if there are several redundant indicators, and their deletion makes the number of indicators that meet the condition the same, then sum the VIF values ​​of these indicators and delete the indicator with the smallest VIF value sum; then return to step 4 until the number of indicators corresponding to the latent variable is less than or equal to 3, or the VIF values ​​are all less than 10; Step 6: Reliability and validity assessment of the indicator system; Cronbach's alpha coefficient was used to evaluate the reliability of the index system, and the average variance extracted was used to evaluate the convergent validity of the index system to verify the accuracy and reliability of the index system prediction.

2. The method for establishing a comprehensive evaluation system for aerial image quality according to claim 1, characterized in that: The image denoising algorithm is the FFDNet algorithm.

3. The method for establishing a comprehensive evaluation system for aerial image quality according to claim 1, characterized in that: The specific steps of the index system fitting in step 4 are: ① Calculate the measurement model through formula (2) and get ξ i The value of ②Perform partial least squares regression and calculate the path coefficient β i ; ③ζ is set to 0, and the calculated β i and i The new ④Use and x ji Calculate the correlation through bivariate regression and update w ji ; ⑤ General Assign to ξ i ; Repeat the above five steps. When the updated value of the weight is less than 10 -7 , indicating that the model has converged, and the index system fitting is completed.

Images