An infrared image fidelity evaluation method based on analytic hierarchy process

Abstract

The application relates to an infrared image fidelity evaluation method based on an analytic hierarchy process, which comprises the following steps: firstly, constructing an analytic hierarchy process model overall framework; then, constructing a judgment matrix of each layer evaluation factor; finally, determining the weight of each layer factor, and performing judgment matrix consistency checking to determine the final fidelity. The method adopts relative scales to compare multiple factors with each other, reduces the difficulty of comparing factors with different properties with each other, and improves the accuracy; and quantitatively determines the weight between each evaluation index, and then completes comprehensive evaluation of the image fidelity.

Description

Technical Field

[0001] This application relates to the fields of infrared scene simulation technology and high-fidelity scene image technology, specifically, to an infrared image fidelity evaluation method based on the analytic hierarchy process. Background Technology

[0002] Infrared scene simulation offers numerous advantages. First, it is highly efficient. Field tests require repeated setup and adjustments, leading to low efficiency, especially in the early stages of development and delays. Simulation, however, involves computer modeling and rendering, allowing for rapid and repeatable testing. Second, infrared scene simulation systems are reusable, enabling convenient switching of battlefield conditions. By adjusting parameters within the system, different combat scenarios can be simulated, generating simulated images under varying weather conditions, infrared bands, backgrounds, and distances. Furthermore, infrared scene simulation carries extremely low risk, particularly crucial in the development and testing of new weapon systems where system instability can pose significant risks. These advantages are unattainable in field testing, making infrared scene simulation an essential choice for extensive testing during the development process.

[0003] With the widespread application of scene simulation technology, the requirements for infrared scene simulation are becoming increasingly stringent. How to evaluate the results of infrared scene simulation has become a problem that must be solved. If the simulated image deviates significantly from the real scene, it will severely affect the accuracy of weapon systems and even cause the failure of a large amount of research and development work, resulting in huge losses. Therefore, evaluating simulation results is a crucial step in the development of infrared scene simulation. Currently, there is limited research on evaluation schemes for infrared scene simulation, and effective evaluation methods and mature evaluation systems have not yet been established. Summary of the Invention

[0004] To overcome at least one deficiency in the prior art, this application provides an infrared image fidelity evaluation method based on the analytic hierarchy process.

[0005] Firstly, a method for evaluating the fidelity of infrared images based on the analytic hierarchy process (AHP) is provided, including:

[0006] Construct the overall architecture of the analytic hierarchy process (AHP) model; the overall architecture of the AHP model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective.

[0007] Based on multiple evaluation criteria and the evaluation objectives corresponding to each evaluation criterion, an evaluation criterion judgment matrix and an evaluation objective judgment matrix containing multiple evaluation objectives are constructed.

[0008] For both infrared simulation images and real-world images, the fidelity of each evaluation target is calculated; the fidelity of evaluation targets belonging to the same evaluation criterion constitutes the fidelity of the evaluation criterion.

[0009] Perform a consistency check on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency check;

[0010] Based on the evaluation criterion judgment matrix, calculate the total weight; based on the evaluation target judgment matrix that has passed the consistency test, calculate the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test; and determine the weight of the evaluation target judgment matrix that has not passed the consistency test, as well as the weight of the evaluation criterion containing one evaluation target.

[0011] The final fidelity is determined based on the weights and evaluation criteria.

[0012] In one embodiment, the multiple evaluation criteria include at least two of image structure, image information, network inversion, neural network, and domain adaptation evaluation.

[0013] In one embodiment, the evaluation target corresponding to the image structure includes at least one of color fidelity, structural fidelity, shape fidelity, texture fidelity, and radiation fidelity.

[0014] In one embodiment, the evaluation target corresponding to the image information includes at least one of grayscale variance, information entropy, edge density, correlation coefficient, and image quality.

[0015] In one embodiment, the evaluation target corresponding to the neural network includes at least one of a Siamese network and an object detection network.

[0016] In one embodiment, the method further includes constructing a judgment matrix based on the evaluation scale of the judgment matrix.

[0017] In one embodiment, a consistency check is performed on all evaluation target judgment matrices to determine the evaluation target judgment matrices that pass the consistency check, including:

[0018] Calculate the largest eigenvalue of the evaluation target judgment matrix;

[0019] Calculate the consistency ratio based on the largest eigenvalue;

[0020] If the consistency ratio is less than the set value, the evaluation target judgment matrix passes the consistency test.

[0021] In one embodiment, the total weight is calculated based on the evaluation criterion judgment matrix; the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test is calculated based on the evaluation target judgment matrix that has passed the consistency test; the weights are calculated using the following formula:

[0022]

[0023] Among them, a ij To determine the element in the i-th row and j-th column of a matrix, w i Let be the weight corresponding to the i-th row, and n be the number of rows and columns of the judgment matrix.

[0024] In one embodiment, determining the final fidelity based on the fidelity of weights and evaluation criteria includes:

[0025]

[0026] Among them, L 总 For the ultimate realism, w 总 For the total weight, w k L represents the weight corresponding to the k-th evaluation criterion. k Let K be the fidelity of the k-th evaluation criterion, where K is the number of evaluation criteria.

[0027] Secondly, an infrared image fidelity evaluation device based on the analytic hierarchy process (AHP) is provided, comprising:

[0028] The architecture building module is used to construct the overall architecture of the analytic hierarchy process (AHP) model. The overall architecture of the AHP model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective.

[0029] The judgment matrix construction module is used to construct an evaluation criterion judgment matrix and an evaluation target judgment matrix containing multiple evaluation targets, based on multiple evaluation criteria and the evaluation target corresponding to each evaluation criterion.

[0030] The realism calculation module is used to calculate the realism of each evaluation target for infrared simulation images and real images; the realism of evaluation targets belonging to the same evaluation criterion constitutes the realism of the evaluation criterion.

[0031] The consistency verification module is used to perform consistency verification on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency verification.

[0032] The weight calculation module is used to calculate the total weight based on the evaluation criterion judgment matrix; calculate the weight of the evaluation criteria corresponding to the evaluation target judgment matrix that has passed the consistency test based on the evaluation target judgment matrix that has passed the consistency test; and determine the weight of the evaluation target judgment matrix that has not passed the consistency test, as well as the weight of the evaluation criteria that contain one evaluation target.

[0033] The fidelity determination module is used to determine the final fidelity based on weights and evaluation criteria.

[0034] Compared with existing technologies, this application has the following advantages: This application provides an infrared image fidelity evaluation method based on the analytic hierarchy process (AHP). First, it constructs the overall architecture of the AHP model; then, it constructs the judgment matrix for each layer of evaluation factors; finally, it determines the weights of each layer of factors and performs a consistency check on the judgment matrix to determine the final fidelity. This method uses a relative scale to compare multiple factors pairwise, reducing the difficulty of comparing factors with different properties and thus improving accuracy; and it quantitatively determines the weights between various evaluation indicators, thereby completing a comprehensive evaluation of image fidelity. Attached Figure Description

[0035] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings:

[0036] Figure 1 A flowchart of an infrared image fidelity evaluation method based on the analytic hierarchy process according to an embodiment of this application is shown.

[0037] Figure 2 A schematic diagram of the overall architecture of the analytic hierarchy process (AHP) model is shown.

[0038] Figure 3 A structural block diagram of an infrared image fidelity evaluation device based on the analytic hierarchy process according to an embodiment of this application is shown. Detailed Implementation

[0039] Exemplary embodiments of the present application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer’s specific objectives, and these decisions may vary as the embodiments differ.

[0040] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution according to this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0041] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0042] This application provides a method for evaluating the fidelity of infrared images based on tomographic analysis. Figure 1A flowchart illustrating an infrared image fidelity evaluation method based on the analytic hierarchy process according to an embodiment of this application is shown. See also... Figure 1 The methods include:

[0043] Step S1: Construct the overall architecture of the analytic hierarchy process (AHP) model. The overall architecture of the AHP model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective.

[0044] Figure 2 A schematic diagram of the overall architecture of the analytic hierarchy process (AHP) model is shown below. Figure 2 The evaluation criteria include image structure, image information, network inversion, neural networks, and domain adaptation evaluation. Among them, the evaluation objectives corresponding to image structure include color fidelity, structural fidelity, shape fidelity, texture fidelity, and radiometric fidelity; the evaluation objectives corresponding to image information include gray-level variance, information entropy, edge density, correlation coefficient, and image quality; and the evaluation objectives corresponding to neural networks include Siamese networks and object detection networks.

[0045] Step S2 involves constructing an evaluation criterion judgment matrix and an evaluation target judgment matrix for evaluation criteria that include multiple evaluation targets, based on multiple evaluation criteria and the corresponding evaluation objectives for each criterion. Here, evaluation criteria with only one evaluation objective do not require the construction of an evaluation target judgment matrix, such as neural network and domain adaptation evaluation.

[0046] Here, the consistent matrix method can be used to construct the judgment matrix, that is, instead of comparing all factors together, we compare them pairwise; and we use relative scales to minimize the difficulty of comparing factors with different properties, thereby improving accuracy. The evaluation scales between pairs of factors are shown in Table 1.

[0047] Table 1 Evaluation Scale of the Judgment Matrix

[0048] 1 The i-indicator is just as important as the j-indicator. 3 The i indicator is slightly more important than the j indicator. 5 The i-indicator is significantly more important than the j-indicator. 7 The i-indicator is much more important than the j-indicator. 9 The i-indicator is much more important than the j-indicator. 2，4，6，8 Based on the median of the above pairwise judgments reciprocal Describe the importance of the j-index and the i-index.

[0049] The constructed evaluation criterion judgment matrix is ​​shown in Table 2.

[0050] Table 2 Evaluation Criteria Judgment Matrix

[0051] Image structure 1 3 5 5 3 Image information 1 / 3 1 3 3 2 Network inversion 1 / 5 1 / 3 1 3 2 Neural Networks 1 / 5 1 / 3 1 / 3 1 2 Domain Adaptation Assessment 1 / 3 1 / 2 1 / 2 1 / 2 1

[0052] Based on the established hierarchical structure model, each factor at each level is compared pairwise with the factor at the level above it, using the scaling method in Table 1 to construct a judgment matrix. The scaling values ​​of the indicators are determined by experts, and the judgment matrix A = [a ij ] m×n a ijTo determine the element in the i-th row and j-th column of matrix A, where m is the row number and n is the column number, the specific evaluation criteria and their corresponding evaluation target judgment matrices are shown in Tables 3, 4, and 5. Since the fidelity evaluation index based on network inversion and domain adaptation evaluation has only one evaluation target, a judgment matrix is ​​not required. The five evaluation criteria employed encompass image structure, information, and the characteristics of image applications in deep learning networks, making the evaluation task comprehensive and diverse, reflecting the overall credibility, ensuring the accuracy of the evaluation method, and increasing its persuasiveness.

[0053] Table 3 Evaluation target judgment matrix corresponding to image structure.

[0054] Color realism 1 7 5 7 5 Structural realism 1 / 7 1 2 3 3 Shape realism 1 / 5 1 / 2 1 2 3 Texture realism 1 / 7 1 / 3 1 / 2 1 3 Radiation fidelity 1 / 5 1 / 3 1 / 3 1 / 3 1

[0055] Table 4 Evaluation Target Judgment Matrix Corresponding to Image Information

[0056] Correlation coefficient 1 7 7 7 7 9 Gray variance 1 / 7 1 2 3 3 5 Information entropy 1 / 7 1 / 2 1 2 2 5 Edge density 1 / 7 1 / 3 1 / 2 1 2 5 Edge strength 1 / 7 1 / 3 1 / 2 1 / 2 1 5 Image quality 1 / 9 1 / 5 1 / 5 1 / 5 1 / 5 1

[0057] Table 5 Evaluation Target Judgment Matrix Corresponding to Neural Network

[0058] Twin Network 1 3 Target detection 1 / 3 1

[0059] Step S3: For both the infrared simulation image and the actual captured image, calculate the fidelity corresponding to each evaluation target; the fidelity corresponding to evaluation targets belonging to the same evaluation criterion constitutes the fidelity of the evaluation criterion. The calculation method for the fidelity corresponding to each evaluation target is described in detail below.

[0060] (I) Evaluation Objectives Corresponding to Image Structure

[0061] (1) Color fidelity

[0062] Histogram fidelity is the most common method for comparing image color features. A grayscale histogram is used to classify the grayscale values ​​of an image, count the number of image pixels contained in each grayscale range, and then normalize them, that is, to count the proportion of each grayscale range in the image. This process can also be called grayscale quantization.

[0063] Here, grayscale histograms are statistically analyzed for both the infrared simulation image and the real-world image to obtain their respective histogram vectors; then, the cosine distance between the two histogram vectors is calculated as the color fidelity.

[0064] (2) Structural realism

[0065] SSIM (Structural Similarity) is a metric for measuring the similarity between two images. Here, we calculate the structural similarity between an infrared simulation image and a real-world image as the structural fidelity.

[0066] (3) Shape realism

[0067] For image shape, Hu shape invariant moments (HIMs) can be used to obtain the characteristic moments of the image. Moment functions are a crucial reference metric in image analysis. The calculated HIMs not only encompass the global features of the image shape but also include geometric information regarding size, position, orientation, and shape. Therefore, HIMs are used to evaluate the fidelity of image content.

[0068] Here, the second and third moments of the infrared simulation image and the real-world image are first calculated separately. Then, based on the second and third moments, seven shape-invariant moments for each image are obtained. The shape fidelity is calculated using the following formula:

[0069]

[0070] Where, sim Hu For shape realism, M 1i M is the i-th shape-invariant moment of the infrared simulation image. 2i Let be the i-th shape-invariant moment of the real-shot image.

[0071] (4) Texture realism

[0072] Texture features are primarily used to describe the arrangement and organization of surfaces in an image; they are a type of sequentially distributed visual feature. First, the simulated infrared image and the actual captured image are input into a Gabor filter for Gabor transform, resulting in texture feature maps for the two images. These texture feature maps are then converted into feature vectors, and the distance between the two feature vectors is calculated using a distance function, which is used as the texture fidelity.

[0073] (5) Radiation fidelity

[0074] The imaging principle of infrared images is to convert infrared thermal radiation into corresponding electrical signals, and then into an image that can be recognized by the human eye. Therefore, the determining factors for contrast and brightness differences in infrared images lie in the changes in radiation intensity, and these differences can be directly compared by comparing the differences between image pixels. Thus, comparing this aspect of realism is mainly done by comparing the grayscale differences between the pixels of the original image and the simulated image. The most commonly used methods are mean square error (MSE) and peak signal-to-noise ratio (PSNR). PSNR is actually a variation of MSE, representing the ratio of the maximum power of a signal to the destructive noise power that affects its accuracy. For most image signals, there is usually a very wide dynamic range, so the logarithm is typically taken. Therefore, the PSNR of the original image and the simulated image can be defined as:

[0075]

[0076] Among them, MAX i This represents the maximum value defined for an image pixel, typically 255. Both MES and PSNR are the most widely used metrics for evaluating image errors, but both assume that image pixels are independent of each other, ignoring the correlation between image pixels and location information, and failing to explain many subtle differences in human perception.

[0077] Here, the mean square error (MSE) of the infrared simulation image and the actual captured image can be calculated separately. The ratio of the smaller MSE to the larger MSE in the two images is used as the final radiometric fidelity. It should be noted that the MSE can also be expressed as the peak signal-to-noise ratio (PSNR).

[0078] (II) Evaluation Objectives Corresponding to Image Information

[0079] (1) Gray-scale variance

[0080] The grayscale variance of an image reflects the average degree of grayscale variation. The greater the average degree of grayscale variation, the clearer the image. The grayscale variance algorithm uses the average grayscale value of all pixels in the image as a reference, calculates the difference between the grayscale values ​​of each pixel, sums the squares, and then normalizes it using the total number of pixels. Here, the grayscale variance of the infrared simulation image and the real-world image can be calculated separately, and the ratio of the smaller grayscale variance value to the larger grayscale variance value in the two images can be used as the final evaluation target value.

[0081] (2) Information Entropy

[0082] The significance of image entropy lies in the fact that it characterizes the clustering properties of image grayscale distribution, but it cannot reflect the spatial characteristics of the grayscale distribution. To characterize these spatial characteristics, a feature quantity that reflects the spatial characteristics of the grayscale distribution can be introduced on top of one-dimensional entropy to form two-dimensional image entropy. The greater the probability of an event occurring, the smaller its information content, and the smaller the entropy value. The larger the entropy value of an image, the better the image quality.

[0083] Image information entropy is used as an indicator of similarity. However, since the information entropy values ​​of most complex images are quite similar and fall within a narrow, finite range, directly using a ratio to represent the difference between the information entropy of two images cannot provide a very intuitive evaluation of their similarity. Therefore, leveraging the explosive growth characteristic of exponential operations, we first calculate the information entropy of the infrared simulation image and the actual captured image separately. Then, we calculate the exponential value of the information entropy. The ratio of the exponential value of the smaller information entropy in the two images to the exponential value of the larger information entropy in the two images is used as the final evaluation target value. That is:

[0084]

[0085] Among them, Eentropy e is an information entropy evaluation metric. min For a smaller information entropy, e max It has a relatively large information entropy.

[0086] (3) Edge density

[0087] Image edge density reflects the distribution of image edges throughout the entire image and can effectively describe the edge distribution characteristics of the image. The steps for calculating image edge density are as follows: First, edge detection is performed on the image using the Canny operator. Then, the distances from the edges to other pixels in the image are calculated. These distance values ​​are normalized, and their average value is taken to obtain the image edge density.

[0088] Image edge density, as an indicator of similarity evaluation, can be directly expressed as a ratio representing the difference in gray-level variance between two images. To standardize the evaluation indicator, the ratio of the smaller edge density value to the larger edge density value in the two images is used as the final evaluation target value E. ed :

[0089]

[0090] Among them, ed min and ed max These represent the smaller and larger values ​​of the image edge density, respectively.

[0091] (4) Edge strength

[0092] The semantic information of an image is mainly represented by the edge strength of each pixel, which quantifies the chance or probability that each pixel belongs to the edge of a certain semantic object. By analyzing the features of semantic edges and defining edge strength based on these features, an image quality metric based on edge strength similarity can be proposed.

[0093] Here, we first calculate the edge intensity around each pixel in the infrared simulation image and the real-world image, and then use the following formula to calculate the target value E for edge intensity evaluation. ESSIM :

[0094]

[0095] Where E(f,i) is the edge intensity around pixel i in image f, E(g,i) is the edge intensity around pixel i in image g, N is the number of pixels in the image, and C is an arbitrary constant.

[0096] (5) Correlation coefficient

[0097] Covariance is an indicator of the correlation between two random variables (a covariance greater than 0 indicates a positive correlation, and a covariance less than 0 indicates a negative correlation). However, the magnitude of the covariance value cannot accurately measure the degree of correlation between two random variables, as it is affected by the dimensions of the two variables and is not suitable for comparison. To better measure the correlation between two random variables, the Pearson correlation coefficient was introduced, which is calculated by dividing the covariance by the standard deviations of the two random variables, thus eliminating the influence of dimensions.

[0098] The fidelity between images is measured using the Pearson correlation coefficient. The Pearson correlation coefficient is calculated by dividing the covariance of two variables by the product of their standard deviations. Here, the correlation coefficient R(X,Y) between two images X and Y is calculated using the following formula:

[0099]

[0100] Among them, X i and Y i Let i and represent the pixel values ​​of pixel i in the two images, respectively. and represents the mean of the pixel values ​​of the two images, and n is the total number of pixels in the image.

[0101] (6) Image quality

[0102] Here, the NIQE algorithm is used to determine the NIQE index between the infrared simulation image and the real image, which serves as the target value for image quality assessment.

[0103] (III) Network Inversion

[0104] The evaluation objective is the inversion fidelity. Specifically, the inversion network used is either a CUT-based image inversion model or a BCE-CycleGAN-based image inversion model. Infrared simulation images and real-world images are input into the inversion network, and the loss function, i.e., the inversion fidelity L, is calculated based on the network output. all The following formula is used:

[0105] L all =S L1 +S L2 +Smooth L1 +L BCE +D KL

[0106] Among them, S L1 Let S be the L1 norm loss function. L2 Smooth is the L2 norm loss function. L1 Let L be the smoothed L1 norm loss function. BCELet BCEWithLogitsLoss be the loss function, and D be the loss function. KL Let KL divergence be denoted as KL divergence.

[0107] (iv) Evaluation of neural network fidelity

[0108] (1) Simulation fidelity evaluation based on Siamese neural network

[0109] Specifically, firstly, a Siamese neural network is constructed, which includes two feature extraction models with the same network structure and a fidelity calculation module; the feature extraction model is a residual neural network with an embedded convolutional attention mechanism.

[0110] Then, the Siamese neural network is trained based on the training dataset to obtain the trained Siamese neural network. The samples in the training dataset are image pairs. The two images in the image pair are input into two feature extraction models, and two image features are output. The fidelity calculation module calculates the fidelity based on the two image features. Here, the left and right paths of the Siamese neural network (feature extraction models) are completely identical and share the weights W, where F W (X) is the feature mapping function fitted by the network, where X is the input. Its function is to map sample pairs to the target feature space and measure the fidelity of the features of both pairs within the space. G W (X1, X2) is the network output, representing the fidelity of the input image pair, denoted by the formula G. W (X1, X2) = ||F W (X1)-F W (X2)||, where G w (X1, X2) represents the fidelity of the image pair X1, X2, F W (X1) represents the image features of image X1, F W (X2) represents the image features of image X2.

[0111] Finally, the real-world images and simulation images are input into the trained Siamese neural network to obtain the realism.

[0112] (2) Simulation fidelity evaluation based on target detection task

[0113] Here, based on an object detection network, such as the YOLO network, two images to be evaluated containing the target are sequentially detected to obtain their respective detection results, including the detection category, confidence score, and bounding box coordinates. Then, the detection results of the two images are compared sequentially, and the target results with the same detection category and bounding box coordinates that meet the set intersection-union threshold are selected. Finally, the evaluation fidelity of the detection results that do not meet the selection criteria is 0, and the evaluation fidelity of the detection results that meet the selection criteria is obtained by calculating the difference in confidence scores of the selected results. Finally, the evaluation fidelity of all detection results is traversed and the average value is taken as the final fidelity of the two images.

[0114] The specific calculation formula is as follows, where the target detection results of images A and B are O. A and O B , can be represented as: (O A1 O A2 , ..., O An ), (O B1 O B2 , ..., O Bm The detection confidence level for the two images is P. A and P B , can be represented as: (P A1 P A2 , ..., P An ), (P B1 P B2 , ..., P Bm ), n≥m.

[0115]

[0116]

[0117] Where n and m are the number of target detection results in the two images, S(A, B) is the evaluation fidelity of the corresponding target detection results in the two images A and B, θ is the set cross-union ratio threshold, and S is the final fidelity of the two images based on target detection evaluation. Ai O A The detection result of the i-th target in the middle, O Bj O B The detection result of the j-th target in the middle.

[0118] (V) Domain Adaptation Fidelity Assessment

[0119] Deep domain adaptation techniques, specifically, employ two merged alignment methods. The entire network is divided into a feature extraction module, a global alignment module, a fully convolutional module, and a center alignment module. First, the feature extractor module extracts features from the source and target domains; the backbone network extracts image features. The global alignment module performs global feature alignment using a global discriminator and prediction loss. The fully convolutional module predicts the target pixel-wise and performs centering mapping. Finally, the center alignment module merges the extracted features.

[0120] Given a source domain image I s and target domain image I t Given a shared feature extractor G, features are first extracted, and global alignment is performed using a global discriminator and a prediction loss. The global alignment loss function L1(x, y) is shown in the following equation:

[0121] L1(x, y) = ||f(x) - f(y)|| 2

[0122] Where f(x) and f(y) represent the global feature vectors of the source domain image x (infrared simulation image) and the target domain image y (real shot image), respectively. 2 This represents the L2 norm.

[0123] Following G, a fully convolutional module P is used to predict the pixel-wise target and perform centering mapping. The center alignment loss function L2(x, y) is shown in the following equation:

[0124] L2(x, y) = ||c1 - c2|| 2

[0125] In the formula, c1 and c2 represent the center coordinates of the source domain image and the target domain image, respectively.

[0126] Overall realism L all It can be represented as:

[0127] L all = L1(x, y) + L2(x, y)

[0128] Step S4: Perform a consistency check on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency check.

[0129] Step S5: Calculate the total weight based on the evaluation criterion judgment matrix; calculate the weight of the evaluation criteria corresponding to the evaluation target judgment matrix that passed the consistency test based on the evaluation target judgment matrix that passed the consistency test; and determine the weight of the evaluation target judgment matrix that failed the consistency test, as well as the weight of the evaluation criteria that contain only one evaluation target; here, the weight of the evaluation target judgment matrix that failed the consistency test is 0, and the weight of the evaluation criteria that contain only one evaluation target is 1.

[0130] Step S6: Determine the final fidelity based on the weights and evaluation criteria.

[0131] In this embodiment, the overall architecture of the analytic hierarchy process (AHP) model is first constructed, then the judgment matrix of each layer of evaluation factors is constructed, and finally the weights of each layer of factors are determined, and the consistency of the judgment matrix is ​​checked. This method uses a relative scale to compare multiple factors pairwise, reducing the difficulty of comparing factors with different properties and thus improving accuracy; and it quantitatively determines the weights between each evaluation index, thereby completing the comprehensive evaluation of image realism.

[0132] In one embodiment, consistency testing refers to determining the acceptable range of inconsistencies for judgment matrix A. Step S4 involves performing consistency testing on all evaluation target judgment matrices to determine the evaluation target judgment matrices that pass the consistency test, including:

[0133] Step S41: Calculate the largest eigenvalue of the evaluation target judgment matrix.

[0134] Here, the judgment matrix is ​​first normalized to obtain the normalized weights for the rows and columns:

[0135]

[0136]

[0137]

[0138] Among them, a ij To determine the element in the i-th row and j-th column of a matrix, The normalized weights corresponding to the rows, For the normalized weights corresponding to the columns, For a ij The corresponding normalized element, where n is the number of rows or columns in the judgment matrix.

[0139] Then, calculate the feature vector weights:

[0140]

[0141] Then, calculate the largest eigenvalue λ. max :

[0142]

[0143] Step S42: Calculate the consistency ratio CR based on the largest eigenvalue;

[0144]

[0145] Wherein, RI is the random consistency index, and its values ​​are shown in Table 6:

[0146] Table 6 Random Consistency Indicators

[0147] RI 0 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 1.49 1.51

[0148] Step S43: If the consistency ratio is less than the set value, the target judgment matrix passes the consistency test. Here, when CR < 0.1, the degree of inconsistency of the judgment matrix is ​​considered to be within the acceptable range, with satisfactory consistency, and the consistency test can be passed.

[0149] In one embodiment, in step S5, the total weight is calculated based on the evaluation criterion judgment matrix; the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test is calculated based on the evaluation target judgment matrix that has passed the consistency test; the weights are calculated using the following formula:

[0150]

[0151] Among them, a ij To determine the element in the i-th row and j-th column of a matrix, w i Let represent the weight corresponding to the i-th row, and n be the number of rows and columns of the judgment matrix. The judgment matrix here can be either the evaluation criterion judgment matrix or the evaluation target judgment matrix that has passed the consistency test.

[0152] In some specific embodiments, the total weight is: w 总 =[0.467,0.2258,0.1363,0.0873,0.0859];

[0153] The weights corresponding to the image structure are: w1 = [0.4647, 0.2258, 0.1363, 0.0873, 0.0859].

[0154] In one embodiment, step S6, determining the final fidelity based on the weights and evaluation criteria, includes:

[0155]

[0156] Among them, L 总 For the ultimate realism, w 总 For the total weight, wk L represents the weight corresponding to the k-th evaluation criterion. k Let K be the fidelity of the k-th evaluation criterion, where K is the number of evaluation criteria.

[0157] Here, the fidelity L of the k-th evaluation criterion k The evaluation matrix is ​​composed of the realism of all evaluation objectives under this evaluation criterion. It should be noted that the weight of the evaluation objective judgment matrix that fails the consistency test is 0, that is, the weight of the evaluation criterion corresponding to the evaluation objective judgment matrix that fails the consistency test is 0, and the weight of the evaluation criterion that contains only 1 evaluation objective is 1.

[0158] Specifically, K = 5, k = 1, 2, 3, 4, 5, representing image structure, image information, network inversion, neural network, and domain adaptation evaluation, respectively, and W3 = 1, W5 = 1.

[0159] The infrared image fidelity evaluation method of this application reasonably quantifies the structure, information, and other application characteristics of infrared images, and conducts multi-angle and multi-faceted analysis of infrared images to obtain the fidelity of each node in the bottom target layer, the intermediate criterion layer, and the top target layer of this hierarchical analysis method. The simulation fidelity evaluations of three different sets of real-world images and infrared simulation images are shown in Tables 7, 8, and 9.

[0160] In summary, this application decomposes five relevant factors—image structure, image information, network inversion, neural network, and pre-adaptive evaluation—into several levels from top to bottom according to their different attributes. It then quantitatively determines the weights among the various evaluation indicators to achieve a comprehensive evaluation of the fidelity of infrared images. This application employs a relative scale to compare each factor pairwise, reducing the difficulty of comparing factors with different properties, thereby improving accuracy and laying the foundation for subsequent work.

[0161] Table 7

[0162]

[0163] Table 8

[0164]

[0165] Table 9

[0166]

[0167]

[0168] Employing the same inventive concept as the analytic hierarchy process (AHP)-based infrared image fidelity evaluation method, this embodiment also provides a corresponding AHP-based infrared image fidelity evaluation device. Figure 3A structural block diagram of an infrared image fidelity evaluation device based on the analytic hierarchy process according to an embodiment of this application is shown, comprising:

[0169] Architecture building module 31 is used to build the overall architecture of the analytic hierarchy process (AHP) model; the overall architecture of the AHP model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective.

[0170] The judgment matrix construction module 32 is used to construct an evaluation criterion judgment matrix and an evaluation target judgment matrix containing multiple evaluation targets based on multiple evaluation criteria and the evaluation target corresponding to each evaluation criterion.

[0171] The realism calculation module 33 is used to calculate the realism of each evaluation target for infrared simulation images and real images; the realism of evaluation targets belonging to the same evaluation criterion constitutes the realism of the evaluation criterion.

[0172] The consistency verification module 34 is used to perform consistency verification on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency verification.

[0173] The weight calculation module 35 is used to calculate the total weight based on the evaluation criterion judgment matrix; calculate the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test based on the evaluation target judgment matrix that has passed the consistency test; and determine the weight of the evaluation target judgment matrix that has not passed the consistency test, as well as the weight of the evaluation criterion containing one evaluation target.

[0174] The fidelity determination module 36 is used to determine the final fidelity based on the weights and evaluation criteria.

[0175] The infrared image fidelity evaluation device based on the analytic hierarchy process in this embodiment has the same inventive concept as the infrared image fidelity evaluation method based on the analytic hierarchy process described above. Therefore, the specific implementation of this device can be found in the embodiment section of the infrared image fidelity evaluation method based on the analytic hierarchy process described above, and its technical effects correspond to the technical effects of the above method, so it will not be repeated here.

[0176] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for evaluating the fidelity of infrared images based on the analytic hierarchy process (AHP), characterized in that, include: Construct the overall architecture of the analytic hierarchy process (AHP) model; The overall architecture of the analytic hierarchy process model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective. Based on the multiple evaluation criteria and the evaluation objectives corresponding to each evaluation criterion, an evaluation criterion judgment matrix and an evaluation objective judgment matrix containing multiple evaluation objectives corresponding to the evaluation criteria are constructed. For both infrared simulation images and real-world images, the fidelity of each evaluation target is calculated. The fidelity of evaluation targets belonging to the same evaluation criterion constitutes the fidelity of the evaluation criterion. Perform a consistency check on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency check; Calculate the total weight based on the evaluation criteria judgment matrix; Based on the evaluation target judgment matrix that has passed the consistency test, calculate the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test; And determine the weights of the evaluation target judgment matrix that failed the consistency test, as well as the weights of the evaluation criteria that contain one evaluation target; Based on the realism of the weights and the evaluation criteria, the final realism is determined; The multiple evaluation criteria include image structure, image information, network inversion, neural network, and domain adaptation evaluation; The evaluation targets corresponding to the neural network include Siamese networks and object detection networks; The fidelity calculation method for object detection networks is as follows: Two images and The target detection results are and , respectively represented as: , The detection confidence levels corresponding to the two images are: and , respectively represented as: , , ; Where n and m are two images respectively. and The number of target detection results, Two images and The corresponding evaluation of the fidelity of the target detection results. The cross-union ratio threshold is set. The final fidelity of the two images is evaluated based on object detection. express The Middle Target detection results express The Middle Target detection results, for and The assessment of realism.

2. The method as described in claim 1, characterized in that, The evaluation targets corresponding to the image structure include at least one of color fidelity, structural fidelity, shape fidelity, texture fidelity, and radiation fidelity.

3. The method as described in claim 1, characterized in that, The evaluation targets corresponding to the image information include at least one of grayscale variance, information entropy, edge density, correlation coefficient, and image quality.

4. The method as described in claim 1, characterized in that, The method also includes constructing a judgment matrix based on the evaluation scale of the judgment matrix.

5. The method as described in claim 1, characterized in that, in, Perform a consistency test on all evaluation target judgment matrices, and determine the evaluation target judgment matrices that pass the consistency test, including: Calculate the largest eigenvalue of the evaluation target judgment matrix; Calculate the consistency ratio based on the largest eigenvalue; If the consistency ratio is less than the set value, the evaluation target judgment matrix passes the consistency test.

6. The method as described in claim 1, characterized in that, in, Calculate the total weight based on the evaluation criteria judgment matrix; Based on the evaluation target judgment matrix that has passed the consistency test, calculate the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test; The weights are calculated using the following formula: in, To determine the matrix of the first i Line number j Column elements, For the first i The weight corresponding to the row n To determine the number of rows and columns of a matrix.

7. The method as described in claim 1, characterized in that, in, Based on the realism of the weights and the evaluation criteria, the final realism is determined, including: in, For the ultimate level of realism, For the total weight, For the first k The weights corresponding to each evaluation criterion For the first k The realism of each evaluation criterion K The number of evaluation criteria.

8. An infrared image fidelity evaluation device based on the analytic hierarchy process (AHP), characterized in that, To implement the method according to any one of claims 1-7, comprising: An architecture building module is used to construct the overall architecture of the analytic hierarchy process (AHP) model; the overall architecture of the AHP model includes multiple evaluation criteria, and each evaluation criterion includes at least one evaluation objective. The judgment matrix construction module is used to construct an evaluation criterion judgment matrix and an evaluation target judgment matrix containing multiple evaluation targets based on the multiple evaluation criteria and the evaluation target corresponding to each evaluation criterion. The realism calculation module is used to calculate the realism of each evaluation target for infrared simulation images and real images; the realism of evaluation targets belonging to the same evaluation criterion constitutes the realism of the evaluation criterion. The consistency verification module is used to perform consistency verification on all evaluation target judgment matrices and determine the evaluation target judgment matrices that pass the consistency verification. The weight calculation module is used to calculate the total weight based on the evaluation criterion judgment matrix; calculate the weight of the evaluation criterion corresponding to the evaluation target judgment matrix that has passed the consistency test based on the evaluation target judgment matrix that has passed the consistency test; and determine the weight of the evaluation target judgment matrix that has not passed the consistency test, as well as the weight of the evaluation criterion containing one evaluation target. The fidelity determination module is used to determine the final fidelity based on the weights and the evaluation criteria.

Images