Equipment capability assessment method based on dual assessment framework

Abstract

The invention relates to the technical field of equipment capability evaluation under a system combat background, and discloses an equipment capability evaluation method based on a dual evaluation framework, which comprises the following steps: S1, establishing the dual evaluation framework which comprises a meta evaluation layer, an equipment evaluation layer and a meta evaluation verification layer; s2, in the element evaluation layer, establishing an element evaluation index model comprising at least six core indexes, and adopting an entropy weight method-TOPSIS combination model to optimize candidate evaluation methods; s3, in the equipment evaluation layer, a three-level equipment evaluation index model is established, and the three-level equipment evaluation index model comprises a target layer, a criterion layer and a sub-index layer; and S4, in the meta-evaluation verification layer, carrying out credibility verification on an equipment evaluation result by adopting a group decision evaluation method. The method is suitable for equipment capability assessment and combat plan selection decision in the military field, can improve the accuracy, stability and decision efficiency of an assessment result, is highly consistent with expert consensus, and has practical application feasibility and operability.

Description

Technical Field

[0001] This invention relates to the field of equipment capability assessment technology in the context of system warfare, and specifically to an equipment capability assessment method based on a dual assessment framework. Background Technology

[0002] In modern military operations, equipment capability assessment is a core component of iterative optimization of operational plans and equipment selection decisions. System-of-systems operational plan decisions require stable and reliable equipment capability assessment results. Therefore, to obtain an excellent equipment capability assessment method, the assessment method itself needs to be evaluated; hence, this study refers to it as a dual assessment. This method requires cascaded assessments, constructing a unified assessment framework, and using the optimal assessment method obtained as a reliable assessment method for subsequent equipment capability assessments.

[0003] To effectively address the aforementioned challenges, there is an urgent need for an equipment capability assessment method that can achieve "objective optimization of assessment methods, quantitative assessment of equipment capabilities, and closed-loop verification of result credibility," thereby overcoming the shortcomings of existing technologies and improving the scientific nature of equipment assessment and its support capabilities for operational decision-making. Currently, optimization algorithms such as the entropy weight method and the TOPSIS method have been initially applied in the field of equipment assessment. Among them, the entropy weight method can achieve objective weighting based on the degree of data variation, the TOPSIS method can achieve multi-scheme optimization through dual-benchmark distance measurement, and the AHP method can integrate expert experience to reflect tactical requirements. However, existing technologies have not combined these three methods with meta-assessment theory to form a closed-loop framework, making it difficult to simultaneously ensure the objectivity, accuracy, and credibility of the assessment.

[0004] The equipment capability assessment method based on the dual assessment framework incorporates the concept of meta-assessment, constructing a dual assessment framework and process of "meta-assessment layer - equipment assessment layer". The meta-assessment layer uses the entropy weight method-TOPSIS model to objectively optimize the assessment method; the equipment assessment layer determines the weights of each capability assessment index of the equipment through the entropy weight-AHP method, and then uses the assessment method determined by the meta-assessment layer to quantify the equipment capability assessment, thereby effectively improving the accuracy and stability of the assessment results. Summary of the Invention

[0005] To address the aforementioned shortcomings in existing technologies, this invention provides an equipment capability assessment method based on a dual assessment framework. By constructing a closed-loop assessment framework, optimizing algorithm combinations, and defining clear constraints, this method achieves a scientific assessment of equipment capabilities.

[0006] To achieve the above-mentioned objectives, the technical solution adopted by this invention is: an equipment capability assessment method based on a dual assessment framework, comprising the following steps:

[0007] S1. Establish a dual evaluation framework, which includes a meta-evaluation layer, an equipment evaluation layer, and a meta-evaluation verification layer.

[0008] S2, In the meta-evaluation layer, a meta-evaluation index model containing at least 6 core indicators is established, and the entropy weight method-TOPSIS combined model is used to select the best evaluation method and output the optimal evaluation method.

[0009] S3. In the equipment evaluation layer, a three-level equipment evaluation index model is established, which includes a target layer, a criterion layer, and a sub-index layer. The entropy weight-AHP combined weighting method is used to determine the weight of each index, and the optimal evaluation method is used to quantitatively evaluate the equipment capability.

[0010] S4. In the meta-evaluation verification layer, the group decision-making evaluation method is used to verify the credibility of the equipment evaluation results. If the verification fails, the results are fed back to the meta-evaluation layer or the equipment evaluation layer for parameter adjustment, forming a closed-loop optimization.

[0011] Furthermore, the above-mentioned equipment capability assessment method based on the dual assessment framework includes the following six indicators in its meta-assessment index model: assessment accuracy, result stability, process objectivity, computational complexity, ability to handle uncertainty, and result interpretability. Among them, assessment accuracy, result stability, and computational complexity are cost-type indicators, while process objectivity, ability to handle uncertainty, and result interpretability are benefit-type indicators.

[0012] Furthermore, the implementation of the above-mentioned equipment capability assessment method based on the dual assessment framework, the S2 entropy weight method-TOPSIS combined model, in the meta-assessment layer includes the following steps:

[0013] S21, Construct the meta-evaluation decision matrix;

[0014] S22, standardize the indicator data;

[0015] S23, the entropy weight method is used to calculate the weight of each indicator;

[0016] S24. The TOPSIS method is used to calculate the closeness of each candidate method, and the method with the highest closeness is selected as the optimal evaluation method.

[0017] Furthermore, the implementation of the above-mentioned equipment capability assessment method based on the dual assessment framework, the S3 entropy weight-AHP combined weighting method, in the equipment assessment layer includes the following steps:

[0018] S31, Construct the original equipment data matrix;

[0019] S32, standardize the data;

[0020] S33, the objective weights are calculated using the entropy weight method, and the subjective weights are calculated using the AHP method;

[0021] S34, the objective weights and subjective weights are combined according to a preset coefficient to obtain the combined weights.

[0022] Furthermore, the formula for calculating the combined weights in the above-mentioned equipment capability assessment method based on the dual assessment framework is as follows: ,in, For entropy weighting, The weights are those for the AHP method. This is the preference coefficient.

[0023] Furthermore, the implementation of the S4 group decision evaluation method in the meta-evaluation verification layer of the above-mentioned equipment capability evaluation method based on the dual evaluation framework includes the following steps:

[0024] S41, Experts independently rank the equipment options to form an expert consensus ranking;

[0025] S42, Calculate the Spearman rank correlation coefficient ρ between the ranking of the evaluation results and the ranking of the expert consensus;

[0026] S43, if ρ≥0.90, then the verification passes; otherwise, feedback is given to adjust the parameters and re-evaluate.

[0027] Furthermore, in the above-mentioned equipment capability assessment method based on the dual assessment framework, the three-level equipment assessment index model includes at least five capability dimensions in the criterion layer: strike capability, protection capability, mobility capability, reconnaissance capability, and communication capability; and the sub-index layer includes quantifiable parameters under each capability dimension.

[0028] Furthermore, the above-mentioned equipment capability assessment method based on the dual assessment framework includes at least two of the following candidate assessment methods: fuzzy logic method, TOPSIS method, and utility function method.

[0029] The beneficial effects of this invention are as follows: This invention constructs a closed-loop dual evaluation framework of "meta-evaluation layer - equipment evaluation layer - meta-evaluation verification layer", and combines multi-algorithm combination optimization and multi-level verification mechanism. It breaks through the limitations of existing technologies in the entire process from evaluation method selection, equipment capability quantification to result credibility assurance, and provides a scientific, efficient and reliable solution for equipment capability evaluation in the context of system warfare. Its core value is reflected in improving evaluation quality and decision support capabilities in multiple dimensions.

[0030] In the evaluation method selection stage, this invention abandons the traditional model that relies on subjective experience to select evaluation methods. Instead, it establishes a meta-evaluation index model encompassing six core indicators, including evaluation accuracy and result stability. This model, combined with the entropy weight method and the TOPSIS combined model, enables quantitative screening of candidate evaluation methods. Specifically, the entropy weight method objectively calculates index weights based on data variation, avoiding interference from human bias. The TOPSIS method accurately judges the closeness of each method through a dual-benchmark distance metric, ultimately outputting the optimal evaluation method suited to the current combat mission. This ensures the reliability of subsequent equipment evaluation methods from the outset, effectively solving the problems of subjective selection and insufficient adaptability in evaluation method selection.

[0031] In the equipment capability quantitative assessment stage, this invention constructs a three-level assessment index model of "target layer - criterion layer - sub-index layer" based on the principle of combat effectiveness decomposition. It refines the comprehensive equipment capability into core dimensions such as strike, protection, and mobility, and further decomposes them into directly collectable quantitative parameters, achieving a precise mapping between "equipment capability - measured data - assessment results" and avoiding assessment bias caused by index ambiguity. Simultaneously, it employs an entropy weight-AHP combined weighting method to determine index weights. This method not only mines objective laws in the original data through entropy weighting but also integrates expert tactical experience through AHP. By pre-setting preference coefficients (α∈[0.4,0.6]), it balances data objectivity with actual combat needs. Finally, it combines the meta-assessment layer optimization method to complete the quantitative assessment. The output equipment scheme priority is more aligned with battlefield realities, providing precise data support for iterative optimization of combat schemes and equipment selection decisions. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the dual evaluation framework.

[0033] Figure 2 This is a schematic diagram of the three-level indicator system structure for equipment evaluation. Detailed Implementation

[0034] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0035] like Figures 1-2 As shown, this embodiment provides an equipment capability assessment method based on a dual assessment framework, including the following steps:

[0036] Step 1: Establish a dual evaluation framework, which includes a meta-evaluation layer, an equipment evaluation layer, and a meta-evaluation verification layer.

[0037] Step 2: In the meta-evaluation layer, establish a meta-evaluation index model containing at least 6 core indicators, and use the entropy weight method-TOPSIS combined model to select the best evaluation method and output the optimal evaluation method.

[0038] Step 3: In the equipment evaluation layer, a three-level equipment evaluation index model is established, which includes a target layer, a criterion layer, and a sub-index layer. The entropy weight-AHP combined weighting method is used to determine the weight of each index, and the optimal evaluation method is used to quantitatively evaluate the equipment capability.

[0039] Step four: In the meta-evaluation verification layer, the group decision-making evaluation method is used to verify the credibility of the equipment evaluation results. If the verification fails, the results are fed back to the meta-evaluation layer or the equipment evaluation layer for parameter adjustment, forming a closed-loop optimization.

[0040] The core process of this invention mainly includes four stages: system initialization, meta-evaluation layer method optimization, equipment evaluation layer capability quantification, and meta-evaluation verification layer credibility verification.

[0041] During the system initialization phase, the specific requirements of the evaluation task, the set of alternative equipment solutions, and the set of candidate evaluation methods are determined. The scoring criteria for the meta-evaluation indicators, the data sources and collection dimensions for equipment evaluation are initialized, and an expert review group that meets the background requirements is formed.

[0042] In the method selection stage of the meta-evaluation layer, a meta-evaluation decision matrix is ​​first constructed. Experts score candidate evaluation methods on meta-evaluation indicators, forming the decision matrix. Then, the original scores are standardized according to indicator type (benefit-oriented, cost-oriented) to eliminate the influence of dimensions. Next, based on the standardized data, the entropy value of each meta-evaluation indicator is calculated to determine its objective weight. The smaller the entropy value, the greater the role of the indicator in distinguishing different evaluation methods, and the higher its weight. Using the weights obtained by the entropy weight method, a weighted standardization matrix is ​​constructed to calculate the relative closeness of each candidate method to the positive and negative ideal solutions. The method with the highest closeness is selected as the optimal evaluation method (M). opt ), and output to the equipment evaluation layer.

[0043] In the equipment capability quantification assessment phase of the equipment evaluation layer, raw performance data of all candidate equipment schemes on the three-level evaluation indicators (sub-indicator layer) are collected. The raw data is then standardized. For cost-based indicators, a forwarding process (e.g., taking the reciprocal) is performed to transform them into benefit-based indicators before standardization. The entropy weight method is used to calculate the objective weights of each three-level indicator based on the standardized data. The AHP method is used, with experts constructing a judgment matrix to calculate the subjective weights of each three-level indicator. The subjective and objective weights are then weighted and fused according to a preset preference coefficient α to obtain a comprehensive combined weight, which is then normalized. The optimal evaluation method M output from the meta-evaluation layer is then invoked.opt By combining the standardized equipment data matrix and combined weights, the comprehensive evaluation value of each equipment scheme is calculated, and the schemes are sorted accordingly to obtain the equipment scheme priority queue (R_calc).

[0044] In the credibility verification phase of the meta-evaluation verification layer, the expert team independently ranks the equipment schemes based on standardized performance data. They then integrate the statistical "scheme advantage consensus rate" to form an expert consensus ranking vector (R_exp). Correlation calculation and verification: The Spearman rank correlation coefficient ρ between the ranking result output by the equipment evaluation layer (R_calc) and the expert consensus ranking (R_exp) is calculated. If ρ ≥ 0.90, the evaluation result is considered credible, and the final equipment priority, weakness analysis, and decision-making recommendations are output. If ρ < 0.90, the point of disagreement is identified and fed back to the meta-evaluation layer (adjusting candidate methods) or the equipment evaluation layer (adjusting indicator weights), and the evaluation process is re-executed until the credibility requirements are met.

[0045] This embodiment uses a land-based tactical assault mission as a case study to demonstrate the specific implementation process of this method in equipment selection and operational plan optimization. In this embodiment, it is assumed that there are four candidate equipment options (S1~S4), and the most suitable equipment for performing the assault mission must be selected from among them.

[0046] I. System Initialization and Parameter Setting

[0047] Assess mission requirements: For land-based tactical assault missions, the focus is on evaluating the equipment's strike capabilities, mobility, and protection capabilities.

[0048] Alternative equipment options set: S={S1,S2,S3,S4}.

[0049] Candidate evaluation method set: M = {M1: Fuzzy logic method, M2: TOPSIS method, M3: Utility function method}.

[0050] The scoring range for the meta-evaluation indicators is 1 to 10 points, with 10 points being the optimal score.

[0051] Equipment evaluation data sources: equipment performance parameter database (accuracy ≥ 95%), battlefield simulation and deduction system (matching degree ≥ 80%), historical combat data (sample size ≥ 50 sets).

[0052] II. Meta-evaluation Layer: Optimal Evaluation Methods

[0053] 1. Construct a meta-evaluation decision matrix. Invite 5 experts to score M1, M2, and M3 on 6 meta-evaluation indicators (C1~C6). Take the average score to construct the original decision matrix X, as shown in Table 1.

[0054] Table 1:

[0055]

[0056] 2. Standardize the indicators according to their type (C1, C2, and C4 are cost-based, while C3, C5, and C6 are benefit-based) using the corresponding formulas to obtain the standardized matrix Y.

[0057] The benefit-type indicators (C3, C5, C6) are calculated using formula 1:

[0058] ,

[0059] The cost indicators (C1, C2, C4) are calculated using formula 2:

[0060] .

[0061] 3. Calculate the entropy value of each indicator based on the standardized matrix Y. And entropy weight The formula for calculating entropy is Equation 3:

[0062] ;in ( To minimize, avoid (When logarithms are not meaningful).

[0063] The formula for calculating entropy weight is Equation 4:

[0064] Entropy weight reflects the discriminative power of indicator data (the smaller the entropy value, the greater the weight).

[0065] Assume the calculated entropy weight vector is:

[0066] W=[0.18,0.16,0.20,0.14,0.17,0.15].

[0067] 4. TOPSIS method for optimal evaluation:

[0068] Construct a weighted normalization matrix: .

[0069] Determine the positive and negative ideal solutions: Positive ideal solution (optimal solution): in .

[0070] Negative ideal solution (worst solution): in .

[0071] method Distance to the ideal solution: ;

[0072] Distance to the negative ideal solution: ;

[0073] Proximity: ( The closer the value is to 1, the better the method.

[0074] Assumption The calculated values ​​are M1 = 0.62, M2 = 0.85, and M3 = 0.48. Therefore, M2 is the optimal evaluation method. .

[0075] III. Equipment Assessment Layer: Quantitative Assessment of Equipment Capabilities

[0076] 1. Construct the original equipment data matrix

[0077] Raw data for four types of equipment under Level 3 indicators were collected, and some examples are shown in Table 2.

[0078] Table 2:

[0079]

[0080] 2. Data standardization processing

[0081] Benefit-oriented metrics (such as first-shot success rate and maximum cross-country speed) use benefit-oriented standardized formulas:

[0082] ;

[0083] Cost-based indicators (such as reconnaissance accuracy): first convert them into benefit-based indicators. Then, standardize according to the benefit-type formula to obtain the standardized matrix. .

[0084] 3. Entropy weighting-AHP combination weighting

[0085] Objective weighting (entropy weighting method): based on Calculate the entropy weights of each third-level indicator. ;

[0086] Subjective weighting (AHP method): Experts construct judgment matrices for the criterion layer and sub-indicator layer, calculate the hierarchical single ranking and overall ranking, and obtain the AHP weights. ;

[0087] Combined weights: After normalization, the weight vector is obtained. Assume some weights are as shown in Table 3.

[0088] Table 3:

[0089]

[0090] 4. Comprehensive evaluation using the M_opt method

[0091] Using M_opt (such as the TOPSIS method), based on and Calculate the comprehensive evaluation value of each scheme (such as the closeness of the TOPSIS method), sort them in descending order of comprehensive evaluation value, and the sorting results are shown in Table 4.

[0092] Table 4:

[0093]

[0094] Output: Equipment scheme priority queue R_calc=[S2,S4,S1,S3].

[0095] IV. Verification of the credibility of the results from the meta-evaluation verification layer

[0096] 1. Expert Ranking and Consensus Integration

[0097] Five experts independently ranked the equipment schemes based on standardized data, and calculated the "scheme advantage consensus rate" (if the percentage of experts who believe a particular scheme is superior to another scheme is ≥60%, priority is established), forming an expert consensus ranking R_exp=[S2,S4,S1,S3].

[0098] 2. Correlation Calculation

[0099] The consistency between R_calc and R_exp is measured by the Spearman rank correlation coefficient ρ:

[0100] ,in, The difference between the rankings of scheme i in R_calc and R_exp is 0 in this embodiment, and the calculated ρ is 0.

[0101] 3. Result Verification

[0102] If ρ≥0.90, directly output the equipment scheme priority queue and weakness analysis (such as a scheme with weak protection capability); if ρ<0.90, locate the divergence dimension (such as reconnaissance capability index), adjust the index weight of this dimension or the candidate method of the meta-evaluation layer until ρ≥0.90; in this embodiment, ρ=1.0≥0.90, which passes the verification.

[0103] Output priority queue and weakness analysis: S2 is the best overall with no obvious weaknesses; S4 has slightly weak protection capabilities, and it is recommended to strengthen its armor; S1 has insufficient reconnaissance and communication capabilities; S3 has weak attack and mobility capabilities and is not recommended for assault missions.

[0104] V. Output the final evaluation results

[0105] Optimal equipment configuration: S2

[0106] Alternative solution: S4 (requires enhanced protection)

[0107] Elimination plan: S1, S3

[0108] Confidence level assessment: ρ=1.0, the results are highly reliable.

[0109] Recommendation: Prioritize the deployment of S2 for assault missions, with S4 as a backup option and armor upgrades recommended.

Claims

1. A method for assessing equipment capabilities based on a dual assessment framework, characterized in that, Includes the following steps: S1. Establish a dual evaluation framework, which includes a meta-evaluation layer, an equipment evaluation layer, and a meta-evaluation verification layer. S2, In the meta-evaluation layer, a meta-evaluation index model containing at least 6 core indicators is established, and the entropy weight method-TOPSIS combined model is used to select the best evaluation method and output the optimal evaluation method. S3. In the equipment evaluation layer, a three-level equipment evaluation index model is established, which includes a target layer, a criterion layer, and a sub-index layer. The weight of each index is determined by the entropy weight-AHP combined weighting method, and the optimal evaluation method is used to quantitatively evaluate the equipment capability. S4. In the meta-evaluation verification layer, the group decision-making evaluation method is used to verify the credibility of the equipment evaluation results. If the verification fails, the results are fed back to the meta-evaluation layer or the equipment evaluation layer for parameter adjustment, forming a closed-loop optimization.

2. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The meta-evaluation index model includes the following six indicators: evaluation accuracy, result stability, process objectivity, computational complexity, ability to handle uncertainty, and result interpretability; among them, evaluation accuracy, result stability, and computational complexity are cost-type indicators, while process objectivity, ability to handle uncertainty, and result interpretability are benefit-type indicators.

3. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The implementation of the entropy weight method-TOPSIS combined model in the meta-evaluation layer as described in S2 includes the following steps: S21, Construct the meta-evaluation decision matrix; S22, standardize the indicator data; S23, the entropy weight method is used to calculate the weight of each indicator; S24. The TOPSIS method is used to calculate the closeness of each candidate method, and the method with the highest closeness is selected as the optimal evaluation method.

4. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The implementation of the entropy weight-AHP combined weighting method described in S3 at the equipment evaluation layer includes the following steps: S31, Construct the original equipment data matrix; S32, standardize the data; S33, the objective weights are calculated using the entropy weight method, and the subjective weights are calculated using the AHP method; S34, the objective weights and subjective weights are combined according to a preset coefficient to obtain the combined weights.

5. The equipment capability assessment method based on a dual assessment framework according to claim 4, characterized in that, The formula for calculating the combined weights is as follows: ,in, For entropy weighting, The weights are those for the AHP method. This is the preference coefficient.

6. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The implementation of the group decision evaluation method described in S4 in the meta-evaluation verification layer includes the following steps: S41, Experts independently rank the equipment options to form an expert consensus ranking; S42, Calculate the Spearman rank correlation coefficient ρ between the ranking of the evaluation results and the ranking of the expert consensus; S43, if ρ≥0.90, then the verification passes; otherwise, feedback is given to adjust the parameters and re-evaluate.

7. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The three-level equipment evaluation index model includes at least five capability dimensions in the criteria layer: strike capability, protection capability, mobility capability, reconnaissance capability, and communication capability; and the sub-index layer includes quantifiable parameters under each capability dimension.

8. The equipment capability assessment method based on a dual assessment framework according to claim 1, characterized in that, The candidate evaluation methods include at least two of the following: fuzzy logic method, TOPSIS method, and utility function method.

Images