Probabilistic method for estimating the reliability of equipment combat test evaluation

The Bayesian network framework establishes a factor model for influencing equipment combat test evaluation, which solves the problem of insufficient universality and credibility of the credibility estimation method of equipment combat test evaluation in the existing technology, and realizes the evaluation of equipment combat tests Quantitative estimation of reliability and improve the credibility of the evaluation results.

CN112488514BActive Publication Date: 2024-11-08XINYUN LINGCHUANG (BEIJING) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011378027.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-30
Publication Date
2024-11-08
Estimated Expiration
2040-11-30

AI Technical Summary

Technical Problem

The universality, operability and credibility of the existing equipment combat trial evaluation credibility estimation methods are poor, and it is impossible to effectively describe the impact of complex nonlinear and random factors in equipment combat trial evaluation.

Method used

Using Bayesian network framework, we sort out the five key factors in the process, methods, personnel, data and indicators of equipment combat experiment evaluation, establish a conditional probability correlation model between influencing factors, and infer it through hard evidence or soft evidence. A probability distribution of the assessment credibility is derived.

Benefits of technology

It provides a quantitative assessment credibility estimation method, which conforms to the nature of nonlinearity and randomness, reduces the complexity of the model calculation and improves the credibility and universality of the assessment results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112488514B_ABST
    Figure CN112488514B_ABST
Patent Text Reader

Abstract

The present invention discloses a probabilistic method for estimating the credibility of equipment combat test evaluation, comprising the following steps: determining and classifying the influencing factors of equipment combat test evaluation credibility; establishing a conditional probability association model of the influencing factors, establishing a correlation network of the influencing factors, assigning conditional probability parameters, and assigning evidence values ​​to the influencing factor nodes in the influencing factor correlation network according to the actual implementation of the equipment combat test evaluation task; substituting the evidence values ​​into the conditional probability association model of the influencing factors to estimate the credibility of the evaluation. The present invention has sorted out in detail 20 main factors that can have a negative impact on the credibility of the evaluation from five main aspects, namely, the process, method, personnel, data and indicators of the equipment combat test evaluation, provided a fundamental basis for the quantitative estimation of the credibility of the evaluation, and hierarchically established a conditional probability association model of the credibility of the evaluation, which is in line with the nonlinear and random nature of the influencing relationship.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to a method for estimating the credibility of weapon equipment test evaluation, and in particular relates to a probabilistic method for estimating the credibility of equipment combat test evaluation. Background Art

[0002] Equipment combat test is an experimental activity to identify and evaluate weapons and equipment for actual combat by building realistic battlefield environment, confrontation conditions and combat process before the weapons and equipment are officially put into service. Equipment combat test evaluation is the ultimate goal of equipment combat test. Equipment combat test evaluation needs to quantitatively evaluate the combat effectiveness and combat adaptability of weapons and equipment under conditions close to actual combat conditions based on the data and information collected during the test, and test the matching of equipment with combat use requirements through scientific means such as statistical analysis or system modeling. The reason why equipment combat test evaluation is important is that it can provide decision-making basis for key activities such as combat operation, equipment development, and risk analysis. This requires that the evaluation results must have a considerable degree of credibility, that is, to be consistent with the actual level of combat capability of weapons and equipment, otherwise it may lead to wrong decisions. However, how to measure and evaluate credibility is an extremely complex task, because many subjective experience factors and objective constraints in the process of equipment combat test evaluation can affect the credibility of the evaluation. The main factors are test design, indicator selection, personnel level and data quality, and this influence relationship is usually nonlinear and random, which is difficult to describe with a deterministic explicit function relationship.

[0003] Compared with simple equipment combat test evaluation problems, the estimation of equipment combat test evaluation credibility has a higher level of abstraction, more difficult data acquisition, and stronger element correlation, which makes the classic evaluation methods basically inapplicable directly. At present, there is no mature formal solution for the typical meta-evaluation problem of equipment combat test evaluation credibility estimation. In most cases, it is still necessary to rely on domain experts to verify the evaluation results, or to use simple estimation methods such as comprehensive comparison of historical case data. Although the principle is simple, it cannot be described by a unified and strict mathematical model, resulting in poor versatility, operability and credibility. Summary of the invention

[0004] In order to solve the problems of poor versatility, operability and credibility of the existing estimation methods for the credibility of equipment combat test evaluation, the present invention provides a probabilistic method for estimating the credibility of equipment combat test evaluation, which sorts out the key factors that may have a negative impact on the credibility of the evaluation from five main aspects of the process, method, personnel, data and indicators of the equipment combat test evaluation, and establishes a conditional probability association model between these influencing factors and their credibility of the evaluation based on the implementation steps and process relationship of the evaluation activities under the framework of the Bayesian network. When used specifically, according to the actual situation of the evaluation task, the data can be substituted into the model in the form of hard evidence or soft evidence for inference to obtain the probability distribution of the credibility of the evaluation.

[0005] The present invention provides a probabilistic method for estimating the credibility of equipment combat test evaluation, comprising the following steps:

[0006] S1, determine and classify the factors affecting the credibility of equipment combat test assessment;

[0007] S2, establish the conditional probability association model of influencing factors;

[0008] S21, establish the correlation network of influencing factors;

[0009] S22, assignment conditional probability parameters;

[0010] S23, assigning evidence values ​​to the influencing factor nodes in the influencing factor association network based on the actual implementation of the equipment combat test evaluation task;

[0011] S3, substitute the evidence value into the conditional probability association model of the influencing factors to estimate the credibility of the assessment.

[0012] The step S1 specifically includes, according to all elements and the whole process of equipment combat test evaluation, from five aspects such as process, method, personnel, data and indicators, determining 20 key factors that have a negative impact on the credibility of the evaluation: 1. Whether the analysis of the purpose of the evaluation is sufficient; 2. Whether the analysis of the evaluation object is accurate; 3. Whether the evaluation subject is under full control; 4. Whether the evaluation results are verified; 5. Whether the classical method is adopted; 6. Whether mature tools are adopted; 7. Whether supervised samples are adopted; 8. Whether it is suitable for the characteristics of the evaluation data; 9. Whether it is suitable for the characteristics of the evaluation business; 10. Whether technical personnel in the business field participate; 11. Whether technical personnel in the data field participate; 12. Whether technical personnel in the evaluation field participate; 13. Test design quality; 14. Data collection process quality; 15. Whether data cleaning is performed; 16. Whether data quality assessment is performed; 17. Whether the indicators are complete; 18. Whether the indicators are independent; 19. Whether the indicators are consistent; 20. Whether the indicators are objective.

[0013] The above 20 key factors are classified, among which factors 1 to 4 belong to process factors, factors 5 to 9 belong to method factors, factors 10 to 12 belong to personnel factors, factors 13 to 16 belong to data factors, and factors 17 to 20 belong to indicator factors.

[0014] The step S2 comprises:

[0015] Step S21, establishing a correlation network of influencing factors;

[0016] The step S21, establishing an association network of influencing factors, specifically includes, using a hierarchical Bayesian network method, introducing 1 root node: evaluation credibility; then introducing 8 intermediate nodes: evaluation process credibility, evaluation method credibility, personnel technical level, primary data product quality, intermediate data product quality, final data product quality, evaluation indicator construction quality, and quantitative indicator credibility; according to the knowledge and experience of the business field, data field and evaluation field, establishing a probabilistic causal association network of 20 influencing factor nodes and 8 intermediate nodes to the root node, in which the personnel technical level factor is jointly determined by whether the business field technicians participate, whether the data field technicians participate, and whether the evaluation field technicians participate; the evaluation method credibility factor is jointly determined by whether the classical method is adopted, whether the mature tool is adopted, whether the supervised sample is adopted, whether it is adapted to the evaluation data characteristics, whether it is adapted to the evaluation business characteristics and the personnel technical level; the evaluation process The credibility of the process is jointly determined by five factors: whether the analysis of the evaluation purpose is sufficient, whether the analysis of the evaluation object is accurate, whether the evaluation subject has full control over the process, whether the evaluation results are verified, and the technical level of personnel. The quality factors of the construction of evaluation indicators are jointly determined by six factors: whether the indicators are complete, whether the indicators are independent, whether the indicators are consistent, whether the indicators are objective, the credibility of the evaluation process, and the technical level of personnel. The credibility of quantitative indicators is determined by two factors: the quality of the construction of evaluation indicators and the quality of the final data product. The quality of primary data products is jointly determined by two factors: the quality of experimental design and the quality of data collection process. The quality of intermediate data products is jointly determined by three factors: whether data cleaning is performed, whether data quality assessment is performed, and the quality of primary data products. The quality of the final data product is determined by two factors: the quality of intermediate data products and the technical level of personnel. The credibility of the final root node's evaluation is jointly determined by four factors: the technical level of personnel, the credibility of the evaluation process, the credibility of the evaluation method, and the credibility of the quantitative indicators.

[0017] Step S22, assigning conditional probability parameters;

[0018] The step S22, assigning conditional probability parameters, specifically includes: first, limiting the influencing factor association network established in step S21 to a Boolean network, and all node values ​​are True or False, respectively representing the occurrence or non-occurrence of the negative aspects of the corresponding influencing factor events; secondly, for any non-leaf node of the network, that is, 1 root node and 8 intermediate nodes, establish a conditional probability dependency relationship between the non-leaf node and all its corresponding parent nodes, that is, P (non-leaf node | all parent nodes), expressed in the form of a conditional probability expression or a Leaky Noisyor function, and assign values ​​to the conditional probability parameters by adopting the expert consultation method. When there are fewer parent nodes, the conditional probability expression is directly used to express; when there are more parent nodes, the Leaky Noisyor function noisyor() is used to express. The parent node of a certain node is the node at the upper level of the node. The specific expression of the Leaky Noisyor function noisyor() is:

[0019] Let X be any node in a Boolean Bayesian network, and its N parent nodes form the set n is the parent node number, is the nth parent node of X, let express All nodes in the set have the value False. express Remove Node Except for True, all other nodes are False. express All the node indices whose values ​​are True in , and then let:

[0020]

[0021] Represents all parent nodes of X The conditional probability that X takes the value True when both take the value False. Represents all parent nodes of X Remove Node The conditional probability that X takes the value True when all other nodes take the value False except for True; and let X and its parent node set The conditional probability expression is:

[0022]

[0023] Correspondingly, when there are many parent nodes, the expression of the conditional probability dependency relationship between non-leaf nodes and all parent nodes adopts the Leaky Noisyor function noisyor(), and its specific expression is:

[0024]

[0025] The function input is all parent nodes of X Value pair and p0.

[0026] The step S23 includes assigning evidence values ​​to 20 influencing factor nodes in the influencing factor association network according to the actual implementation of the equipment combat test evaluation task. The evidence value is the probability of the occurrence of the influencing factor, that is, the probability of the influencing factor node taking True. For any influencing factor node Y, its evidence is divided into hard evidence and soft evidence. The hard evidence takes the value of 1 or 0, that is, P(Y=True)=1 or P(Y=True)=0, which is used to indicate that the influencing factor event occurs or does not occur. The soft evidence takes the value of α∈(0,1), that is, 0<P(Y=True)=α<1, which is used to express the objective or subjective probability of the occurrence of the influencing factor event, or to express the proportion of the segment range of the influencing factor event in the overall space or time of the equipment combat test evaluation task. The evidence of all influencing factor nodes constitutes the network evidence set Evidence. Using Bayesian network modeling software, the conditional probability P(A=False|Evidence) of the network root node taking False when the network evidence set Evidence is known is obtained, where A represents the root node, as the estimated result of the credibility of the equipment combat test evaluation.

[0027] The beneficial effects of the present invention are as follows: first, the present invention has sorted out in detail 20 main factors that can have a negative impact on the credibility of the evaluation from five main aspects, namely, the process, method, personnel, data and index of the equipment combat test evaluation, and provided a fundamental basis for the quantitative estimation of the credibility of the evaluation; second, the present invention has hierarchically established a conditional probability association model of these influencing factors for the credibility of the evaluation under the framework of the Bayesian network, which conforms to the nonlinear and random nature of the influencing relationship; third, all nodes in the network model of the present invention are discrete binary types (True and False), which is convenient for users to assign values ​​to the nodes and reduces the calculation complexity of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the implementation steps of the method of the present invention.

[0029] Figure 2 It is a network diagram of the correlation relationship of the influencing factors of the present invention.

[0030] Figure 3 4 is a diagram of the credibility estimation result in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Figure 1 .

[0032] A probabilistic method for estimating the credibility of equipment combat test evaluation includes the following steps:

[0033] S1, determine and classify the factors affecting the credibility of equipment combat test assessment;

[0034] The step S1 specifically includes, according to all elements and the whole process of equipment combat test evaluation, from five aspects such as process, method, personnel, data and indicators, determining 20 key factors that have a negative impact on the credibility of the evaluation: 1. Whether the analysis of the purpose of the evaluation is sufficient; 2. Whether the analysis of the evaluation object is accurate; 3. Whether the evaluation subject is under full control; 4. Whether the evaluation results are verified; 5. Whether the classical method is adopted; 6. Whether mature tools are adopted; 7. Whether supervised samples are adopted; 8. Whether it is suitable for the characteristics of the evaluation data; 9. Whether it is suitable for the characteristics of the evaluation business; 10. Whether there are technical personnel in the business field involved; 11. Whether there are technical personnel in the data field involved; 12. Whether there are technical personnel in the evaluation field involved; 13. The quality of the test design; 14. The quality of the data collection process; 15. Whether data cleaning is performed; 16. Whether data quality assessment is performed; 17. Whether the indicators are complete; 18. Whether the indicators are independent; 19. Whether the indicators are consistent; 20. Whether the indicators are objective.

[0035] The above 20 key factors are classified into categories, among which factors 1 to 4 belong to process factors, factors 5 to 9 belong to method factors, factors 10 to 12 belong to personnel factors, factors 13 to 16 belong to data factors, and factors 17 to 20 belong to indicator factors. The specific meanings of these factors are shown in Table 1. The technical personnel in the business field here specifically refer to the technical personnel with operational and equipment business knowledge and experience.

[0036] Table 1 20 key factors that negatively affect the credibility of evaluations

[0037]

[0038] The analysis of the evaluation purpose is insufficient, which specifically refers to the failure to fully analyze the specific goals, results and focus of the evaluation activities.

[0039] The analysis of the evaluation object is inaccurate, and the specific object of the evaluation activity is not accurately analyzed.

[0040] The assessment subject did not manage the entire process, and the assessment subject did not control and guide the assessment activities throughout the entire process.

[0041] The unverified evaluation results are those that have not been verified by business field experts or evaluation entities.

[0042] The described method does not adopt the classical method, and does not use the classical evaluation method that has been tested in long-term practice and is complete and reliable.

[0043] The described method did not use mature tools, and the assessment method was applied without the help of mature open source or commercial software tools.

[0044] The described method does not adopt supervised samples, and does not use historical sample data containing business field expert knowledge, experience and consensus.

[0045] The aforementioned evaluation method is not suitable for the characteristics of the evaluation data, and the evaluation method cannot adapt to the characteristics of the indicator data, such as data volume, data integrity and data quality.

[0046] The above-mentioned inability to adapt to the business characteristics of the assessment means that the assessment method cannot adapt to the characteristics of the business field to which the assessment activity belongs, such as the characteristics of the assessment object, indicator data association, test assumption elements, etc.

[0047] There was no involvement of technical personnel in the business field, and technical personnel in the business field did not participate in key activities such as indicator system design, test plan design and evaluation result verification.

[0048] As stated above, there was no involvement of data field technical personnel, and data field technical personnel did not participate in key activities such as data collection plan design, data cleaning and data quality assessment, data review, and full-cycle management of indicator data.

[0049] There was no participation of technical personnel in the evaluation field, and technical personnel in the evaluation field did not participate in key activities such as evaluation process design, indicator system design and implementation of evaluation plan.

[0050] The quality of the test design is low. When designing specific operational tests for the evaluation objects, the evaluation objectives, indicator system, and data collection requirements were not strictly followed.

[0051] The data collection process described is of low quality, and no quality control is performed on key steps such as collection demand analysis, collection plan design, collection data source analysis, collection method analysis, specific collection implementation and collection data management.

[0052] The data cleaning was not performed, and the primary data products formed by data collection were not subjected to data standardization, data analysis, data deduplication, data enhancement and outlier processing.

[0053] No data quality assessment was conducted, and the intermediate data products formed by data cleaning were not checked for completeness, accuracy, consistency, redundancy, and timeliness.

[0054] The indicators mentioned are not complete, and the indicator system cannot comprehensively and comprehensively reflect the key characteristic elements of the evaluation object.

[0055] The indicators mentioned are not independent, and there is correlation between the indicators in practical or statistical sense.

[0056] The indicators described are not consistent, the indicator system is not strictly formulated around the evaluation purpose, and irrelevant factors exist.

[0057] The indicators described are not objective, the indicator system does not meet the requirements of authoritative sources, clear meanings, and quantitative measurability, and there are too many subjective factors and ambiguities.

[0058] The step S2 comprises:

[0059] Step S21, establishing a correlation network of influencing factors;

[0060] The step S21, establishing an association network of influencing factors, specifically includes, using a hierarchical Bayesian network method, introducing 1 root node: evaluation credibility; then introducing 8 intermediate nodes: evaluation process credibility, evaluation method credibility, personnel technical level, primary data product quality, intermediate data product quality, final data product quality, evaluation indicator construction quality, quantitative indicator credibility; according to the business field, data field and evaluation field knowledge and experience, establishing a probabilistic causal association network of 20 influencing factor nodes and 8 intermediate nodes to the root node, wherein the personnel technical level factor is jointly determined by whether there are business field technicians involved, whether there are data field technicians involved, and whether there are evaluation field technicians involved; the evaluation method credibility factor is jointly determined by whether the classical method is adopted, whether the mature tool is adopted, whether the supervised sample is adopted, whether it is adapted to the evaluation data characteristics, whether it is adapted to the evaluation business characteristics and the personnel technical level; the evaluation The credibility of the process is jointly determined by five factors: whether the analysis of the purpose of the assessment is sufficient, whether the analysis of the assessment object is accurate, whether the assessment subject has full control over the whole process, whether the assessment results are verified, and the technical level of the personnel. The quality factors of the construction of the assessment indicators are jointly determined by six factors: whether the indicators are complete, whether the indicators are independent, whether the indicators are consistent, whether the indicators are objective, the credibility of the assessment process, and the technical level of the personnel. The credibility of the quantitative indicators is determined by two factors: the quality of the construction of the assessment indicators and the quality of the final data product. The quality of the primary data product is jointly determined by two factors: the quality of the experimental design and the quality of the data collection process. The quality of the intermediate data product is jointly determined by three factors: whether data cleaning is performed, whether data quality assessment is performed, and the quality of the primary data product. The quality of the final data product is determined by two factors: the quality of the intermediate data product and the technical level of the personnel. The credibility of the final root node's assessment is jointly determined by four factors: the technical level of the personnel, the credibility of the assessment process, the credibility of the assessment method, and the credibility of the quantitative indicators. The network structure and node identification are as follows: Figure 2As shown, the oval nodes represent random variables (the dark nodes are leaf nodes corresponding to the 20 influencing factors), and the causal relationship between variables is represented by directed edges. The direction is from "cause" to "effect", indicating that the "cause" directly affects the "effect" or the "effect" depends on the "cause". Specifically, B depends on B1~B3, C depends on C1~C5 and B, D depends on D1~D4 and B, E1 depends on E2~E5, D and B, E depends on E1 and F6, F1 depends on F and F2, F4 depends on F3, F5 and F1, F6 depends on F4 and B, and finally, A depends on B, C, D and E.

[0061] Step S22, assigning conditional probability parameters;

[0062] The step S22, assigning conditional probability parameters, specifically includes: first, limiting the influencing factor association network established in step S21 to a Boolean network, and all node values ​​are True or False, respectively representing the occurrence or non-occurrence of the negative aspects of the corresponding influencing factor events; secondly, for any non-leaf node of the network, that is, 1 root node and 8 intermediate nodes, establish a conditional probability dependency relationship between the non-leaf node and all its corresponding parent nodes, that is, P (non-leaf node|all parent nodes), expressed in the form of a conditional probability expression or a Leaky Noisyor function, and assign values ​​to the conditional probability parameters by adopting the expert consultation method. When there are fewer parent nodes, the conditional probability expression is directly used to express; when there are more parent nodes, the Leaky Noisyor function noisyor() is used to express. The parent node of a certain node is the node at the upper level of the node. The specific expression of the Leaky Noisyor function noisyor() is:

[0063] Let X be any node in a Boolean Bayesian network, and its N parent nodes form the set n is the parent node number, is the nth parent node of X, let express All nodes in the set have the value False. express Remove Node Except for True, all other nodes are False. express All the node indices whose values ​​are True in , and then let:

[0064]

[0065] Represents all parent nodes of X The conditional probability that X takes the value True when both take the value False. Represents all parent nodes of X Remove Node The conditional probability that X takes the value True when all other nodes take the value False except for True; and let X and its parent node set The conditional probability expression is:

[0066]

[0067] Correspondingly, when there are many parent nodes, the expression of the conditional probability dependency relationship between non-leaf nodes and all parent nodes adopts the Leaky Noisyor function noisyor(), and its specific expression is:

[0068]

[0069] The function input is all parent nodes of X Value pair and p0.

[0070] In the embodiment, conditional probability parameters are assigned to all non-leaf nodes of the network. If the number of parent nodes is less than or equal to 2, the conditional probability formula is used, otherwise the Leaky Noisyor function is used.

[0071] For node A, we have:

[0072] noisyor(B,0.33,C,0.46,D,0.28,E,0.58,0.01)

[0073] For node B, we have:

[0074] noisyor(B1,0.4,B2,0.28,B3,0.35,0.05)

[0075] For node C, we have:

[0076] noisyor(C1,0.2,C2,0.25,C3,0.28,C4,0.33,C5,0.38,B,0.31,0.04)

[0077] For node D, we have:

[0078] noisyor(D1,0.41,D2,0.47,D3,0.33,D4,0.35,B,0.3,0.03)

[0079] For node E1, we have:

[0080] noisyor(E2,0.5,E3,0.26,E4,0.55,E5,0.37,B,0.32,0.05)

[0081] For node E, we have:

[0082]

[0083] For node F1, we have:

[0084]

[0085] For node F4, we have:

[0086] noisyor(F3,0.32,F5,0.23,F1,0.43,0.05)

[0087] For node F6, we have:

[0088]

[0089] The step S23 includes assigning evidence values ​​to 20 influencing factor nodes in the influencing factor association network according to the actual implementation of the equipment combat test evaluation task. The evidence value is the probability of the occurrence of the influencing factor (the node takes True), that is, the probability of the influencing factor node taking True. For any influencing factor node Y, the evidence is divided into hard evidence and soft evidence. The hard evidence takes the value of 1 or 0, that is, P(Y=True)=1 or P(Y=True)=0, which is used to indicate that the influencing factor event occurs or does not occur. The soft evidence takes the value of α∈(0,1), that is, 0<P(Y=True)=α<1, which is used to express the objective or subjective probability of the occurrence of the influencing factor event, or to express the proportion of the segment range of the influencing factor event in the overall space or time of the equipment combat test evaluation task. The evidence of all influencing factor nodes constitutes the network evidence set Evidence. Using the Bayesian network modeling software, the conditional probability P(A=False|Evidence) of the network root node taking False when the network evidence set Evidence is known is obtained, where A represents the root node, as the estimated result of the credibility of the equipment combat test evaluation.

[0090] In the embodiment, AgenaRisk software (available from https: / / www.agenarisk.com / ) is used to establish the conditional probability association model of the influencing factors of the present invention, and a credibility estimation analysis is performed based on the three groups of equipment combat test evaluation scenarios in Table 1. In Scenario 1, all influencing factors occur (hard evidence), in Scenario 2, all influencing factors do not occur (hard evidence), and Scenario 3 is generally consistent with Scenario 2. The difference is that Scenario 3 does not use classical methods and mature tools, and the quality of the data collection process is low (hard evidence). At the same time, technical personnel in the business field only participate in about 40% of the entire evaluation process, and 30% of the indicators are correlated (soft evidence). From the perspective of subjective experience, Scenario 1 and Scenario 2 correspond to the lowest and highest evaluation credibility, respectively, while Scenario 3 is between the two. Using the software for quantitative calculations, the evaluation credibility of the three groups of scenarios is shown in Table 1. Figure 3 , scenario 1, scenario 2, and scenario 3 are approximately 16.3%, 84.7%, and 54.9%, respectively. The calculation results not only conform to empirical intuition, but also provide specific quantitative values, which are more convenient for fine-grained analysis and comparison.

[0091] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A probabilistic method for estimating the credibility of equipment combat test assessment. It is characterized in that The steps include: S1, determine and classify the factors affecting the credibility of equipment combat test assessment; S2, establish the conditional probability association model of influencing factors; S21, establish the correlation network of influencing factors; S22, assignment conditional probability parameters; S23, assigning evidence values ​​to the influencing factor nodes in the influencing factor association network based on the actual implementation of the equipment combat test evaluation task; S3, substitute the evidence value into the conditional probability association model of the influencing factors to estimate the credibility of the assessment; The step S21, establishing an association network of influencing factors, specifically includes, using a hierarchical Bayesian network method, introducing 1 root node: evaluation credibility; then introducing 8 intermediate nodes: evaluation process credibility, evaluation method credibility, personnel technical level, primary data product quality, intermediate data product quality, final data product quality, evaluation indicator construction quality, and quantitative indicator credibility; according to the knowledge and experience of the business field, data field and evaluation field, establishing a probabilistic causal association network of 20 influencing factor nodes and 8 intermediate nodes to the root node, in which the personnel technical level factor is jointly determined by whether the business field technicians participate, whether the data field technicians participate, and whether the evaluation field technicians participate; the evaluation method credibility factor is jointly determined by whether the classical method is adopted, whether the mature tool is adopted, whether the supervised sample is adopted, whether it is adapted to the evaluation data characteristics, whether it is adapted to the evaluation business characteristics and the personnel technical level; the evaluation process The credibility of the process is jointly determined by five factors: whether the analysis of the evaluation purpose is sufficient, whether the analysis of the evaluation object is accurate, whether the evaluation subject has full control over the process, whether the evaluation results are verified, and the technical level of personnel. The quality factors of the construction of evaluation indicators are jointly determined by six factors: whether the indicators are complete, whether the indicators are independent, whether the indicators are consistent, whether the indicators are objective, the credibility of the evaluation process, and the technical level of personnel. The credibility of quantitative indicators is determined by two factors: the quality of the construction of evaluation indicators and the quality of the final data product. The quality of primary data products is jointly determined by two factors: the quality of experimental design and the quality of data collection process. The quality of intermediate data products is jointly determined by three factors: whether data cleaning is performed, whether data quality assessment is performed, and the quality of primary data products. The quality of the final data product is determined by two factors: the quality of intermediate data products and the technical level of personnel. The credibility of the final root node's evaluation is jointly determined by four factors: the technical level of personnel, the credibility of the evaluation process, the credibility of the evaluation method, and the credibility of the quantitative indicators. The step S22, assigning conditional probability parameters, specifically includes: first, limiting the influencing factor association network established in step S21 to a Boolean network, and all node values ​​are True or False, respectively representing the occurrence or non-occurrence of the negative aspects of the corresponding influencing factor events; secondly, for any non-leaf node of the network, that is, 1 root node and 8 intermediate nodes, establish a conditional probability dependency relationship between the non-leaf node and all its corresponding parent nodes, that is, P (non-leaf node|all parent nodes), express it in the form of a conditional probability expression or a Leaky Noisyor function, and assign values ​​to the conditional probability parameters by adopting an expert consultation method; when there are fewer parent nodes, directly use the conditional probability expression to express it; when there are more parent nodes, use the Leaky Noisyor function noisyor() to express it; the parent node corresponding to the non-leaf node is the upper-level node of the non-leaf node; the specific expression of the Leaky Noisyor function noisyor() is: Let X be any node in a Boolean Bayesian network, and its N parent nodes form the set n is the parent node number, is the nth parent node of X, let express All nodes in the set have the value False. express Remove Node Except for True, all other nodes are False. express All the node indices whose values ​​are True in , and then let: Represents all parent nodes of X The conditional probability that X takes the value True when both take the value False. Represents all parent nodes of X Remove Node The conditional probability that X takes the value True when all other nodes take the value False except True; when the number of parent nodes is less than or equal to 2, let X and its parent node set The conditional probability expression is: Correspondingly, when the number of parent nodes is greater than 2, the expression of the conditional probability dependency relationship between non-leaf nodes and all parent nodes adopts the Leaky Noisyor function noisyor(), and its specific expression is: The function input is all parent nodes of X Value pairs and p 0 .

2. A probabilistic method for estimating the credibility of equipment combat test evaluation as claimed in claim 1, It is characterized in that The step S1 specifically includes: According to the full elements and process of equipment combat test evaluation, 20 key factors that have a negative impact on the credibility of the evaluation are identified from five aspects: process, method, personnel, data and indicators:

1. Whether the analysis of the purpose of the evaluation is sufficient; 2. Whether the analysis of the assessment object is accurate; 3. Whether the assessment subject is under full control; 4. Whether the assessment results are verified; 5. Whether the classical method is used; 6. Whether mature tools are used; 7. Whether supervised samples are used; 8. Whether it is suitable for the characteristics of the evaluation data; 9. Whether it is suitable for the characteristics of the evaluation business; 10. Whether the technical personnel in the business field are involved; 11. Whether the technical personnel in the data field are involved; 12. Whether the technical personnel in the evaluation field are involved; 13. The quality of the experimental design; 14. The quality of the data collection process; 15. Whether data cleaning is performed; 16. Whether data quality assessment is performed; 17. Whether the indicators are complete; 18. Whether the indicators are independent; 19. Whether the indicators are consistent; 20. Whether the indicators are objective; The above 20 key factors are classified, among which factors 1 to 4 belong to process factors, factors 5 to 9 belong to method factors, factors 10 to 12 belong to personnel factors, factors 13 to 16 belong to data factors, and factors 17 to 20 belong to indicator factors.

3. A probabilistic method for estimating the credibility of equipment combat test evaluation as claimed in claim 1, It is characterized in that The step S23 includes assigning evidence values ​​to 20 influencing factor nodes in the influencing factor association network according to the actual implementation of the equipment combat test evaluation task, and the evidence value is the probability of the occurrence of the influencing factor, that is, the probability of the influencing factor node taking True; for any influencing factor node Y, its evidence is divided into hard evidence and soft evidence, the hard evidence takes a value of 1 or 0, that is, P(Y=True)=1 or P(Y=True)=0, which is used to indicate that the influencing factor event occurs or does not occur, and the soft evidence takes a value of α∈(0,1), that is, 0<P(Y=True)=α<1, which is used to express the objective or subjective probability of the occurrence of the influencing factor event, or to express the proportion of the segment range of the occurrence of the influencing factor event in the overall space or time of the equipment combat test evaluation task; the evidence of all influencing factor nodes constitutes a network evidence set Evidence; using Bayesian network modeling software, the conditional probability P(A=False|Evidence) of the network root node taking False when the network evidence set Evidence is known is calculated, where A represents the root node, as the estimation result of the credibility of the equipment combat test evaluation.