Method for improving reliability of human decision

Abstract

The invention discloses a method for improving the reliability of human decisions. The method comprises the following steps: 1) acquiring nine kinds of unmanned system control general performance condition data through a master control system; 2) respectively calculating membership degrees, human decision reliability score values and human factor error probability values of nine unmanned system control general performance conditions; 3) inputting the membership degrees of the nine unmanned system control general performance conditions, the human decision reliability score value and the human factor error probability value into a human decision reliability prediction model, wherein the human decision reliability prediction model outputs the error type and the reliability score of the human decision; 4) judging whether the error type of the human decision occurs according to the reliability score; if the reliability score is greater than or equal to 50%, the prediction result is that the occurrence possibility of the error type of the human decision is large, and an alarm is given to prompt an operator; and if the reliability score is less than 50%, the occurrence possibility of the error type of the human decision is low, and additional operations such as alarm and the like are not carried out.

Description

Technical Field

[0001] The present invention relates to the field of decision reliability, and in particular to a method for improving the reliability of human decision making. Background Art

[0002] Intelligence, autonomy and unmanned operation are the inevitable trends in the development of ground intelligent equipment in the future. However, due to the complex environment, changing situations, diverse tasks and frequent emergencies, ground intelligent equipment still needs to be supplemented by certain human decisions to solve specific problems. Therefore, the systems of most ground intelligent equipment are designed to be able to deploy and navigate autonomously under human supervision, and these systems ensure that ground intelligent equipment has a certain ability to make decisions autonomously and complete tasks independently. With the development of technology, at present, some ground intelligent equipment can already complete tasks independently without human command, but based on safety and other issues, ground intelligent equipment is still equipped with corresponding operators for supervision, operation and decision-making. Considering that some ground intelligent equipment can already complete tasks independently without human command, an operator will supervise, operate and make decisions on multiple ground intelligent equipment at the same time, which makes the operator's human decision-making and avoidance of operator human decision-making errors more and more important.

[0003] Since the decision safety of ground intelligent equipment is mainly affected by the reliability of equipment functions and the human factors of operators. In the research and development process of ground intelligent equipment, the functional reliability and safety of ground intelligent equipment will be fully discussed and studied, but the reliability of operators is less considered, especially in the environment of performing tasks. Once the operator issues an incorrect decision command, it will lead to serious adverse consequences. Therefore, there is an urgent need for a method to improve the reliability of operator decision making. Summary of the invention

[0004] In response to the above-mentioned problems, the present invention provides a method for improving the reliability of human decision-making. The method can predict in real time the wrong decisions that the operator may make in a scenario and the probability of making the wrong decision based on the environment, capabilities, performance and other information of the ground intelligent device, prompt the operator, and prevent the operator from making the wrong decision, thereby greatly improving the reliability of the operator's decision-making.

[0005] The technical solution of the present invention is: a method for improving the reliability of human decision-making, comprising the following steps:

[0006] 1) Obtain 9 types of general performance condition data of unmanned system control through the master control system;

[0007] 2) Calculate the membership degree, human decision reliability score and human error probability of the nine general performance conditions for unmanned system control based on the nine general performance conditions for unmanned system control;

[0008] 3) Inputting the membership degree of 9 general performance conditions of unmanned system control, the human decision reliability score value, and the human error probability value into a human decision reliability prediction model, the human decision reliability prediction model outputs the error type and reliability score of human decision;

[0009] 4) Determine the probability of the error type of the human decision-making based on the reliability score;

[0010] If the reliability score is greater than or equal to 50%, the probability of the error type in human decision-making is high, and an alarm is issued to prompt the operator; if the reliability score is less than 50%, the probability of the error type in human decision-making is low, and no alarm is issued.

[0011] Preferably, the steps of establishing the human decision reliability prediction model are:

[0012] ① Obtain a historical human decision data set through the master control system, wherein the historical human decision data set includes 9 general performance conditions for unmanned system control, error types of human decision and reliability scores, wherein the 9 general performance conditions for unmanned system control are observation data, and the error types and reliability scores of human decision are labeled data, and the historical human decision data set is divided into a training set and a test set, wherein both the training set and the test set include observation data and labeled data;

[0013] ② Based on the NSGA2 algorithm and fuzzy logic algorithm, the evaluation indicators are established for the membership of 9 common performance conditions of unmanned system control;

[0014] ③ The 9 general performance condition data of unmanned system control in the training set are used to calculate the membership degree of the corresponding general performance condition of unmanned system control according to the evaluation indicators, and the human decision reliability score and human error probability value are calculated;

[0015] ④ Establish a human decision reliability prediction model, and use the membership degree of 9 common performance conditions of unmanned system control, human decision reliability score value, and human error probability value calculated through the observation data of the training set as input. The human decision reliability prediction model calculates and outputs the error type and reliability score of human decision;

[0016] ⑤ Compare the error types and reliability scores of human decisions calculated and output by the human decision reliability prediction model with the error types and reliability scores of human decisions in the corresponding labeled data in the training set, and calculate the first objective function value and the second objective function value of the model;

[0017] ⑥ Save the evaluation index of each membership, the calculated first objective function value, and the second objective function value, and re-establish the evaluation index for the membership of the 9 general performance conditions of unmanned system control according to the NSGA2 algorithm and the fuzzy logic algorithm, repeat steps ③ to ⑥, and sort the results of the first objective function value and the second objective function value calculated by different models until the maximum number of iterations is reached or the model converges, and then enter step ⑦;

[0018] ⑦ Use the test set to evaluate the performance of the trained human decision reliability prediction model.

[0019] Furthermore, the evaluation indicators are piecewise functions, and each evaluation indicator respectively establishes input variable thresholds of multiple piecewise functions.

[0020] Preferably, the nine general performance conditions for unmanned system control are respectively the number of controlled equipment, effective mission distance, situational awareness capability, environmental complexity, number of enemies and unit density, equipment mobility status, hardware and equipment system status, communication status, and mission pressure.

[0021] Preferably, the calculation formula of the human decision reliability score is:

[0022] HPRS=∑C 1 -∑C 2

[0023] In the formula, HPRS is the human decision reliability score, C 1 is the expected impact of 9 common performance conditions of unmanned system control on reliability, and is the number of improvements, C 2 The expected impact of common performance conditions for 9 unmanned system operations on reliability is the number of reductions.

[0024] Preferably, the calculation formula of the human error probability value is:

[0025]

[0026] Where HEP is the probability of human error, n is the number of errors in human decisions, and N is the total number of human decisions.

[0027] The advantages of the present invention are as follows: the present invention inputs the degree of membership of 9 general performance conditions for unmanned system control, the human decision reliability score value, and the human error probability value into a human decision reliability prediction model in real time, and outputs the error type and reliability score of human decision through the human decision reliability prediction model, predicts the type of wrong decision that the operator may make and the probability of making the error type in this scenario, reminds the operator and prevents the operator from making the wrong decision, thereby avoiding human decision errors that may occur in the process of ground intelligent equipment performing tasks and greatly improving the reliability of human decisions.

[0028] Glossary

[0029] Number of controlled equipment: refers to the number of one's own ground intelligent equipment that a single commander needs to control and supervise at the same time.

[0030] Effective mission distance: refers to the current distance between the mission destination and the ground intelligent equipment during the mission execution process.

[0031] Situational awareness capability: refers to the ability of ground intelligent equipment to scan the environment within its field of view based on sensor information during the execution of tasks, generate terrain and topography using image stitching technology, effectively perceive the environment, build maps and perform real-time positioning, mark target locations and types, and comprehensively generate situation maps within the effective perception range of the equipment, predict the target's intentions, determine the target's threat level, and conduct real-time situational awareness and situation assessment of the environment.

[0032] Environmental complexity: refers to the known fixed obstacles, dynamic obstacles and situational environment in the environment.

[0033] Enemy number and unit density: Enemy number refers to the total number of enemies; unit density refers to the average number of enemy units within a one-kilometer radius.

[0034] Equipment maneuvering status: refers to the speed, lateral and longitudinal acceleration, and collision time (TTC) of ground intelligent equipment.

[0035] Hardware and equipment system status: refers to the hardware and functional status of intelligent equipment that affects its ability to complete tasks collaboratively.

[0036] Communication status: refers to the ability to transmit data and information between ground intelligent equipment and ground intelligent equipment, and between ground intelligent equipment and humans.

[0037] Task pressure: refers to the time required to complete a task and the difficulty of completing the task. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the present invention;

[0039] Figure 2 The membership evaluation index diagram of the control equipment number of the present invention having a large membership;

[0040] Figure 3 This is a flow chart of model training of the present invention;

[0041] Figure 4 This is a flow chart of the model evaluation of the present invention. DETAILED DESCRIPTION

[0042] See also Figure 1, a method for improving the reliability of human decision making, comprising the following steps:

[0043] 1) 9 kinds of general performance condition data of unmanned system control are obtained through the master control system; the 9 kinds of general performance conditions of unmanned system control are respectively the number of controlled equipment, effective mission distance, situational awareness capability, environmental complexity, number of enemies and unit density, equipment mobility status, hardware and equipment system status, communication status, and mission pressure. These 9 kinds of general performance conditions of unmanned system control are established by analyzing the cognitive function and possible error types of operators through cognitive reliability and error analysis method (cream method).

[0044] 2) Calculate the membership degree, human decision reliability score value, and human error probability value of the 9 general performance conditions for unmanned system control according to the 9 general performance conditions for unmanned system control respectively; the calculation formula for the human decision reliability score value is:

[0045] HPRS=∑C 1 -∑C 2

[0046] In the formula, HPRS is the human decision reliability score, C 1 is the expected impact of 9 common performance conditions of unmanned system control on reliability, and is the number of improvements, C 2 The expected impact of common performance conditions for 9 unmanned system operations on reliability is the number of reductions.

[0047] The calculation formula of the human error probability value is:

[0048]

[0049] Where HEP is the probability of human error, n is the number of errors in human decisions, and N is the total number of human decisions.

[0050] 3) The membership degree of the nine general performance conditions for unmanned system control, the human decision reliability score value, and the human error probability value are input into a human decision reliability prediction model, and the human decision reliability prediction model outputs the error type and reliability score of human decision; the steps for establishing the human decision reliability prediction model are:

[0051] ① Obtain a historical human decision-making data set through the master control system, wherein the historical human decision-making data set includes 9 general performance conditions for unmanned system control, error types of human decision-making, and reliability scores, wherein the 9 general performance conditions for unmanned system control are observation data, and the error types and reliability scores of human decision-making are labeled data. The reliability scores in the labeled data are provided by expert scoring, and the historical human decision-making data set is divided into a training set and a test set, wherein both the training set and the test set include observation data and labeled data;

[0052] ② According to the NSGA2 algorithm and the fuzzy logic algorithm, the evaluation indicators are respectively established for the membership of the 9 general performance conditions for unmanned system control; the evaluation indicators are piecewise functions, and each evaluation indicator respectively establishes multiple input variable thresholds of the piecewise functions. The NSGA2 algorithm will randomly generate a set of parameters, and assign this set of randomly generated parameters to each input variable threshold for initialization. In this embodiment, taking the establishment of the membership evaluation indicator of the number of control equipment as an example, see Table 1. According to the fuzzy control algorithm, the number of control equipment is first fuzzified into four memberships of many, medium, few, and none, and then the four memberships are respectively used to establish evaluation indicators for each membership of the number of control equipment. In this embodiment, see Figure 2 , establish the membership evaluation index with the majority of the number of control equipment membership, and take the number of control equipment x as the input variable, a 1 、a 2 、a 3 、a 4 Threshold value of input variable as piecewise function:

[0053]

[0054] According to the membership evaluation index of the control equipment number, if the control equipment number is less than a 1 or greater than or equal to a 4 , it means that the degree of membership of the number of control equipment is 0. If the number of control equipment is greater than or equal to a 1 and less than a 2 , then the degree of control of the number of equipment is more (xa 1 ) / (a 2 -a 1 ), if the number of controlled equipment is greater than or equal to a 2 and less than a 3 , it means that the degree of membership of the number of control equipment is 100%, if the number of control equipment is greater than or equal to a 3 and less than a 4 , then the degree of membership of the number of control equipment is more (a 4 -x) / (a 4 -a 3 ).

[0055] Similarly, we can establish evaluation indicators for the control equipment number with medium, few, and none membership, and then we can use this method to establish the membership evaluation indicators of the other 8 general performance conditions for unmanned system control. It is worth noting that the input variable thresholds of the membership evaluation indicators of the general performance conditions for unmanned system control are all four. For example, the threshold of a in the membership evaluation indicator for the control equipment number with a high membership is 1 、a 2、a 3 、a 4 The input variable thresholds of the four piecewise functions are different for each evaluation indicator and need to be determined through repeated iterative optimization during the model training process.

[0056] Table 1

[0057]

[0058]

[0059] ③ The 9 general performance condition data of unmanned system control in the training set are used to calculate the membership degree of the corresponding general performance condition of unmanned system control according to the evaluation indicators, and the human decision reliability score and human error probability value are calculated;

[0060] ④ Establish a human decision reliability prediction model, and use the membership of 9 common performance conditions for unmanned system control calculated through the observation data of the training set, the human decision reliability score value, and the human error probability value as input. The human decision reliability prediction model calculates and outputs the error type and reliability score of human decision; in this embodiment, the error types of human decision include key error, equipment abnormality error, device identification capability mismatch error and other error types.

[0061] ⑤ Compare the error types and reliability scores of human decisions calculated and output by the human decision reliability prediction model with the error types and reliability scores of human decisions in the corresponding labeled data in the training set, and calculate the first objective function value and the second objective function value of the model; the calculation formula of the first objective function value is:

[0062] f 1 (x) = (1-ACC) + (1-DR) + FAR

[0063] In the formula, f 1 (x) is the first objective function value, ACC is the accuracy, DR is the detection rate, and FAR is the false alarm rate.

[0064] The accuracy rate ACC, detection rate DR, and false alarm rate FAR are all calculated through the confusion matrix. In this embodiment, taking the category of "key error" as an example, its confusion matrix is ​​shown in the following figure:

[0065]

[0066] In the figure, true positive (TP) indicates the number of events when the category of the model calculation output is "key error" (i.e., positive value), and the actual error category of the data in the training set is also "key error" (i.e., positive value); false positive (FP) indicates the number of events when the category of the model calculation output is "key error" (i.e., positive value), but the actual error category of the data in the training set is not "key error" (i.e., negative value); true negative (TN) indicates the number of events when the category of the model calculation output is not "key error" (i.e., negative value), and the actual error category of the data in the training set is not "key error" (i.e., negative value); false negative (FN) indicates the number of events when the category of the model calculation output is not "key error" (i.e., negative value), but the actual error category of the data in the training set is "key error" (i.e., positive value).

[0067] According to the confusion matrix, the values ​​of ACC, DR and FAR can be calculated. The calculation formula is as follows:

[0068]

[0069] The calculation formula of the second objective function is:

[0070] f 2 (x) = score 1 -score 2

[0071] In the formula, f 2 (x) is the second objective function, score 1 The reliability score given by the experts in the training set, score 2 Calculate the reliability score of the output for the human decision reliability prediction model.

[0072] ⑥Save the evaluation index of each membership, the calculated first objective function value, and the second objective function value, and re-establish the evaluation index for the membership of the 9 general performance conditions of unmanned system control according to the NSGA2 algorithm and the fuzzy logic algorithm, repeat steps ③ to ⑥, and sort the first objective function value and the second objective function value results calculated by different models until the model reaches the maximum number of iterations or the model converges, and enter step ⑦; the sorting of the first objective function value and the second objective function value results means that the closer the first objective function value and the second objective function value are to 0, the better the effect is and the higher the ranking is.

[0073] ⑦ Use the test set to evaluate the performance of the trained human decision reliability prediction model. Figure 4The performance evaluation steps of the human decision reliability prediction model are to calculate the membership degree, human decision reliability score value, and human error probability value of the 9 general performance conditions of unmanned system control in the test set and input them into the trained human decision reliability prediction model, and compare the human decision error type and reliability score output by the trained human decision reliability prediction model with the human decision error type and reliability score in the corresponding labeled data in the test set, and calculate the model's accuracy ACC, detection rate DR, false alarm rate FAR, first objective function value, and second objective function value; if the accuracy ACC ≥ 90% and the detection rate DR ≥ 80% and the false alarm rate FAR ≤ 10% and the second objective function value f 2 If (x)≤10, the model performance meets the standard and the model is the final model used for practical applications. Otherwise, the model is retrained and the performance of the trained model is re-evaluated until the performance meets the standard, thus completing the establishment of the human decision reliability prediction model.

[0074] 4) Determine the probability of the error type of the human decision-making based on the reliability score;

[0075] If the reliability score is greater than or equal to 50%, the probability of the error type in human decision-making is high, and an alarm is issued to prompt the operator; if the reliability score is less than 50%, the probability of the error type in human decision-making is low, and no alarm is issued.

Claims

1. A method for improving the reliability of human decision making, characterized in that The following steps are involved: 1) Obtain 9 types of general performance condition data of unmanned system control through the master control system; 2) Calculate the membership degree, human decision reliability score and human error probability of the nine general performance conditions for unmanned system control based on the nine general performance conditions for unmanned system control; 3) inputting the membership degree of 9 general performance conditions of unmanned system control, the human decision reliability score value, and the human error probability value into a human decision reliability prediction model, and the human decision reliability prediction model outputs the error type and reliability score of human decision; 4) Determine the probability of the error type of the human decision-making based on the reliability score; If the reliability score is greater than or equal to 50%, the probability of the error type in human decision-making is high, and an alarm is issued to prompt the operator; if the reliability score is less than 50%, the probability of the error type in human decision-making is low, and no alarm is issued.

2. A method for improving the reliability of human decision making according to claim 1, characterized in that: The steps for establishing the human decision reliability prediction model are: ① Obtain a historical human decision data set through the master control system, wherein the historical human decision data set includes 9 general performance conditions for unmanned system control, error types of human decision and reliability scores, wherein the 9 general performance conditions for unmanned system control are observation data, and the error types and reliability scores of human decision are labeled data, and the historical human decision data set is divided into a training set and a test set, wherein both the training set and the test set include observation data and labeled data; ② Based on the NSGA2 algorithm and fuzzy logic algorithm, the evaluation indicators are established for the membership of 9 common performance conditions of unmanned system control; ③ The 9 general performance condition data of unmanned system control in the training set are used to calculate the membership degree of the corresponding general performance condition of unmanned system control according to the evaluation indicators, and the human decision reliability score and human error probability value are calculated; ④ Establish a human decision reliability prediction model, and use the membership degree of 9 common performance conditions of unmanned system control, human decision reliability score value, and human error probability value calculated through the observation data of the training set as input. The human decision reliability prediction model calculates and outputs the error type and reliability score of human decision; ⑤ Compare the error types and reliability scores of human decisions calculated and output by the human decision reliability prediction model with the error types and reliability scores of human decisions in the corresponding labeled data in the training set, and calculate the first objective function value and the second objective function value of the model; ⑥ Save the evaluation index of each membership, the calculated first objective function value, and the second objective function value, and re-establish the evaluation index for the membership of the 9 general performance conditions of unmanned system control according to the NSGA2 algorithm and the fuzzy logic algorithm, repeat steps ③ to ⑥, and sort the results of the first objective function value and the second objective function value calculated by different models until the maximum number of iterations is reached or the model converges, and then enter step ⑦; ⑦ Use the test set to evaluate the performance of the trained human decision reliability prediction model.

3. A method for improving the reliability of human decision making according to claim 2, characterized in that: The evaluation indicators are piecewise functions, and each evaluation indicator respectively sets a plurality of input variable thresholds of the piecewise functions.

4. The method for improving the reliability of human decision making according to claim 1, characterized in that: The nine general performance conditions for unmanned system control are the number of controlled equipment, effective mission distance, situational awareness capability, environmental complexity, number of enemies and unit density, equipment mobility status, hardware and equipment system status, communication status, and mission pressure.

5. The method for improving the reliability of human decision making according to claim 1, characterized in that: The calculation formula of the human decision reliability score value is: HPRS=∑C1-∑C2 Where HPRS is the human decision reliability score, C1 is the number of expected improvements in the reliability of the nine common performance conditions for unmanned system control, and C2 is the number of expected reductions in the reliability of the nine common performance conditions for unmanned system control.

6. The method for improving the reliability of human decision making according to claim 1, characterized in that: The calculation formula of the human error probability value is: Where HEP is the probability of human error, n is the number of errors in human decisions, and N is the total number of human decisions.

Images