Modeling method for relationship between man-machine collaborative decision evaluation indexes

Abstract

The invention discloses a modeling method for a relationship between man-machine collaborative decision evaluation indexes. The method comprises the following steps: establishing a man-machine collaborative decision model; establishing a man-machine collaborative decision evaluation index system; carrying out a man-machine collaborative decision-making human factor experiment, and obtaining experiment data of each index; calculating the score of each man-machine collaborative decision evaluation index, and carrying out index correlation analysis to obtain a correlation coefficient between every two indexes; selecting strongly correlated man-machine collaborative decision evaluation indexes, and establishing a relation function among the evaluation indexes by using a multiple linear regression analysis method; and evaluating the accuracy of the relation function between the evaluation indexes by calculating an average absolute percentage error between a predicted value and an actual value of the relation function. According to the method, the quantitative relation between the man-machine collaborative decision evaluation indexes is determined, a complete man-machine collaborative decision evaluation index system is established and simplified, the quantitative evaluation problem of man-machine decision is effectively solved, and a quantitative standard is provided for improving the accuracy of man-machine decision.

Description

Technical Field

[0001] This invention relates to the field of human-machine collaboration, and in particular to a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI is widely applied in decision-making scenarios for complex problems. Leveraging its superior computing and data analysis capabilities, it solves problems such as massive data mining and pattern recognition that are difficult to handle with traditional methods, effectively improving the efficiency and accuracy of decision-making. However, AI decision-making is also constrained by many factors. Due to a lack of creative thinking and flexible adaptability, it often struggles to fully consider various potential factors when facing complex and uncertain decision-making situations, leading to incomplete, unreasonable, or even biased decision results, failing to consistently and efficiently achieve the expected decision goals. Therefore, it is necessary to fully combine the unique advantages of humans in creative thinking, flexible adaptability, accumulated experience, and intuitive judgment with AI to complement each other, thereby optimizing decision-making effects and making the results more scientific, reasonable, and comprehensive, capable of solving problems stably and efficiently. Human-machine collaborative decision-making is becoming a hot and key area of ​​focus now and in the future.

[0003] Existing research on human-machine collaborative decision-making mainly employs methods such as performance evaluation and influencing factor analysis. Performance evaluation typically includes indicators such as decision time, number of decisions, and accuracy. These indicators often represent objective task performance, neglecting subjective human perception and lacking comprehensiveness. Influencing factor analysis studies how factors such as humans, AI, tasks, and human-machine relationships affect the outcome of human-machine collaborative decision-making. It analyzes the impact of influencing factors on performance indicators, but often lacks research on the relationships between indicators, failing to determine the interaction mechanisms between indicators and obscuring the chain reaction between indicators. This leads to an overly complex approach of sequentially studying the impact of influencing factors on each performance indicator. Summary of the Invention

[0004] This invention provides a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators, which solves the technical problems that existing evaluation indicators are not comprehensive enough and are too complicated.

[0005] This invention provides a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators, comprising:

[0006] A method for modeling the relationship between evaluation indicators in human-machine collaborative decision-making, characterized by the following steps:

[0007] Establish a human-machine collaborative decision-making model, covering three human-machine collaborative decision-making methods;

[0008] Based on the aforementioned human-machine collaborative decision-making model, an evaluation index system for human-machine collaborative decision-making is established.

[0009] Conduct several human factors experiments on human-machine collaborative decision-making to obtain experimental data for each indicator in the human-machine collaborative decision-making evaluation index system;

[0010] Based on the experimental data, the score of each human-machine collaborative decision-making evaluation index was calculated, and the correlation analysis of the indexes was performed to obtain the correlation coefficient between each pair of indexes.

[0011] Based on the aforementioned correlation coefficients, highly correlated human-machine collaborative decision-making evaluation indicators are selected, and a relationship function is established using multiple linear regression analysis.

[0012] Based on the relationship function, the average absolute percentage error between the predicted and actual values ​​of the evaluation index is calculated, and the accuracy of the relationship function is evaluated.

[0013] Furthermore, the aforementioned human-machine collaborative decision-making model encompasses three human-machine collaborative decision-making methods, including:

[0014] Establish a human-machine collaborative decision-making model:

[0015]

[0016] The human-machine collaborative decision-making model considers three human-machine collaborative decision-making methods: human decision-making, AI decision-making, and human-machine collaborative decision-making. As a human decision-making method, For AI decision-making methods, In human condition, In AI state, The difficulty level of the experimental task. These represent the weights for human decision-making and AI decision-making, respectively.

[0017] Furthermore, based on the aforementioned human-machine collaborative decision-making model, the establishment of a human-machine collaborative decision-making evaluation index system includes:

[0018] Establish a human-machine collaborative decision-making evaluation index system, including three categories: objective performance indicators, human subjective perception indicators, and human-machine interaction indicators;

[0019] Establish objective performance indicators, including task completion efficiency. ;

[0020] Establish indicators of people's subjective feelings, including workload. Situational awareness capability Satisfaction ;

[0021] Establish human-computer interaction metrics, including the fluency of human-computer interaction. Human-machine trust level .

[0022] Furthermore, the aforementioned process of conducting several human-machine collaborative decision-making human factor experiments to obtain experimental data for each indicator in the human-machine collaborative decision-making evaluation index system includes:

[0023] Regarding task completion efficiency indicators Several human-machine collaborative decision-making human factors experiments were conducted, and the completion time of each experiment was recorded. And stored in the original timing dataset. middle;

[0024] Regarding workload indicators The experimental dataset was obtained using the NASA-TLX scale. and ;

[0025] Regarding situational awareness capability indicators Experimental datasets were obtained using the SART questionnaire. ;

[0026] Regarding satisfaction indicators Design questionnaires, set up Sub-problems and obtaining experimental datasets. ;

[0027] For the human-computer interaction fluency index Design questionnaires, set up Sub-problems and obtaining experimental datasets. ;

[0028] Regarding human-machine trust index Design questionnaires, set up Sub-problems and obtaining experimental datasets. .

[0029] Furthermore, based on the experimental data, the score for each human-machine collaborative decision-making evaluation indicator is calculated, and correlation analysis is performed to obtain the correlation coefficient between any two indicators, including:

[0030] Calculate the score for the task completion efficiency index by analyzing the experiment completion time. Perform normalization and reverse processing:

[0031]

[0032] in, The score represents the efficiency of task completion. For the original timing dataset The minimum value in, For the original timing dataset The maximum value in, These represent the upper and lower limits of the normalized time range. ;

[0033] Workload, situational awareness, satisfaction, human-computer interaction fluency, and human-computer trust were measured using experimental data obtained through questionnaires. The score range for each question in the questionnaire was set to [a, b].

[0034] Calculate the score for the workload index:

[0035]

[0036] in, The NASA-TLX scale, which measures workload as a score, has six dimensions. These are the experimental data values ​​for a single dimension, representing the score and weight for that dimension, respectively. ;

[0037] The scores for the workload index are processed in reverse:

[0038]

[0039] in, The score for the workload index after reverse processing;

[0040] Calculate the score for the situational awareness capability index:

[0041]

[0042] in, It is a score of situational awareness capability, and the SART questionnaire measures it in three dimensions. The experimental data values ​​represent three dimensions: the demand for attentional resources, the supply of attentional resources, and the level of understanding of the experimental context. ;

[0043] Normalize the situational awareness capability indicators:

[0044]

[0045] in, It is the score of the normalized situational awareness capability index;

[0046] Calculate the score for the satisfaction index:

[0047]

[0048] in, It is the score of the satisfaction index. The number of sub-questions in the questionnaire. , represents the experimental data value of a single subproblem;

[0049] The formula for calculating the score of the human-computer interaction fluency index is:

[0050]

[0051] in, It is a score for the fluency of human-computer interaction. The number of sub-questions in the questionnaire. , represents the experimental data value of a single subproblem;

[0052] The formula for calculating the human-machine trust index score is:

[0053]

[0054] in, It is a score for the human-machine trust index. The number of sub-questions in the questionnaire. , representing the experimental data value of a single subproblem;

[0055] Correlation analysis was performed on the six processed indicators by calculating the Pearson correlation coefficient between each pair of indicators. The larger the correlation coefficient, the better the correlation. The formula for the Pearson correlation coefficient is:

[0056]

[0057] in, , These are the scores of two human-machine collaborative decision-making evaluation indicators. , for covariance, for Standard deviation;

[0058] Further calculate the assumed p-value to test whether the Pearson correlation coefficient is significantly different from zero, and determine whether there is a linear correlation between the indicators. The smaller the p-value, the stronger the linear correlation between the indicators. The p-value is calculated as follows:

[0059] Pearson correlation coefficient Convert to The statistic, with the formula:

[0060]

[0061] in, For the calculation Statistic, This refers to the sample size of the experimental data.

[0062] Calculate the degrees of freedom:

[0063]

[0064] get After statistical analysis, query based on degrees of freedom. The p-value can be calculated using a distribution table or statistical software. For a two-tailed test, the p-value is:

[0065]

[0066] in, yes One-sided probability of the distribution Obeying the degree of freedom of distributed.

[0067] Furthermore, based on the correlation coefficient, the selection of strongly correlated human-machine collaborative decision-making evaluation indicators and the establishment of a relationship function using multiple linear regression analysis include:

[0068] Set threshold , Choose to satisfy both and The two indicators, as strongly correlated human-machine collaborative decision-making evaluation indicators, are denoted as follows: , , , ;

[0069] Determine if there are three indicators that are satisfied pairwise. and If they exist, then the three indicators are selected as strongly correlated human-machine collaborative decision-making evaluation indicators, denoted as... , , ;

[0070] Establish using linear regression method The relationship function between the various indicators:

[0071]

[0072] in, Indicators The score, These are the coefficients of the relational function.

[0073] Furthermore, the step of calculating the average absolute percentage error between the predicted and actual values ​​of the evaluation index based on the relationship function, and evaluating the accuracy of the relationship function, includes:

[0074] Conduct several human factors experiments, obtain experimental data and calculate indicators. actual score And calculate the index based on the relational function. Predicted score ;

[0075] Establish the mean absolute percentage error function as the performance evaluation criterion for the relational function:

[0076]

[0077] in, The mean absolute percentage error, It is the sample size of the experimental data;

[0078] Establish a function to calculate the accuracy of the relational function:

[0079]

[0080] in, For accuracy.

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] (1) This invention considers three human-machine collaborative decision-making methods: human decision-making, AI decision-making, and human-machine collaborative decision-making, and realizes a comparison of the effects of human-machine collaborative decision-making compared with single decision-making.

[0083] (2) This invention takes into account both task performance and human factors, and establishes three categories of indicators, including objective performance indicators, subjective human perception indicators and human-computer interaction indicators, to establish a complete human-computer collaborative decision-making evaluation indicator system, which can scientifically and accurately evaluate the performance of human-computer collaborative decision-making.

[0084] (3) Based on the human-machine collaborative decision-making experiment, this invention realizes the quantitative characterization of human-machine collaborative decision-making evaluation indicators, analyzes the correlation between indicators, and reveals the interaction mechanism between indicators;

[0085] (4) The present invention adopts the multiple linear regression method to establish a quantitative relationship function between human-machine collaborative decision evaluation indicators and calculate the accuracy of the function, which further simplifies the indicator system and makes indicator analysis and prediction easier. Attached Figure Description

[0086] Figure 1 The flowchart illustrates a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators provided by this invention.

[0087] Figure 2 This is a graph showing the relationship between workload, human-machine trust, and satisfaction, which are indicators of this invention.

[0088] Figure 3 This is a graph showing the relationship between the indicators of satisfaction, human-computer interaction fluency, and human-computer trust in this invention. Detailed Implementation

[0089] This invention provides a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators, which addresses the technical problem that existing human-machine collaborative decision-making evaluation indicators are not comprehensive enough and are too complex.

[0090] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0091] Please refer to Figure 1 , Figure 1 The flowchart illustrates the steps of a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators, as provided in an embodiment of the present invention.

[0092] This invention provides a method for modeling the relationship between human-machine collaborative decision-making evaluation indicators, comprising:

[0093] Step 101: Establish a human-machine collaborative decision-making model, covering three human-machine collaborative decision-making methods, including:

[0094] Establish a human-machine collaborative decision-making model:

[0095]

[0096] The human-machine collaborative decision-making model considers three human-machine collaborative decision-making methods: human decision-making, AI decision-making, and human-machine collaborative decision-making. As a human decision-making method, For AI decision-making methods, In human condition, In AI state, The difficulty level of the experimental task. These represent the weights for human decision-making and AI decision-making, respectively.

[0097] When humans make decisions, the decisions rely entirely on human experience and intuition, and AI completely obeys human instructions. When AI makes decisions, the decisions rely entirely on AI's autonomous decision-making, and humans receive the decision results through AI language and use them as a reference to execute tasks. In human-machine collaborative decision-making, humans rely on experience and intuition, while AI makes decisions according to a weighted allocation.

[0098] Step 102: Based on the aforementioned human-machine collaborative decision-making model, establish a human-machine collaborative decision-making evaluation index system, including:

[0099] Establish a human-machine collaborative decision-making evaluation index system, including three categories: objective performance indicators, human subjective perception indicators, and human-machine interaction indicators;

[0100] Establish objective performance indicators, including task completion efficiency. ;

[0101] Establish indicators of people's subjective feelings, including workload. Situational awareness capability Satisfaction ;

[0102] Establish human-computer interaction metrics, including the fluency of human-computer interaction. Human-machine trust level .

[0103] Step 103: Conduct several human-computer collaborative decision-making experiments to obtain experimental data for each indicator in the human-computer collaborative decision-making evaluation index system, including:

[0104] Several human-machine collaborative decision-making experiments were conducted in a simulation environment, covering various tasks such as flight, attack, defense, and target rescue. Experiments were carried out under three decision-making methods: human decision-making, AI decision-making, and human-machine collaborative decision-making. After each experiment, a questionnaire was filled out to evaluate various indicators and obtain experimental data under different decision-making methods.

[0105] The experiment recruited 48 participants, each of whom participated in the experiment under three different decision-making methods. A total of 144 sets of experimental data were collected. The 48 participants were then divided into 24 human-machine teams and 12 all-human teams. Therefore, the human-machine teams obtained a total of 72 sets of experimental data.

[0106] Regarding task completion efficiency indicators Several human-machine collaborative decision-making human factors experiments were conducted, and the completion time of each experiment was recorded. And stored in the original timing dataset. middle;

[0107] Regarding workload indicators The experimental dataset was obtained using the NASA-TLX scale. and ;

[0108] Regarding situational awareness capability indicators Experimental datasets were obtained using the SART questionnaire. ;

[0109] Regarding satisfaction indicators Design a questionnaire with 6 sub-questions and obtain an experimental dataset. ;

[0110] For the human-computer interaction fluency index Design a questionnaire with three sub-questions and obtain the experimental dataset. ;

[0111] Regarding human-machine trust index Design a questionnaire with 5 sub-questions and obtain the experimental dataset. .

[0112] Step 104: Based on the experimental data, calculate the score for each human-machine collaborative decision-making evaluation indicator, and perform correlation analysis to obtain the correlation coefficients between pairs of indicators, including:

[0113] Calculate the score for the task completion efficiency index by analyzing the experiment completion time. Normalize to the range [1,10] and then reverse the process:

[0114]

[0115] in, The score represents the efficiency of task completion. For the original timing dataset The minimum value in, For the original timing dataset The maximum value in, ;

[0116] Workload, situational awareness, satisfaction, human-computer interaction fluency, and human-computer trust were assessed using a questionnaire with a ten-point rating system.

[0117] Calculate the score for the workload index:

[0118]

[0119] in, The NASA-TLX scale, used to assess workload, has six dimensions: mental workload, physical workload, time workload, performance level, effort level, and frustration level. These are the experimental data values ​​for a single dimension, representing the score and weight for that dimension, respectively. ;

[0120] To ensure that all six indicators are aligned, i.e., all are better the higher the score, the workload indicator is reversed. Since the indicator data range is [1, 10], the formula is:

[0121]

[0122] in, The score for the workload index after reverse processing;

[0123] Calculate the score for the situational awareness capability index:

[0124]

[0125] in, It is a score of situational awareness capability, and the SART questionnaire measures it in three dimensions. The experimental data values ​​represent three dimensions: the demand for attentional resources, the supply of attentional resources, and the level of understanding of the experimental context. ;

[0126] To ensure that the score of the situational awareness capability index falls within the range of [1, 10], a linear transformation is used for calculation, as shown in the formula:

[0127]

[0128] in, It is the score of the normalized situational awareness capability index;

[0129] To calculate the satisfaction index score, the satisfaction questionnaire has 6 sub-questions: (1) AI improved the team's productivity; (2) AI was integrated into the work; (3) I am satisfied with the performance of AI; (4) I am satisfied with the performance of the team; (5) The team cooperated very well; (6) I am willing to continue working with AI when completing the next task. The formula is:

[0130]

[0131] in, It is the score of the satisfaction index. , represents the experimental data value of a single subproblem;

[0132] To calculate the score of the human-computer interaction fluency index, the human-computer interaction fluency questionnaire has three sub-questions: (1) The team works together very smoothly; (2) The team has very little free time during the entire task; and (3) AI contributes to the fluency of human-computer interaction. The formula is:

[0133]

[0134] in, It is a score for the fluency of human-computer interaction. , represents the experimental data value of a single subproblem;

[0135] To calculate the score for the human-machine trust index, the questionnaire consists of five sub-questions: (1) AI is trustworthy; (2) I believe AI will make the right decisions; (3) I believe in AI's capabilities; (4) I trust AI; and (5) I work towards a common goal with AI. The formula is:

[0136]

[0137] in, It is a score for the human-machine trust index. , representing the experimental data value of a single subproblem;

[0138] By calculating the Pearson correlation coefficient between each pair of indicators, a correlation analysis was performed on the six processed indicators. The larger the correlation coefficient, the better the correlation. The formula for the Pearson correlation coefficient is:

[0139]

[0140] in, , These are the scores of two human-machine collaborative decision-making evaluation indicators. , for covariance, for The standard deviation.

[0141] Correlation analysis was performed on the six processed indicators by calculating the Pearson correlation coefficient between each pair of indicators. The larger the correlation coefficient, the better the correlation. The formula for the Pearson correlation coefficient is:

[0142]

[0143] in, , These are the scores of two human-machine collaborative decision-making evaluation indicators. , for covariance, for Standard deviation;

[0144] Further calculate the assumed p-value to test whether the Pearson correlation coefficient is significantly different from zero, and determine whether there is a linear correlation between the indicators. The smaller the p-value, the stronger the linear correlation between the indicators. The p-value is calculated as follows:

[0145] Pearson correlation coefficient Convert to The statistic, with the formula:

[0146]

[0147] in, For the calculation Statistic, This refers to the sample size of the experimental data.

[0148] Calculate the degrees of freedom:

[0149]

[0150] get After statistical analysis, query based on degrees of freedom. The p-value can be calculated using a distribution table or statistical software. For a two-tailed test, the p-value is:

[0151]

[0152] in, yes One-sided probability of the distribution Obeying the degree of freedom of Distribution. The calculated Pearson correlation coefficient is as follows:

[0153]

[0154] The data in the table represents the Pearson correlation coefficient between the two indicators. This indicates that the assumed value between the two indicators is p < 0.05. This means p < 0.01.

[0155] Step 105: Based on the correlation coefficient, select strongly correlated human-machine collaborative decision-making evaluation indicators, and establish a relationship function using multiple linear regression analysis, including:

[0156] Set threshold , , Choose to satisfy both and The two indicators, as strongly correlated human-machine collaborative decision-making evaluation indicators, are denoted as follows: , , , ;

[0157] Determine if there are three indicators that are satisfied pairwise. and If they exist, then the three indicators are selected as strongly correlated human-machine collaborative decision-making evaluation indicators, denoted as... , , ;

[0158] Establish using linear regression method The relationship function between the various indicators:

[0159]

[0160] in, Indicators The score, These are the coefficients of the relational function.

[0161] There are two sets of relationship functions between indicators, including: (1) workload, human-computer trust and satisfaction; (2) satisfaction, human-computer interaction fluency and human-computer trust.

[0162] Sixty sets of data were randomly selected from 72 sets of data from the human-machine team, and a relationship function between the two sets of evaluation indicators was established through multiple linear regression analysis.

[0163] Please refer to Figure 2 The relationship function between workload, human-machine trust, and satisfaction is:

[0164]

[0165] in For workload, Human-machine trust level For satisfaction.

[0166] Please refer to Figure 3 The relationship function between satisfaction, human-computer interaction fluency, and human-computer trust is:

[0167]

[0168] in For satisfaction, For the smoothness of human-computer interaction, Human-machine trust level.

[0169] Step 106: Based on the relationship function, calculate the average absolute percentage error between the predicted and actual values ​​of the evaluation index, and evaluate the accuracy of the relationship function, including:

[0170] Conduct several human factors experiments, obtain experimental data and calculate indicators. actual score And calculate the index based on the relational function. Predicted score ;

[0171] Establish the mean absolute percentage error function as the performance evaluation criterion for the relational function:

[0172]

[0173] in, The mean absolute percentage error, It is the sample size of the experimental data;

[0174] Establish a function to calculate the accuracy of the relational function:

[0175]

[0176] in, For accuracy.

[0177] The accuracy of the relationship function between the indicators was verified using the remaining 12 sets of data, and the following calculations were performed:

[0178] The regression equations for workload, human-computer trust, and satisfaction yielded a mean absolute percentage error (MAPE) of 15.703% and an ACC of 84.297%. The regression equations for satisfaction, human-computer interaction fluency, and human-computer trust yielded a mean absolute percentage error (MAPE) of 7.887% and an ACC of 92.113%. When the mean absolute percentage error is less than 20%, the predictive performance of the relational functions is good. Therefore, the accuracy of the three established relational functions is good, and the relevant indicators can be quantitatively represented using other indicators of the relational functions, further simplifying the indicator system.

[0179] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for modeling the relationship between evaluation indicators in human-machine collaborative decision-making, characterized in that, include: Establish a human-machine collaborative decision-making model, covering three human-machine collaborative decision-making methods; Based on the aforementioned human-machine collaborative decision-making model, an evaluation index system for human-machine collaborative decision-making is established. Conduct several human factors experiments on human-machine collaborative decision-making to obtain experimental data for each indicator in the human-machine collaborative decision-making evaluation index system; Based on the experimental data, the score of each human-machine collaborative decision-making evaluation index was calculated, and the correlation analysis of the indexes was performed to obtain the correlation coefficient between each pair of indexes. Based on the aforementioned correlation coefficients, highly correlated human-machine collaborative decision-making evaluation indicators are selected, and a relationship function is established using multiple linear regression analysis. Based on the relationship function, the average absolute percentage error between the predicted and actual values ​​of the evaluation index is calculated, and the accuracy of the relationship function is evaluated.

2. The method for modeling the relationship between human-machine collaborative decision-making evaluation indicators according to claim 1, characterized in that, The aforementioned human-machine collaborative decision-making model encompasses three human-machine collaborative decision-making methods, including: Establish a human-machine collaborative decision-making model: The human-machine collaborative decision-making model considers three human-machine collaborative decision-making methods: human decision-making, AI decision-making, and human-machine collaborative decision-making. As a human decision-making method, For AI decision-making methods, In human condition, In AI state, The difficulty level of the experimental task. These represent the weights for human decision-making and AI decision-making, respectively.

3. The method for modeling the relationship between human-machine collaborative decision-making evaluation indicators according to claim 2, characterized in that, Based on the aforementioned human-machine collaborative decision-making model, an evaluation index system for human-machine collaborative decision-making is established, including: Establish a human-machine collaborative decision-making evaluation index system, including three categories: objective performance indicators, human subjective perception indicators, and human-machine interaction indicators; Establish objective performance indicators, including task completion efficiency. ; Establish indicators of people's subjective feelings, including workload. Situational awareness capability Satisfaction ; Establish human-computer interaction metrics, including the fluency of human-computer interaction. Human-machine trust level .

4. The method for modeling the relationship between human-machine collaborative decision-making evaluation indicators according to claim 3, characterized in that, The aforementioned process involves conducting several human-computer collaborative decision-making experiments to obtain experimental data for each indicator in the human-computer collaborative decision-making evaluation index system, including: Regarding task completion efficiency indicators Several human-machine collaborative decision-making human factors experiments were conducted, and the completion time of each experiment was recorded. And stored in the original timing dataset. middle; Regarding workload indicators The experimental dataset was obtained using the NASA-TLX scale. and ; Regarding situational awareness capability indicators Experimental datasets were obtained using the SART questionnaire. ; Regarding satisfaction indicators Design questionnaires, set up Sub-problems and obtaining experimental datasets. ; For the human-computer interaction fluency index Design questionnaires, set up Sub-problems and obtaining experimental datasets. ; Regarding human-machine trust index Design questionnaires, set up Sub-problems and obtaining experimental datasets. .

5. The method for modeling the relationship between human-machine collaborative decision-making evaluation indicators according to claim 4, characterized in that, Based on the experimental data, the score for each human-machine collaborative decision-making evaluation indicator is calculated, and correlation analysis is performed to obtain the correlation coefficient between any two indicators, including: Calculate the score for the task completion efficiency index by analyzing the experiment completion time. Perform normalization and reverse processing: in, The score represents the efficiency of task completion. For the original timing dataset The minimum value in, For the original timing dataset The maximum value in, These represent the upper and lower limits of the normalized time range. ; Workload, situational awareness, satisfaction, human-computer interaction fluency, and human-computer trust were measured using experimental data obtained through questionnaires. The score range for each question in the questionnaire was set to [a, b]. Calculate the score for the workload index: in, The NASA-TLX scale, which measures workload as a score, has six dimensions. These are the experimental data values ​​for a single dimension, representing the score and weight for that dimension, respectively. ; The scores for the workload index are processed in reverse: in, The score for the workload index after reverse processing; Calculate the score for the situational awareness capability index: in, It is a score of situational awareness capability, and the SART questionnaire measures it in three dimensions. The experimental data values ​​represent three dimensions: the demand for attentional resources, the supply of attentional resources, and the level of understanding of the experimental context. ; Normalize the situational awareness capability indicators: in, It is the score of the normalized situational awareness capability index; Calculate the score for the satisfaction index: in, It is the score of the satisfaction index. The number of sub-questions in the questionnaire. , represents the experimental data value of a single subproblem; The formula for calculating the score of the human-computer interaction fluency index is: in, It is a score for the fluency of human-computer interaction. The number of sub-questions in the questionnaire. , represents the experimental data value of a single subproblem; The formula for calculating the human-machine trust index score is: in, It is a score for the human-machine trust index. The number of sub-questions in the questionnaire. , representing the experimental data value of a single subproblem; Correlation analysis was performed on the six processed indicators by calculating the Pearson correlation coefficient between each pair of indicators. The larger the correlation coefficient, the better the correlation. The formula for the Pearson correlation coefficient is: in, , These are the scores of two human-machine collaborative decision-making evaluation indicators. , for covariance, for Standard deviation; Further calculate the assumed p-value to test whether the Pearson correlation coefficient is significantly different from zero, and determine whether there is a linear correlation between the indicators. The smaller the p-value, the stronger the linear correlation between the indicators. The p-value is calculated as follows: Pearson correlation coefficient Convert to The statistic, with the formula: in, For the calculation Statistic, This refers to the sample size of the experimental data. Calculate the degrees of freedom: get After statistical analysis, query based on degrees of freedom. The p-value can be calculated using a distribution table or statistical software. For a two-tailed test, the p-value is: in, yes One-sided probability of the distribution Obeying the degree of freedom of distributed.

6. The method for modeling the relationship between human-machine collaborative decision-making evaluation indicators according to claim 5, characterized in that, Based on the correlation coefficient, strongly correlated human-machine collaborative decision-making evaluation indicators are selected, and a relationship function is established using multiple linear regression analysis, including: Set threshold , Choose to satisfy both and The two indicators, as strongly correlated human-machine collaborative decision-making evaluation indicators, are denoted as follows: , , , ; Determine if there are three indicators that are satisfied pairwise. and If they exist, then the three indicators are selected as strongly correlated human-machine collaborative decision-making evaluation indicators, denoted as... , , ; Establish using linear regression method The relationship function between the various indicators: in, Indicators The score, These are the coefficients of the relational function.

7. The method for modeling the relationship between evaluation indicators for human-machine collaborative decision-making according to claim 6, characterized in that, The step of calculating the average absolute percentage error between the predicted and actual values ​​of the evaluation index based on the relationship function, and evaluating the accuracy of the relationship function, includes: Conduct several human factors experiments, obtain experimental data and calculate indicators. actual score And calculate the index based on the relational function. Predicted score ; Establish the mean absolute percentage error function as the performance evaluation criterion for the relational function: in, The mean absolute percentage error, It is the sample size of the experimental data; Establish a function to calculate the accuracy of the relational function: in, For accuracy.

Images