CRITIC-based human-machine cooperation system trust degree evaluation method

Abstract

The invention discloses a CRITIC-based man-machine cooperation system trust degree evaluation method, and belongs to the technical field of man-machine interaction and human factor engineering, and the method comprises the steps: S1, obtaining the physiological data and flight data of a pilot when the pilot executes a manned and unmanned system task; s2, preprocessing and standardizing the physiological data and the flight data respectively; s3, weight calculation is carried out based on a CRITIC method, and standardized multi-dimensional feature data is used as input of a CRITIC model and is used for calculating objective weights of all indexes; and S4, calculating a credibility value by using the generated weight matrix. By adopting the CRITIC-based man-machine cooperation system trust degree evaluation method, objective real-time quantitative evaluation is realized; the weight is adaptively determined according to the data features, the method is more scientific than artificial experience, physiological and behavior data are fused to capture psychological changes, the trust level can be monitored, the cooperation strategy is adjusted accordingly, and the task safety and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the fields of human-computer interaction and human factors engineering, and in particular to a trust assessment method for human-computer collaboration systems based on CRITIC. Background Technology

[0002] In high-risk and complex human-machine collaborative systems, the operator's trust in the autonomous system is a crucial factor affecting task success and safety; excessively high or low trust levels can negatively impact decision-making. Traditional trust assessment methods typically rely on post-event expert ratings or scores based on a single indicator. Questionnaire surveys cannot continuously capture changes in trust levels during task execution, while using only a single physiological signal as a trust indicator lacks persuasiveness and fails to encompass the multidimensional factors of trust. Furthermore, they are highly subjective, lack real-time accuracy, and struggle to comprehensively and objectively reflect the operator's instantaneous trust state.

[0003] This invention proposes a human-computer collaboration trust assessment method based on the CRITIC method (Criteria Importance Through Intercriteria Correlation), which simultaneously considers the variability and correlation of indicators to measure trust. By comprehensively calculating the information content of each indicator and the degree of conflict with other indicators, objective and reasonable weights are automatically generated, thereby avoiding human subjective bias. This invention is suitable for modeling trust by integrating multimodal physiological and behavioral signals, and has potential application value in the trust assessment of current human-computer collaboration systems. Summary of the Invention

[0004] The purpose of this invention is to provide a CRITIC-based trust assessment method for human-machine collaboration systems, overcoming the shortcomings of traditional assessments that are highly subjective and rely on a single indicator, thus achieving objective, real-time, and quantitative evaluation. The CRITIC algorithm adaptively determines the weights of each indicator based on the characteristics of the data itself, making it more scientific and objective compared to traditional methods that rely on human experience to set weights. Furthermore, by integrating physiological signals and flight data, it can more comprehensively capture changes in the operator's psychological state, achieving accurate trust assessment.

[0005] To achieve the above objectives, this invention provides a trust evaluation method for human-computer collaboration systems based on CRITIC, comprising the following steps:

[0006] S1. Acquire physiological and flight data of pilots when performing manned or unmanned system missions;

[0007] S2. Preprocess and standardize physiological data and flight data respectively;

[0008] S3. Calculate weights based on the CRITIC method, using standardized multidimensional feature data as input to the CRITIC model to calculate the objective weights of each indicator.

[0009] S4. Calculate the confidence value using the generated weight matrix.

[0010] Preferably, in step S1, the physiological data includes physiological signals and electromyographic signals. The physiological signals include heart rate and heart rate variability obtained through a heart rate monitoring device. The electromyographic signals are obtained by collecting the intensity of upper arm or shoulder muscle activity through an electromyographic sensor and calculating the root mean square value and amplitude of the electromyographic signals. The flight data includes the distribution of the operator's gaze area obtained through an eye tracker, and the number of times the operator manually intervenes in the autonomous system.

[0011] Preferably, step S2 specifically includes: preprocessing the acquired physiological data by channel according to data characteristics, including removing noise interference from the heart rate signal, calculating the time-domain or frequency-domain feature values ​​of heart rate variability, and filtering and denoising the electromyographic signal; and performing uniform standardization processing on all extracted feature indicators.

[0012] Preferably, step S3 includes the following steps:

[0013] S31. Calculation of index variability: Based on the normalized data matrix, the standard deviation is used to quantify the degree of dispersion of the data within the index.

[0014] S32. Calculation of the correlation coefficient matrix of indicators: By calculating the Pearson correlation coefficient of each pair of indicators, the correlation between indicator data is analyzed.

[0015] S33. Calculation of indicator information content: Based on the standard deviation of the above indicator data and the correlation measurement between indicators, calculate the total information content of the selected indicators in the experiment.

[0016] S34. Weight vector generation: Normalize the information content of each indicator obtained from the above calculation to obtain the weight of each indicator.

[0017] Preferably, in step S31, the specific formula for calculating the quantified standard deviation is as follows:

[0018]

[0019] Where M represents the total number of samples, x′ nj This represents the normalized value of the nth sample on the jth indicator. This represents the mean of the j-th indicator.

[0020] Preferably, in step S32, the correlation coefficient matrix of the dimensionless matrix X' is denoted as R = [r ij ]K×K ,in:

[0021]

[0022] Where, x′ ni It is the normalized value of the nth sample on the i-th indicator. r represents the mean of the indicator. ij This represents the Pearson correlation coefficient between the i-th and j-th indicators, with a value range of [-1, 1]. Let represent the mean of the i-th indicator.

[0023] Preferably, in step S33, the amount of information contained in the j-th indicator is calculated as follows:

[0024]

[0025] Where K represents the total number of indicators.

[0026] Preferably, in step S34, the weight of the j-th indicator is:

[0027]

[0028] Among them, w j This represents the normalized weight of the j-th indicator, satisfying...

[0029] Preferably, in step S4, the pilot's overall trust score for the nth test is obtained based on the weight values ​​of the physiological signal indicators and the standardized experimental data corresponding to each indicator. The calculation formula is as follows:

[0030]

[0031] Among them, T n This represents the overall trust score for the nth experiment.

[0032] Therefore, this invention employs the aforementioned CRITIC-based human-computer collaboration system trust assessment method. Firstly, it improves the objectivity and real-time performance of trust assessment. Utilizing the CRITIC objective weighting algorithm, it automatically calculates the importance weights of each indicator based on actually collected data, avoiding subjective bias caused by manually setting weights. Trust scores are directly calculated from real-time acquired physiological and behavioral data, allowing the assessment to be dynamically updated as the task progresses. Compared to traditional methods relying on post-task questionnaires or expert scoring, this invention can continuously and objectively monitor trust levels during task execution, thereby promptly identifying and adjusting abnormal trust states, improving the safety and effectiveness of human-computer collaboration. Secondly, it enhances assessment accuracy by integrating multimodal data. This invention's method integrates various physiological and behavioral signals such as heart rate, HRV, electromyography, eye movement, and human intervention for trust assessment, fully utilizing the different dimensions of information reflected by each signal. The integration of multimodal data makes the assessment model's representation of the operator's psychological state more comprehensive and nuanced. For example, heart rate and HRV can reflect stress, electromyography signals reflect the operator's limb tension, eye movement behavior reveals attention distribution, and the number of human interventions directly reflects trust in the autonomous system's behavior. By objectively determining the relative weights of these indicators using the CRITIC method, the resulting trust score is more accurate and reliable than any single indicator. Experiments show that the model integrating multi-source data has higher sensitivity and accuracy in identifying trust change trends and extreme trust states, significantly outperforming traditional single-indicator evaluation methods. Finally, the method is highly applicable and can be extended to other complex task environments. The trust evaluation framework provided by this invention has good versatility and scalability. Although primarily applied to air combat systems, this method does not depend on specific task scenarios or equipment; it only requires the acquisition of multimodal data related to operator states. For other complex human-machine collaboration domains, corresponding trust evaluation models can be constructed simply by selecting appropriate physiological and behavioral indicators and employing the same CRITIC weighting and weighted summation strategy. For example, in autonomous driving scenarios, signals such as driver gaze patterns and braking intervention frequency can be introduced; in industrial control, information such as operator EEG and operation frequency can be collected. The method framework of this invention can flexibly incorporate new indicators and data sources, adjusting the evaluation content for different application scenarios. Due to the use of an objective weighting strategy, it can adapt to new environments without manually resetting weight parameters. Therefore, this method has broad applicability and is expected to play an important role in trust assessment and management of various human-machine collaborative work systems.

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0034] Figure 1This is a flowchart of a trust evaluation method for a human-computer collaboration system based on CRITIC proposed in this invention;

[0035] Figure 2 This is a flowchart of the weight calculation process based on the CRITIC method proposed in this invention. Detailed Implementation

[0036] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The power modules of the embodiments of the present invention can be arranged and designed in various different configurations. The specific model specifications need to be selected and determined according to actual needs, etc., and therefore will not be described in detail.

[0038] Example

[0039] like Figure 1 As shown, a trust evaluation method for human-computer collaboration systems based on CRITIC has the following main steps:

[0040] Step S1: Acquire physiological and flight data of the pilot during manned and unmanned system missions.

[0041] In this embodiment, a physiological and flight data acquisition system is built in a human-machine collaborative experimental environment to synchronously acquire the operator's multimodal signals and obtain multimodal physiological and flight data. Specifically, the physiological data includes physiological signals and electromyographic (EMG) signals. Physiological signals include heart rate and heart rate variability, which reflect the operator's tension and stress levels. EMG signals are acquired by collecting the intensity of upper arm or shoulder muscle activity through EMG sensors, and the root mean square value and amplitude of the EMG signals are calculated to reflect the operator's muscle exertion and fatigue state. Flight data includes the distribution of the operator's gaze area acquired through an eye tracker to characterize their attention allocation. Simultaneously, the number of times the operator intervenes in the autonomous system is recorded, i.e., the frequency of manual intervention or adjustment during the autonomous system's decision-making process.

[0042] Step S2: Preprocess and standardize the physiological data and flight data respectively.

[0043] The acquired physiological data were preprocessed according to their characteristics, categorized by channel. Necessary preprocessing was performed on various data types, such as removing noise interference from heart rate signals, calculating time-domain or frequency-domain feature values ​​of heart rate variability, and filtering and denoising electromyographic signals. All extracted feature indicators were standardized to eliminate incomparability caused by differences in units of measurement. Standardization ensures that subsequent weighting and comprehensive evaluation of all indicators are performed under the same unit of measurement.

[0044] Step S3: Calculate weights based on the CRITIC method, using standardized multidimensional feature data as input to the CRITIC model to calculate the objective weights of each indicator, such as... Figure 2 As shown, it specifically includes:

[0045] Step S31: Calculation of Indicator Variability. Based on the normalized data matrix, the standard deviation is used to quantify the dispersion of data within the indicator. The larger the standard deviation, the more significant the fluctuation of the indicator value, meaning that the indicator contains more information. The specific calculation formula is as follows.

[0046]

[0047] Where M represents the total number of samples, x′ nj This represents the normalized value of the nth sample on the jth indicator. This represents the mean of the j-th indicator.

[0048] Step S32: Calculate the correlation coefficient matrix of the indicators. Analyze the correlation between indicator data by calculating the Pearson correlation coefficients pairwise for each indicator. Let the correlation coefficient matrix of the dimensionless matrix X' be R = [r...]. ij ] K×K ,in

[0049]

[0050] In the formula, x′ ni It is the normalized value of the nth sample on the i-th indicator. r represents the mean of the indicator. ij This represents the Pearson correlation coefficient between the i-th and j-th indicators, with a value range of [-1, 1]. Let represent the mean of the i-th indicator.

[0051] Step S33: Calculation of Indicator Information Content. Based on the standard deviation of each indicator and the correlation measure between indicators, calculate the total information content of the selected indicators in the experiment. The information content of the j-th indicator is calculated as follows:

[0052]

[0053] Where K represents the total number of indicators.

[0054] Step S34: Weight Vector Generation. The information content of each indicator calculated above is normalized to obtain the weight of each indicator. The weight is proportional to the information content; indicators with more information content have a greater impact on the overall evaluation. The weight of the j-th indicator is w. j The calculation is as follows.

[0055]

[0056] Among them, w j This represents the normalized weight of the j-th indicator, satisfying...

[0057] Step S4: Calculate the trust score using the generated weight matrix. In this embodiment, the pilot's overall trust score for the nth test is obtained based on the weight values ​​of the physiological signal indicators and the standardized experimental data corresponding to each indicator. The calculation formula is as follows:

[0058]

[0059] Among them, T n This represents the overall trust score for the nth experiment.

[0060] Therefore, the present invention adopts the above-mentioned trust evaluation method for human-machine collaboration system based on CRITIC. The system can monitor the operator's trust level during task execution and adjust the human-machine collaboration strategy in a timely manner according to the trust results to improve task safety and efficiency.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A trust evaluation method for human-computer collaboration systems based on CRITIC, characterized in that, Includes the following steps: S1. Acquire physiological and flight data of pilots when performing manned or unmanned system missions; S2. Preprocess and standardize physiological data and flight data respectively; S3. Calculate weights based on the CRITIC method, using standardized multidimensional feature data as input to the CRITIC model to calculate the objective weights of each indicator. S4. Calculate the confidence value using the generated weight matrix.

2. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 1, characterized in that, In step S1, the physiological data includes physiological signals and electromyographic signals. The physiological signals include heart rate and heart rate variability obtained through a heart rate monitoring device. Electromyography (EMG) signals are collected by EMG sensors to measure the intensity of muscle activity in the upper arm or shoulder, and the root mean square value and amplitude of the EMG signals are calculated. Flight data includes the distribution of the operator's gaze area obtained by an eye tracker, and the number of times the operator intervenes in the autonomous system.

3. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 1, characterized in that, Step S2 specifically includes: preprocessing the acquired physiological data by channel according to data characteristics, including removing noise interference from the heart rate signal, calculating the time-domain or frequency-domain feature values ​​of heart rate variability, and filtering and denoising the electromyographic signal; and performing unified standardization processing on all extracted feature indicators.

4. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 1, characterized in that, Step S3 includes the following steps: S31. Calculation of index variability: Based on the normalized data matrix, the standard deviation is used to quantify the degree of dispersion of the data within the index. S32. Calculation of the correlation coefficient matrix of indicators: By calculating the Pearson correlation coefficient of each pair of indicators, the correlation between indicator data is analyzed. S33. Calculation of indicator information content: Combine the standard deviation of each indicator data and the correlation measurement between indicators to calculate the total information content of the selected indicators in the experiment. S34. Weight vector generation: Normalize the calculated information of each indicator to obtain the weight of each indicator.

5. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 4, characterized in that, In step S31, the specific formula for calculating the quantified standard deviation is as follows: Where M represents the total number of samples, x′ nj This represents the normalized value of the nth sample on the jth indicator. This represents the mean of the j-th indicator.

6. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 5, characterized in that, In step S32, the correlation coefficient matrix of the dimensionless matrix X' is denoted as R = [r ij ] K×K ,in: Where, x′ ni It is the normalized value of the nth sample on the i-th indicator. r represents the mean of the indicator. ij This represents the Pearson correlation coefficient between the i-th and j-th indicators, with a value range of [-1, 1]. Let represent the mean of the i-th indicator.

7. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 6, characterized in that, In step S33, the amount of information contained in the j-th indicator is calculated as follows: Where K represents the total number of indicators.

8. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 7, characterized in that, In step S34, the weight of the j-th indicator is calculated as follows: Among them, w j This represents the normalized weight of the j-th indicator, satisfying...

9. The trust evaluation method for a human-computer collaboration system based on CRITIC according to claim 8, characterized in that, In step S4, based on the weight values ​​of the physiological signal indicators and the standardized experimental data corresponding to each indicator, the pilot's overall trust score for the nth test is obtained, calculated using the following formula: Among them, T n This represents the overall trust score for the nth experiment.

Images