A human-al trust multi-source fusion measurement system, method, device and medium

Abstract

The present application relates to a kind of human-AI trust multi-source fusion measurement system, method, equipment and medium, belong to human-AI trust measurement and design evaluation technical field, including the multi-source synchronous acquisition mark module for providing multi-source synchronous mark trust acquisition data output, support multi-source data fusion measurement and feature calculation Fusion measurement module and for carrying out self-reporting and behavior and physiological feature correlation analysis Correlation analysis module, the present application can realize the synchronous acquisition and marking of comprehensive self-reporting, behavior and physiological trust data in human-AI cooperative flight scene by the hardware synchronization of the above-mentioned module with the aid of the multi-source synchronous acquisition mark module and manual marking.

Description

Technical Field

[0001] This invention relates to the field of human-AI trust measurement and design evaluation technology, specifically to a multi-source fusion measurement system, method, device, and medium for human-AI trust. Background Technology

[0002] Human-AI trust can be defined as "an individual's attitude that they believe AI will help them achieve their goals in situations filled with uncertainty and vulnerability." An imbalance in human-AI collaborative trust can have serious consequences. Multi-source fusion measurement of human-AI collaborative trust has become one of the key issues in the field. Solving this problem can support trust measurement and design optimization in typical human-AI collaborative flight mission scenarios, thereby improving the integration efficiency of humans and systems and ensuring human safety during operations.

[0003] Current research on multi-source fusion measurement of trust in human-AI collaboration suffers from the following four shortcomings. First, most studies only involve asynchronous acquisition or labeling of limited categories of trust data, lacking synchronous acquisition and labeling of comprehensive self-report, behavioral, and physiological trust data. Second, most studies use self-report methods for trust data collection and calculation, lacking integration of behavioral attention monitoring and facial expression data acquisition and calculation modules. Third, in addition to self-report and behavioral-based trust measurements, there is also a lack of fusion of physiological-based ECG, pupil, EEG, and near-infrared data acquisition and calculation modules. Finally, most studies only involve limited multidimensional self-report trust data analysis, with less focus on the correlation analysis between self-report trust data and behavioral and physiological trust measurements.

[0004] To address the aforementioned research gaps, a multi-source fusion measurement system, method, device, and product for human-AI collaborative trust are designed for application in the fusion measurement and analysis of human-AI collaborative trust in typical flight scenarios, thereby providing support for the design and optimization of human-AI collaborative trust in intelligent cockpits. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of the prior art and provide a human-AI trusted multi-source fusion measurement system, method, device and medium.

[0006] The technical solution of the present invention is as follows:

[0007] A multi-source fusion measurement system for human-AI trust includes a multi-source synchronous acquisition and labeling module for providing multi-source synchronous labeling trust acquisition data output, a fusion measurement module for supporting multi-source data fusion measurement and feature calculation, and a correlation analysis module for conducting self-reporting and behavioral and physiological feature correlation analysis.

[0008] The integrated measurement module includes a self-report-based trust measurement submodule, a behavior-based trust measurement submodule, and a physiological-based trust measurement submodule;

[0009] The self-report-based trust measurement submodule calculates the trust scale data collected in the multi-source synchronous acquisition and labeling module and outputs the self-reported trust characteristics.

[0010] The behavior-based trust measurement submodule calculates the attention monitoring behavior and facial expression behavior data collected in the multi-source synchronous acquisition and labeling module, and outputs behavior-based trust features.

[0011] The physiological-based trust measurement submodule calculates the EEG, near-infrared, eye movement, pupil, and ECG data collected in the multi-source synchronous acquisition and labeling module, and outputs physiological-based trust features.

[0012] The correlation analysis module includes a sub-module for correlation analysis between self-report and behavioral characteristics, and a sub-module for correlation analysis between self-report and physiological characteristics;

[0013] The self-report and behavioral feature correlation analysis submodule is used to perform correlation analysis between the obtained self-reported trust features and the behavior-based trust features, and output the correlation coefficient between the self-reported trust features and the behavior-based trust features.

[0014] The self-report and physiological characteristic association analysis submodule is used to perform correlation analysis on the obtained self-reported trust characteristics and physiologically based trust characteristics in turn, and output the correlation coefficient between the self-reported trust characteristics and the physiologically based trust characteristics.

[0015] Furthermore, the multi-source synchronous acquisition marker module sends a marker by starting a synchronization program on the near-infrared computer, and connects to the near-infrared computer, the volume port of the eye tracker, and the event button interface of the EEG amplifier via synchronization hardware to achieve synchronous acquisition of near-infrared, eye tracker, and EEG data.

[0016] The ECG, facial expression, and trust scale scores were manually labeled; finally, multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement and pupil, ECG, and trust scale scores were collected and output.

[0017] Furthermore, the calculation steps for the self-report-based trust measurement submodule to output the self-reported trust characteristics are as follows:

[0018] The trust scale data output by the multi-source synchronous acquisition and labeling module is calculated, and the formula for calculating the total score J of the Jian scale is as follows:

[0019] ;

[0020] In the above formula, J i For the score of positive sub-dimension i of the Jian scale, J j The score of the negative sub-dimension j of the Jian scale;

[0021] Then, calculate the total score M of the Madsen scale using the following formula:

[0022] ;

[0023] M k The score for sub-dimension k of the Madsen scale.

[0024] Furthermore, the calculation steps for the behavior-based trust measurement submodule to output behavior-based trust features are as follows:

[0025] The formula for calculating the total duration of attention (TDF) at all fixations is as follows: The attention monitoring behavior and facial expression behavior data output by the multi-source synchronous acquisition and labeling module are calculated.

[0026] ;

[0027] In the above formula, d l Let be the duration of the l-th gaze event, and NF be the number of gaze points within the selected time T;

[0028] Then, calculate the mean M_H of pleasant facial expressions and the standard deviation SD_N of neutral facial expressions, using the following formula:

[0029] ;

[0030] ;

[0031] In the above formula, H P Let N be the value of a happy expression at the p-th time point, where n is the total number of time points within the valid time period. P Let M_N be the neutral expression value at the p-th time point, and M_N be the mean of the neutral expression.

[0032] Furthermore, the calculation steps for outputting physiologically-based trust features by the physiologically-based trust measurement submodule are as follows:

[0033] First, the ECG, eye movement and pupillary data, EEG, and near-infrared data output by the multi-source synchronous acquisition and labeling module are calculated. The formula for calculating the standard deviation of the normal RR interval in ECG using SDNN is as follows:

[0034] ;

[0035] In the above formula, This is the value of the p-th RR interval. Let N be the mean of all RR intervals, and N be the total number of RR intervals.

[0036] Then, the formula for calculating the total power TP of the electrocardiogram is:

[0037] ;

[0038] In the above formula, TP represents the total power. Let df be the power spectral density of the electrocardiogram signal, and df be the derivative of the frequency variable f, i.e., the infinitesimal change.

[0039] The formula for calculating the high-frequency ratio (HF_ratio) in an electrocardiogram (ECG) is:

[0040] ;

[0041] In the above formula, low-frequency power High-frequency power ;

[0042] For eye movement pupil features, calculate the pupil diameter (AWFPD) at all fixation points;

[0043] For EEG characteristics, the formulas for calculating the δ relative power δ_r and the α relative power α_r are:

[0044] ;

[0045] ;

[0046] In the above formula, Let be the power spectral density of the EEG signal; and from this, the δ relative power δ_r_CZ at the CZ electrode point, the δ relative power δ_r_C4 at the C4 electrode point, the δ relative power δ_r_P4 at the P4 electrode point, the α relative power α_r_CZ at the CZ electrode point, and the α relative power α_r_C4 at the C4 electrode point can be obtained respectively;

[0047] Finally, the changes in near-infrared oxygenated hemoglobin concentration (O2Hb) and deoxygenated hemoglobin concentration (HHb) are calculated using the following formula:

[0048] ;

[0049] ;

[0050] In the above formula, This represents the absorbance change at wavelength λ. Let be the molar extinction coefficient of O2Hb at wavelength λ. Let λ be the molar extinction coefficient of HHb at wavelength λ, L be the distance between the light source and the probe, DPF be the optical path diffusion coefficient, and λ1 and λ2 be the two wavelengths used by the near-infrared device. From this, the changes in oxygenated hemoglobin concentration O2Hb_R2_T3 and deoxygenated hemoglobin concentration HHb_R2_T3 in the R2_T3 channel can be obtained respectively.

[0051] Furthermore, the self-report and behavioral characteristic correlation analysis submodule outputs the correlation coefficient r between the trust characteristics of self-reported information and the trust characteristics based on behavior. J,Y and r M,Y The calculation formula is:

[0052] ;

[0053] ;

[0054] ;

[0055] In the above formula, r XY Let X be the Pearson correlation coefficient between variables X and Y. i Let Y be the i-th observation of variable X. i Let Y be the i-th observation value. For the sample size, Let X be the sample mean. Let y be the sample mean of variable Y, SX be the standard deviation of variable X, and SY be the standard deviation of variable Y.

[0056] Furthermore, the self-report and behavioral characteristic correlation analysis submodule outputs the correlation coefficient r between the self-reported trust characteristics and the physiologically based trust characteristics. J,Y' and r M,Y' The calculation formula is:

[0057] ;

[0058] ;

[0059] In the above formula, Let X and X be variables. The Pearson correlation coefficient between them For variables The i-th observation, For variables The sample mean.

[0060] A multi-source fusion measurement method for human-AI trust includes the following steps:

[0061] S1 provides output of multi-source synchronously labeled and trusted data collection in human-AI collaborative flight scenarios;

[0062] S2. For the multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement and pupil, ECG and trust scale score data read in step S1, multi-source data fusion measurement and feature calculation are performed through the fusion measurement module.

[0063] S3. After inputting the calculated value from step S2 into the association analysis module, calculate and output the association analysis results between self-report and behavior and physiological characteristics.

[0064] An electronic device, comprising:

[0065] At least one processor;

[0066] A memory communicatively connected to the at least one processor; wherein,

[0067] The memory stores a computer program that can be executed by the at least one processor, which, when executed by the at least one processor, causes the at least one processor to perform a human-AI trusted multi-source fusion measurement method.

[0068] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements a human-AI trust-based multi-source fusion measurement method.

[0069] Compared with the prior art, the beneficial effects of the present invention are:

[0070] 1. It provides a multi-source fusion measurement system for human-AI collaborative trust. With the help of hardware synchronization and manual marking of the multi-source synchronous acquisition and marking module, it can realize the synchronous acquisition and marking of comprehensive self-reporting, behavior and physiological trust data in human-AI collaborative flight scenarios.

[0071] 2. The multi-source fusion measurement system based on human-AI collaborative trust provides a fusion measurement method for behavior-based attention monitoring and facial expression behavior data, as well as physiological data such as ECG, pupil, EEG, and near-infrared data in human-AI collaborative flight scenarios. By simply inputting the collected near-infrared, EEG, eye-tracking behavior, facial expression, eye-tracking pupil, ECG, and trust scale scores, 17 features sensitive to human-AI collaborative trust can be calculated.

[0072] 3. Compared with traditional human-AI collaborative trust measurement methods, the present invention can effectively integrate self-report, behavioral and physiological trust data to calculate specific trust sensitivity characteristics and the correlation between self-report and behavioral and physiological trust measurements. It can be applied to the fusion measurement and analysis of human-AI collaborative trust in typical flight scenarios, thereby providing support for the design and optimization of human-AI collaborative trust in intelligent cockpits. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the overall structure of the multi-source fusion measurement system for human-AI collaborative trust in this invention.

[0074] Figure 2 This is the overall flowchart of the multi-source fusion measurement system for human-AI collaborative trust in this invention.

[0075] Figure 3 This is the overall flowchart of the multi-source synchronous acquisition and marking module in this invention.

[0076] Figure 4 This is a flowchart of the fusion measurement module in this invention.

[0077] Figure 5 This is a flowchart of the correlation analysis module in this invention.

[0078] Figure 6 This is the initial homepage diagram of the multi-source fusion measurement system for human-AI collaborative trust in this invention.

[0079] Figure 7 This is a screenshot of the interface after selecting the "Instructions for Use" option on the initial homepage of this invention.

[0080] Figure 8 This is a diagram showing the interface layout after selecting the multi-source synchronous acquisition marker option on the initial homepage of this invention.

[0081] Figure 9 This is a diagram showing the interface layout after selecting the fusion measurement option on the initial homepage of this invention.

[0082] Figure 10 This is a diagram showing the interface layout after selecting the correlation analysis option on the initial homepage of this invention. Detailed Implementation

[0083] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0084] Example 1

[0085] like Figure 1As shown, this embodiment provides a multi-source fusion measurement system for human-AI collaborative trust, including a multi-source synchronous acquisition and labeling module for providing multi-source synchronous labeling trust acquisition data output, a fusion measurement module for supporting multi-source data fusion measurement and feature calculation, and a correlation analysis module for conducting self-reporting and behavioral and physiological feature correlation analysis.

[0086] The multi-source synchronous acquisition and labeling module sends markers by starting a synchronization program on a near-infrared computer. Through synchronization hardware connections to the near-infrared computer, the volume port of the eye tracker, and the event button interface of the EEG amplifier, it achieves synchronous acquisition of near-infrared, eye tracker, and EEG data. It also manually labels ECG, facial expression, and trust scale scores. Finally, it acquires and outputs multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement pupils, ECG, and trust scale scores.

[0087] The fusion measurement module outputs 17 trust measurement sensitive features. These features are all obtained from previous human-AI collaborative trust experiments and are the focus of this patent. Specifically:

[0088] Two self-reported trust characteristics, including the total score J of the Jian Scale and the total score M of the Madsen Scale;

[0089] Four behavior-based trust features include the number of fixations (NF) within a selected time T for attentional monitoring behavior and the total duration (TDF) of all fixations; as well as the mean of pleasant facial expression behavior (M_H) and the standard deviation of neutral expression (SD_N).

[0090] Eleven physiologically based trust features were included, comprising three electrocardiogram (SDNN, standard deviation of normal RR intervals, total power TP, and high-frequency ratio HF_ratio), one oculomotor pupillary feature (AWFPD of pupils at all fixations), five electroencephalogram (δ_r_CZ, δ_r_C4, δ_r_P4, α_r_CZ, and α_r_C4) and two near-infrared (O2Hb_R2_T3 and HHb_R2_T3) features in the R2_T3 channel.

[0091] The integrated measurement module includes a self-report-based trust measurement submodule, a behavior-based trust measurement submodule, and a physiological-based trust measurement submodule;

[0092] The self-report-based trust measurement submodule calculates the trust scale data collected in the multi-source synchronous acquisition and labeling module to output the self-reported trust characteristics.

[0093] In detail, the trust scale data output by the multi-source synchronous acquisition and labeling module is first calculated. The formula for calculating the total score J of the Jian scale is as follows:

[0094] (1),

[0095] In the above formula (1), J i For the score of positive sub-dimension i of the Jian scale, J j The score for the negative sub-dimension j of the Jian scale is calculated; then, the total score M of the Madsen scale is calculated using the following formula:

[0096] (2),

[0097] In equation (2), M k The score for sub-dimension k of the Madsen scale;

[0098] The behavior-based trust measurement submodule calculates the attention monitoring behavior and facial expression behavior data collected in the multi-source synchronous acquisition and labeling module to output behavior-based trust features.

[0099] In detail, firstly, the attention monitoring behavior and facial expression behavior data output by the multi-source synchronous acquisition and labeling module are calculated. The formula for calculating the total duration of attention (TDF) at all fixation points is as follows:

[0100] (3),

[0101] In the above formula (3), d l Let be the duration of the l-th gaze event, and NF be the number of gaze points within the selected time T; then, calculate the mean of pleasant facial expression M_H and the standard deviation of neutral expression SD_N, using the following formula:

[0102] (4),

[0103] (5),

[0104] In the above formulas (4)-(5), H P Let N be the value of a happy expression at the p-th time point, where n is the total number of time points within the valid time period. P Let M_N be the neutral expression value at the p-th time point, and M_N be the mean of the neutral expression.

[0105] The physiological-based trust measurement submodule calculates the EEG, near-infrared, eye movement, pupil, and ECG data collected in the multi-source synchronous acquisition and labeling module to output physiological-based trust features.

[0106] In detail, the ECG, eye movement and pupillary data, EEG, and near-infrared data output by the multi-source synchronous acquisition and labeling module are first calculated. The formula for calculating the standard deviation of the normal RR interval of ECG using SDNN is as follows:

[0107] (6),

[0108] In the above formula (6), This is the value of the p-th RR interval. Let N be the mean of all RR intervals, and N be the total number of RR intervals. Then, the formula for calculating the total power TP of the electrocardiogram is:

[0109] (7),

[0110] In the above formula (7), TP is the total power. Here is the power spectral density of the electrocardiogram (ECG) signal; the formula for calculating the high-frequency ratio (HF_ratio) of the ECG signal is:

[0111] (8),

[0112] In the above formula (8), the low-frequency power High-frequency power ;

[0113] For eye movement pupil features, calculate the pupil diameter (AWFPD) at all fixation points;

[0114] For EEG characteristics, the formulas for calculating the δ relative power δ_r and the α relative power α_r are:

[0115] (9),

[0116] (10)

[0117] In the above formulas (9)-(10), Let be the power spectral density of the EEG signal; and from this, the δ relative power δ_r_CZ at the CZ electrode point, the δ relative power δ_r_C4 at the C4 electrode point, the δ relative power δ_r_P4 at the P4 electrode point, the α relative power α_r_CZ at the CZ electrode point, and the α relative power α_r_C4 at the C4 electrode point can be obtained respectively;

[0118] Finally, the changes in near-infrared oxygenated hemoglobin concentration (O2Hb) and deoxygenated hemoglobin concentration (HHb) are calculated using the following formula:

[0119] (11),

[0120] (12),

[0121] In the above formulas (11)-(12), This represents the absorbance change at wavelength λ. Let be the molar extinction coefficient of O2Hb at wavelength λ. Let λ be the molar extinction coefficient of HHb at wavelength λ, L be the distance between the light source and the probe, DPF be the optical path diffusion coefficient, and λ1 and λ2 be the two wavelengths used by the near-infrared device. From this, the changes in oxygenated hemoglobin concentration O2Hb_R2_T3 and deoxygenated hemoglobin concentration HHb_R2_T3 in the R2_T3 channel can be obtained respectively.

[0122] The correlation analysis module includes a sub-module for correlation analysis between self-report and behavioral characteristics, and a sub-module for correlation analysis between self-report and physiological characteristics;

[0123] The self-report and behavioral characteristic correlation analysis submodule is used to perform correlation analysis on the obtained self-reported trust characteristics and behavior-based trust characteristics, and outputs the correlation coefficient r between the self-reported trust characteristics and the behavior-based trust characteristics. J,Y and r M,Y The detailed calculation formula is as follows:

[0124] (13)

[0125] (14)

[0126] (15)

[0127] In the above formulas (13)-(15), r XY Let X be the Pearson correlation coefficient between variables X and Y. i Let Y be the i-th observation of variable X. i Let Y be the i-th observation value. For the sample size, Let X be the sample mean. Let Y be the sample mean of the variable Y;

[0128] The self-report and physiological characteristic association analysis submodule is used to perform correlation analysis on the obtained self-reported trust characteristics and physiologically based trust characteristics, and outputs the correlation coefficient r between the self-reported trust characteristics and the physiologically based trust characteristics. J,Y' and r M,Y' The detailed calculation formula is as follows:

[0129] (16)

[0130] (17)

[0131] In the above formulas (16)-(17), Let X and X be variables. The Pearson correlation coefficient between them For variables The i-th observation, For variables The sample mean, X i Y i , The definition is consistent with formula (15).

[0132] Example 2

[0133] like Figure 2-5 As shown, a multi-source fusion measurement method for human-AI collaboration trust is presented, with the following workflow:

[0134] S1, on the initial homepage interface ( Figure 6 After selecting the "Instructions for Use" option, you can enter the system instructions interface. Figure 7 In human-AI collaborative flight scenarios, it provides output of multi-source synchronously labeled and trusted data collection data;

[0135] S1-1, the multi-source synchronous acquisition and marking module sends a marker by starting a synchronization program on the near-infrared computer. Through synchronization hardware connections to the near-infrared computer, the eye tracker's volume port, and the EEG amplifier's event button interface, it achieves synchronous acquisition of near-infrared, eye tracker, and EEG data, storing it in the computer's internal storage medium (the computer integrates a human-AI collaborative trust multi-source fusion measurement system, and...). Figure 8-10 (Synchronize the interface display)

[0136] S1-2. Manually label ECG, facial expression, and trust scale score data;

[0137] S1-3, Output multi-source synchronously labeled near-infrared, EEG, attentional monitoring behavior, facial expression behavior, eye movement and pupillary data, ECG, and trust scale scores (e.g.) Figure 8 (as shown)

[0138] S2. For the multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement and pupil, ECG, and trust scale score data read in step S1-1, multi-source data fusion measurement and feature calculation are performed through the fusion measurement module to obtain 17 trust measurement sensitive features, which are then displayed on the display interface (e.g., ...). Figure 9 (as shown)

[0139] S2-1. The trust scale data collected in step S1 is calculated by the self-report-based trust measurement submodule to obtain two self-reported trust characteristics, which are then displayed on the display interface for export.

[0140] S2-1-1. Calculate the total score J of the Jian scale;

[0141] S2-1-2, Calculate the total score M of the Madsen scale;

[0142] S2-1-3. Output and display the self-reported trust characteristics J and M on the display interface;

[0143] S2-2. The behavior-based trust measurement submodule calculates the attention monitoring behavior and facial expression behavior data collected in step S1 to obtain four behavior-based trust features, which are then displayed on the display interface for export.

[0144] S2-2-1. Calculate the number of fixations NF and the total duration TDF of all fixations within the selected time T of the attentional monitoring behavior;

[0145] S2-2-2 Calculate the mean of pleasant facial expression M_H and the standard deviation of neutral facial expression SD_N;

[0146] S2-2-3, Output the behavior-based trust features NF, TDF, M_H and SD_N;

[0147] S2-3. The EEG, near-infrared, eye movement, pupil, and ECG data collected in step S1 are calculated using the physiological-based trust measurement submodule to obtain 11 physiological-based trust features, which are then displayed on the display interface for export.

[0148] S2-3-1. Calculate three ECG characteristics, including the standard deviation of normal RR interval (SDNN), total power (TP), and high-frequency ratio (HF_ratio).

[0149] S2-3-2. Calculate one eye movement pupillary feature, namely the pupillary diameter (AWFPD) at all fixation points;

[0150] S2-3-3 Calculate five EEG pupillary features, including the relative delta power δ_r_CZ at the CZ electrode point, the relative delta power δ_r_C4 at the C4 electrode point, the relative delta power δ_r_P4 at the P4 electrode point, the relative alpha power α_r_CZ at the CZ electrode point, and the relative alpha power α_r_C4 at the C4 electrode point.

[0151] S2-3-4. Calculate 5 near-infrared features, including the changes in oxygenated hemoglobin concentration (O2Hb_R2_T3) and deoxygenated hemoglobin concentration (HHb_R2_T3) in the R2_T3 channel;

[0152] S2-3-5. Output and display the physiological trust features SDNN, TP, HF_ratio, AWFPD, δ_r_CZ, δ_r_C4, δ_r_PZ, α_r_CZ, α_r_C4, O2Hb_R2_T3 and HHb_R2_T3 on the display interface.

[0153] S3. Input the calculated value from step S2 into the association analysis module, calculate and output the association analysis results between self-report and behavior and physiological characteristics;

[0154] S3-1. Calculate and output the correlation coefficient between the self-reported trust features J and M output from S2-1-3 and the behavior-based trust features NF, TDF, M_H and SD_N output from S2-2-3, showing the correlation between the two.

[0155] S3-1-1 Calculate the correlation coefficient r between self-reported trust characteristics and behavior-based trust characteristics. J,Y and r M,Y ;

[0156] S3-1-2. Output and display the correlation coefficient r between self-reported trust characteristics and behavior-based trust characteristics on the interface. J,Y and r M,Y ;

[0157] S3-2. Using the self-reported trust features J and M output from S2-1-3, and the physiologically based trust features SDNN, TP, HF_ratio, AWFPD, δ_r_CZ, δ_r_C4, δ_r_PZ, α_r_CZ, α_r_C4, O2Hb_R2_T3, and HHb_R2_T3 output from S2-3-5, calculate and output the correlation coefficient r between the two. J,Y' and r M,Y' ;

[0158] S3-2-1. Calculate the correlation coefficient r between self-reported trust characteristics and physiologically based trust characteristics. J,Y' and r M,Y'；

[0159] S3-2-2, Output and display the correlation coefficient r between self-reported trust characteristics and physiologically based trust characteristics on the interface. J,Y' and r M,Y' (like Figure 10 (As shown).

[0160] Example 3

[0161] An electronic device, comprising:

[0162] At least one processor;

[0163] A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the at least one processor to perform the above-described method.

[0164] Example 4

[0165] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method;

[0166] In this embodiment, the computer-readable storage medium may be a USB flash drive or other systems or devices that are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof.

[0167] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-source fusion measurement system for human-AI trust, characterized in that: It includes a multi-source synchronous acquisition and tagging module for providing multi-source synchronous tagging trust acquisition data output, a fusion measurement module for supporting multi-source data fusion measurement and feature calculation, and an association analysis module for conducting self-reporting and behavioral and physiological characteristic correlation analysis. The integrated measurement module includes a self-report-based trust measurement submodule, a behavior-based trust measurement submodule, and a physiological-based trust measurement submodule; The self-report-based trust measurement submodule calculates the trust scale data collected in the multi-source synchronous acquisition and labeling module and outputs the self-reported trust characteristics. The behavior-based trust measurement submodule calculates the attention monitoring behavior and facial expression behavior data collected in the multi-source synchronous acquisition and labeling module, and outputs behavior-based trust features. The physiological-based trust measurement submodule calculates the EEG, near-infrared, eye movement, pupil, and ECG data collected in the multi-source synchronous acquisition and labeling module, and outputs physiological-based trust features. The correlation analysis module includes a sub-module for correlation analysis between self-report and behavioral characteristics, and a sub-module for correlation analysis between self-report and physiological characteristics; The self-report and behavioral feature correlation analysis submodule is used to perform correlation analysis between the obtained self-reported trust features and the behavior-based trust features, and output the correlation coefficient between the self-reported trust features and the behavior-based trust features. The self-report and physiological characteristic correlation analysis submodule is used to perform correlation analysis between the obtained self-reported trust characteristics and physiologically based trust characteristics, and output the correlation coefficient between the self-reported trust characteristics and the physiologically based trust characteristics. The calculation steps for the self-report-based trust measurement submodule to output the self-reported trust characteristics are as follows: The trust scale data output by the multi-source synchronous acquisition and labeling module is calculated, and the formula for calculating the total score J of the Jian scale is as follows: ； In the above formula, J i For the score of positive sub-dimension i of the Jian scale, J j The score of the negative sub-dimension j of the Jian scale; Then, calculate the total score M of the Madsen scale using the following formula: ； M k The score for sub-dimension k of the Madsen scale; The behavior-based trust measurement submodule is used to output the behavior-based trust features through the following calculation steps: The formula for calculating the total duration of attention (TDF) at all fixations is as follows: The attention monitoring behavior and facial expression behavior data output by the multi-source synchronous acquisition and labeling module are calculated. ； In the above formula, d l Let be the duration of the l-th gaze event, and NF be the number of gaze points within the selected time T; Then, calculate the mean M_H of pleasant facial expressions and the standard deviation SD_N of neutral facial expressions, using the following formula: ； ； In the above formula, H P Let N be the value of a happy expression at the p-th time point, where n is the total number of time points within the valid time period. P Let M_N be the neutral expression value at the p-th time point, and M_N be the mean of the neutral expression. The physiological-based trust measurement submodule is used to output the calculation steps of physiological-based trust features as follows: First, the ECG, eye movement and pupillary data, EEG, and near-infrared data output by the multi-source synchronous acquisition and labeling module are calculated. The formula for calculating the standard deviation of the normal RR interval in ECG using SDNN is as follows: ； In the above formula, This is the value of the p-th RR interval. Let N be the mean of all RR intervals, and N be the total number of RR intervals. Then, the formula for calculating the total power TP of the electrocardiogram is: ； In the above formula, TP represents the total power. Let df be the power spectral density of the electrocardiogram signal, and df be the derivative of the frequency variable f, i.e., the infinitesimal change. The formula for calculating the high-frequency ratio (HF_ratio) in an electrocardiogram (ECG) is: ； In the above formula, low-frequency power High-frequency power ; For eye movement pupil features, calculate the pupil diameter (AWFPD) at all fixation points; For EEG characteristics, the formulas for calculating the δ relative power δ_r and the α relative power α_r are: ； ； In the above formula, The power spectral density of the EEG signal; And from this, the δ relative power δ_r_CZ at electrode CZ, the δ relative power δ_r_C4 at electrode C4, the δ relative power δ_r_P4 at electrode P4, the α relative power α_r_CZ at electrode CZ, and the α relative power α_r_C4 at electrode C4 can be obtained respectively; Finally, the changes in near-infrared oxygenated hemoglobin concentration (O2Hb) and deoxygenated hemoglobin concentration (HHb) are calculated using the following formula: ； ； In the above formula, This represents the absorbance change at wavelength λ. Let be the molar extinction coefficient of O2Hb at wavelength λ. Let λ be the molar extinction coefficient of HHb at wavelength λ, L be the distance between the light source and the probe, DPF be the optical path diffusion coefficient, and λ1 and λ2 be the two wavelengths used by the near-infrared device. From this, the changes in oxygenated hemoglobin concentration O2Hb_R2_T3 and deoxygenated hemoglobin concentration HHb_R2_T3 in the R2_T3 channel can be obtained respectively.

2. The multi-source fusion measurement system for human-AI trust according to claim 1, characterized in that: The multi-source synchronous acquisition marker module sends a marker by starting a synchronization program on the near-infrared computer, and connects to the near-infrared computer, the volume port of the eye tracker, and the event button interface of the EEG amplifier via synchronization hardware to achieve synchronous acquisition of near-infrared, eye tracker, and EEG data. The ECG, facial expression, and trust scale scores were manually labeled; finally, multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement and pupil, ECG, and trust scale scores were collected and output.

3. The multi-source fusion measurement system for human-AI trust as described in claim 1, characterized in that: The self-report and behavioral characteristic correlation analysis submodule outputs the correlation coefficient r between the trust characteristics of self-report and the trust characteristics based on behavior. J,Y and r M,Y The calculation formula is: ； ； ； In the above formula, r XY Let X be the Pearson correlation coefficient between variables X and Y. i Let Y be the i-th observation of variable X. i Let Y be the i-th observation value. For the sample size, Let X be the sample mean. Let y be the sample mean of variable Y, SX be the standard deviation of variable X, and SY be the standard deviation of variable Y.

4. The multi-source fusion measurement system for human-AI trust according to claim 3, characterized in that: The self-report and behavioral characteristic correlation analysis submodule outputs the correlation coefficient r between the trust characteristics of self-reported data and the physiologically based trust characteristics. J,Y' and r M,Y' The calculation formula is: ； ； In the above formula, Let X and X be variables. The Pearson correlation coefficient between them For variables The i-th observation, For variables The sample mean.

5. A measurement method for a multi-source fusion measurement system for human-AI trust as described in any one of claims 1-4, characterized in that: Includes the following steps: S1 provides output of multi-source synchronously labeled and trusted data collection in human-AI collaborative flight scenarios; S2. For the multi-source synchronously labeled near-infrared, EEG, attention monitoring behavior, facial expression behavior, eye movement and pupil, ECG and trust scale score data read in step S1, multi-source data fusion measurement and feature calculation are performed through the fusion measurement module. S3. After inputting the calculated value from step S2 into the association analysis module, calculate and output the association analysis results between self-report and behavior and physiological characteristics.

6. An electronic device, characterized in that, include: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which, when executed by the at least one processor, causes the at least one processor to perform the method as described in claim 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in claim 5.

Images