A dynamic trust measurement method based on human-computer collaborative task

Abstract

This invention discloses a dynamic trust measurement method based on human-computer collaborative tasks, belonging to the field of human-computer interaction trust assessment technology. The method includes constructing a human-computer collaborative task experimental platform, driven by decision-making tasks and equipped with an intelligent auxiliary system providing decision suggestions; conducting resource exploration and decision-making on the human-computer collaborative task experimental platform in S1, where participants combine prior knowledge and intelligent system suggestions to make decisions, and iterating multiple times with the goal of maximizing benefits; recruiting a predetermined number of participants based on the human-computer collaborative task experimental platform in S1 to perform human-computer interaction collaborative tasks, and collecting prior experimental data and interaction experimental data; and obtaining the prior experimental data and interaction experimental data obtained in S3 to determine the results of human-computer collaborative tasks under different conditions. This dynamic trust measurement method based on human-computer collaborative tasks can provide constraints for the design specifications of intelligent systems in related human-computer collaboration fields and provide prior warnings for trust repair.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction trust assessment technology, specifically a dynamic trust measurement method based on human-computer collaborative tasks. Background Technology

[0002] Against the backdrop of the rapid development of artificial intelligence (AI) technology, AI has been widely applied across various industries, greatly improving work efficiency and content quality. However, as AI and humans collaborate more closely, the issue of trust in AI systems has gradually emerged during human-machine collaboration. Trust is not only the cornerstone of the successful application of AI technology but also an important prerequisite for the smooth implementation of human-machine collaboration. In order to effectively establish and maintain human trust in AI systems, it is necessary to find appropriate methods to calculate human-machine trust indicators and summarize the changing patterns of AI system capabilities and trust levels.

[0003] For example, Chinese patent application No. 202410615849.3, filed on May 17, 2024, describes a human-computer collaborative platform interface design and evaluation method based on transparency levels. This method optimizes the efficiency and quality of information transmission by finely controlling the transparency of information displayed on the interface, thereby improving the operator's trust in the platform, the accuracy and security of task operations, and reducing the operator's cognitive load.

[0004] As new human-machine collaborative systems continue to emerge, different users have different trust thresholds, system capabilities, and acceptable trust ranges. Therefore, it is crucial to conduct capability-trust mapping tests before the system is put into operation to ensure system reliability and user trust. Based on this, we can further guide the optimization of system design and the constraint specifications of capability performance. Currently, there is a lack of effective system testing methods around dynamic human-machine trust. Therefore, we propose a new experimental paradigm. Summary of the Invention

[0005] The purpose of this invention is to provide a dynamic trust measurement method based on human-machine collaborative tasks, so as to solve the problem of lack of effective system testing methods for dynamic human-machine trust mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a dynamic trust measurement method based on human-machine collaborative tasks, comprising:

[0007] S1. Construct a human-machine collaborative task experimental platform. The platform is driven by decision-making tasks and has a built-in intelligent auxiliary system to provide decision-making suggestions.

[0008] S2. Resource exploration and decision-making are carried out on the human-machine collaborative task experimental platform in S1. The subjects make decisions by combining prior knowledge and suggestions from the intelligent system, and make multiple decision-making iterations with the goal of maximizing benefits.

[0009] S3. Based on the human-computer collaboration task experimental platform in S1, recruit a predetermined number of subjects to perform human-computer interaction collaboration tasks and collect prior experimental data and interaction experimental data.

[0010] S4. Based on the prior experimental data and interactive experimental data obtained in S3, we can obtain the rules of human compliance with AI suggestions under different conditions of human-computer collaborative tasks. The different conditions include different AI capabilities, different prior knowledge, and different experimental durations.

[0011] Preferably, S1 specifically includes the following processes:

[0012] S11. Clarify the types and complexity of human-computer collaboration tasks that the platform needs to support. Select binary decision-making tasks for human-computer collaboration tasks. This method can simplify the decision-making process, facilitate quantitative analysis and control of experimental variables, simulate real-world scenarios, improve user acceptance and data reliability, and make the research results more reproducible and practical, thereby more accurately assessing the level of user trust in machines and its dynamic changes.

[0013] S12. The platform's built-in intelligent assistance system dynamically generates decision suggestions based on system capability settings and external information constraints of decision rounds. When working, the platform's built-in intelligent assistance system adaptively adjusts its capability level through real-time assessment of user feedback and task complexity to flexibly respond to different decision-making environments. To ensure the accuracy and timeliness of decisions, the platform's built-in intelligent assistance system has the ability to manage external information for each round of decision-making, ensuring that decisions are always based on the latest data. Through advanced thinking chain technology, the platform's built-in intelligent assistance system can simulate human thinking processes, providing logically rigorous and coherent reasoning and suggestions. At the same time, the platform's built-in intelligent assistance system uses prompting technology to guide users to clearly express their needs, and combined with content formatting functions, ensures that the output information is both professional and easy to understand, meeting the diverse needs of different users and scenarios. The synergistic effect of these functions makes the intelligent assistance system a powerful tool to support efficient and accurate decision-making.

[0014] S13. Analyze the specific needs of dynamic trust measurement, clarify the construction of measurement indicators and feedback mechanisms. The experimental platform pays special attention to three key trust assessment dimensions: pre-exposure trust, real-time trust, and post-exposure trust. Before the experiment begins, participants need to complete an initial trust rating questionnaire to assess their initial level of trust in the experiment. During the experiment, participants need to make a quantitative trust rating of 0 to 100 points after each decision to reflect their trust changes in real time. After the experiment, participants will fill out a trust end rating questionnaire to assess their trust feelings throughout the entire experimental process.

[0015] Preferably, S2 specifically includes the following processes:

[0016] S21. Design the materials required for collaborative task experiments, and develop experimental tasks on the experimental platform to simulate a real experimental environment;

[0017] S22. Design a script for a human-computer collaboration task experiment, and preset the number of experiments to be completed and related parameters such as experiment settings;

[0018] S23. Construct an experimental paradigm to explore the level of trust participants have in AI assistants. The experiment will provide participants with prior knowledge to learn decision-making experience. In each actual decision-making trial, relevant external information will be generated for participants' reference, and the intelligent assistance system will also provide decision-making suggestions. The intelligent assistance system will dynamically adjust the accuracy of its suggestions based on prior knowledge conditions and participants' experimental feedback, thereby repairing participants' trust. A series of scenarios will be designed, changing the AI ​​assistant's ability level (high / low), the participants' prior knowledge strength (high / medium / no knowledge), and the binary decision-making suggestions provided by the AI ​​assistant. Participants will make choices in 12 different scenarios and evaluate their trust in the AI ​​assistant. By collecting and analyzing data such as answer accuracy, decision-making time, compliance, and trust scores, the experiment will explore how humans establish and adjust their trust in AI assistants under different conditions.

[0019] Preferably, S3 specifically includes the following processes:

[0020] S31. Before the experiment, the participants fill in their personal information and complete a questionnaire to understand their knowledge and use of AI. After completing the questionnaire, they enter the experiment prompt interface.

[0021] S32. Each participant reads the experimental instructions and then fills out the AI ​​Trust Initial Rating Questionnaire to quantify the participant's prior trust.

[0022] S33. After completing the prior trust test, participants will enter the practice phase. This phase aims to assist them in judging the suggestions of artificial intelligence by learning prior knowledge. In this phase, participants will be exposed to various types of prior knowledge. The goal is to help them understand and master the relationship between this knowledge and available resources. Participants need to evaluate and judge the provided prior knowledge within a specified time. The system will provide immediate feedback based on their judgments. Only if they answer correctly four times in a row will the participants be considered to have fully understood the connection between prior knowledge and resources and be eligible to enter the next stage of the experimental process.

[0023] S34. After completing the pre-set exercises, the participants enter the decision-making experiment stage. After the decision-making experiment, the participants fill out the AI ​​trust end rating questionnaire to quantify the participants' post-event trust.

[0024] Preferably, S34 specifically includes the following processes:

[0025] S341. The human-computer collaboration task experiment platform displays background information and AI suggestion information. The background information and AI suggestion information are calculated by the system. Participants need to choose between two options based on the information provided. This process will last for a preset period of time. The background information, i.e., prior knowledge, is calculated by the system through a weighted summation based on the strength of the participants' prior knowledge and the accuracy of their decisions in the previous rounds of experiments. The AI ​​suggestion information is adjusted based on the number of times the AI ​​suggestion matches the actual resource information in the previous rounds of experiments. In order to more realistically reflect the fluctuations in the participants' trust and more easily detect the patterns of trust, the system will change the AI ​​suggestion information after it has matched several times in a row, so that it no longer matches, thereby introducing the volatility of trust.

[0026] S342. The human-computer collaborative task experimental platform displays selection feedback to let the participants know whether their selection is correct. The participants are ranked and given feedback every 10 experiments to provide them with motivation. The selection feedback interface lasts for the second duration, and the ranking feedback interface lasts for the third duration.

[0027] S343. The human-computer collaboration task experiment platform displays a trust rating interface, and the subjects give a trust rating to the AI ​​after each experiment.

[0028] Preferably, participants with experience using electronic devices and a history of exposure to artificial intelligence products will be selected. Each participant will participate in two rounds of experiments, each round consisting of 60 decision-making tasks. In these tasks, the accuracy rate of the AI's suggestions will be designed to vary to simulate uncertainty in the real world. Specifically, each round of experiments will be divided into three parts, with 20 tasks in each part, corresponding to high, medium, and low prior knowledge levels, respectively. This design aims to assess the participants' trust in and reliance on AI suggestions under different knowledge backgrounds.

[0029] Preferably, S4 specifically includes the following processes:

[0030] S41. Prior experimental data processing: The human-machine collaborative task experimental platform collects questionnaire data filled out by the subjects before the experiment and analyzes the subjects' main components, their understanding of AI, and their trust in AI.

[0031] S42. The human-machine collaborative task experimental platform collects prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data. The prior information is generated by the system based on previous answer feedback. The judgment time is obtained by a timer, which represents the time interval between the appearance of the selection interface and the moment the subject triggers the selection button. Experimental data of subjects who timed out are removed. Then, a mixed method analysis is performed on the prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data, including main effect analysis and interaction analysis for each indicator.

[0032] Ideally, a professional user experience expert team should be assembled, consisting of two interaction designers with over five years of industry experience and four experts with deep expertise in human-computer trust. The team will be responsible for planning and executing each round of experiments, ensuring that the number of decision-making experiments is set reasonably. In addition, the team will conduct a comprehensive evaluation of the AI ​​assistant's performance under different capability values ​​and determine its performance standards accordingly.

[0033] A dynamic trust measurement method based on human-computer collaborative tasks, the dynamic trust measurement method using a dynamic trust evaluation system based on human-computer collaborative tasks, including the collaborative task experimental platform;

[0034] The collaborative task experimental platform includes a user interface simulating collaborative tasks, and further includes:

[0035] The parameter setting module is used to set the AI ​​capabilities, feedback duration, selection duration, and prior knowledge strength in collaborative tasks.

[0036] The experiment execution module is used to execute human-computer collaborative task experiments and collect prior experimental data and interactive experimental data.

[0037] The data analysis module is used to analyze the dynamic changes and patterns of trust during the human-computer collaboration process based on the prior experimental data and interactive experimental data obtained from the experimental execution module.

[0038] Compared with the prior art, the beneficial effects of the present invention are: it enables the use of a human-computer collaborative task simulation system that makes decisions based on AI suggestions and prior knowledge; it allows for the use of this system to conduct user experiments, design collaborative task experiments based on AI suggestions and prior knowledge, explore the changing patterns of dynamic trust between humans and machines, and provide an example for evaluating the user trust range and system capability specifications in human-computer collaborative systems. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the dynamic evaluation system for human-machine collaborative tasks according to the present invention;

[0040] Figure 2 This is a schematic diagram of the dynamic trust assessment experiment of the present invention, taking a resource simulation exploration task as an example;

[0041] Figure 3 This is a schematic diagram of the experimental process of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the expected interactive experimental results of the present invention without considering prior knowledge.

[0043] Figure 5 This is a schematic diagram illustrating the results of an interactive experiment of the present invention when the AI ​​assistant's suggestion is contrary to the subject's prior knowledge expectation;

[0044] Figure 6 This is a schematic diagram illustrating the results of an interactive experiment of the present invention when the AI ​​assistant's suggestion is the same as the subject's prior knowledge expectation;

[0045] Figure 7 This is a schematic diagram illustrating 12 possible scenarios of the game theory in this invention;

[0046] Figure 8 This is a schematic diagram of the single-sample t-test results of the present invention;

[0047] Figure 9 This is a schematic diagram illustrating the initial expected conditions of the T-Test in this invention;

[0048] Figure 10 This is a schematic diagram of the T-Test results of the present invention;

[0049] Figure 11 This is a schematic diagram illustrating the T-TestAI recommendations of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] This invention provides the following technical solution: a dynamic trust measurement method based on human-machine collaborative tasks. It includes the following steps:

[0052] (1) Construct a human-machine collaborative task experimental platform to realize the custom adjustment of prior information strength and AI capabilities, and adjust the platform parameters to achieve efficient and diversified collaborative experiments;

[0053] (2) For human-machine collaborative tasks, a future task of human and AI jointly exploring lunar resources was designed. Under the condition of given certain mineral point information and AI suggestion information, the exploration intensity is selected, and points are given as feedback based on the selection result.

[0054] (3) Conduct lunar resource exploration simulation experiments in conjunction with the experimental platform in step (1), clarify the human-machine collaborative task experimental process including recruiting subjects, pre-experiment, pilot learning, practice experiment, and formal experiment, and collect experimental platform data;

[0055] (4) Summarize the changing pattern of human-computer dynamic trust based on the prior experimental data and interactive experimental data obtained in step (3).

[0056] The specific process is as follows:

[0057] (1) Develop a human-computer collaborative task experimental platform

[0058] (11) The experimental platform was developed using C# and the Unity engine, running on a Surface Pro 6 (Intel Core i5-8350U, 8GB). The Unity engine was used to simulate the real experimental environment, and C# was used to implement task logic and data processing. GPT-3.5 was integrated into the platform through API interfaces to provide real-time decision support, such as... Figure 1 As shown.

[0059] (12) Set the prior information intensity of the experiment. The prior information intensity represents the reliability of the prior information. The higher the prior information intensity, the greater the probability that the prior information is correct; conversely, the lower the prior information intensity, the lower the probability that the prior information is correct.

[0060] The strength of prior information is set dynamically, and its value is related to the results of previous consecutive decision-making experiments. The formula for calculating the strength of prior information can be simplified as follows:

[0061]

[0062] ACC indicates whether the choice is correct, with 1 indicating correct and 0 indicating incorrect; the prior information strength of the current round of experiment is the weighted sum of the decision correctness of the previous k rounds and the prior information strength.

[0063] (13) Set the AI ​​assistant capability value and the probability of correctness of each AI assistant suggestion: The AI ​​assistant capability value represents the accuracy of the AI ​​assistant in the whole round of experiment. For example, if there are 60 selection experiments in a round of experiment, then when the AI ​​assistant capability value is 80%, it means that in the 60 selection experiments, the AI ​​assistant suggestion is correct in 48 times and incorrect in the other 12 times.

[0064] The accuracy setting for AI assistant suggestions is also dynamic, related to the previous consecutive selections of experiments and the AI ​​assistant's capability value. Its calculation formula can be simplified as follows:

[0065]

[0066] ACC indicates whether the choice is correct, with 1 indicating correct and 0 indicating incorrect. The accuracy of the AI ​​assistant's suggestion in the current round of the experiment is a weighted sum of the decision accuracy of the previous k rounds, the strength of prior information, and the accuracy of the AI ​​assistant's suggestion.

[0067] In addition, the system will also adjust based on the AI ​​assistant's capabilities. If the accuracy of the AI ​​assistant's suggestions reaches the AI ​​assistant's capabilities, the AI ​​assistant will provide reverse suggestions in the future.

[0068] (14) Construct prompt word templates and construct the LLM model thinking chain so that the AI ​​assistant can provide suggestions and explanations when the subjects' trust in it decreases, thereby increasing the subjects' trust in it.

[0069] Design prompt word templates, which include mineral information, user decision information, user rating information, etc. Construct prompt word instances based on prompt word templates to guide the LLM model to think in a chain-like manner. The model will learn how to deduce conclusions step by step from the input information and generate coherent explanations.

[0070] The prompt word templates are shown in the table below:

[0071]

[0072]

[0073] The following table shows examples of prompt word templates:

[0074]

[0075]

[0076] (2) Designing human-computer collaboration tasks and creating experimental scripts

[0077] (21) Design experimental instructions. The experimental instructions introduce the background information of the experiment and introduce some prior knowledge to the subjects. In this example, the connection between mineral information, lunar landform information and large and small mineral points, as well as the accuracy of AI suggestions will be introduced.

[0078] The mineral element Chang'e Stone A and the lunar landform Dragon Pattern Landform 2 are both highly likely to be associated with large mineral deposits; the mineral element Lunar Basalt D and the lunar lowland landform 3 are both highly likely to be associated with small mineral deposits; the mineral element Noblel B or Lunar Breccia C and the lunar highland landform 1 or Crater 4 are unrelated to the size of the mineral deposit and are considered interfering information. There are a total of 16 combinations of mineral elements and lunar landforms: A1~4, B1~4, C1~4, D1~4.

[0079] Participants start with 1000 mineral energy points. Deep exploration consumes 100 mineral energy points, while shallow exploration consumes 10. Large mineral deposits, located deeper, grant 200 energy points with deep exploration but none with shallow exploration; small mineral deposits, located shallower, grant 20 energy points with shallow exploration but none with deep exploration.

[0080] (22) Design an initial trust quantification questionnaire. The initial trust quantification questionnaire is used to assess and quantify the subjects' attitudes toward the AI ​​assistant. The experiment can only proceed to the next stage after the subjects have answered all the questions.

[0081] The initial trust quantification questionnaire consists of 6 questions, each rated from 1 to 5, where a score of 1 indicates strong disagreement and a score of 5 indicates strong agreement.

[0082] A1. I believe this AI is capable of performing its task;

[0083] A2. I trust this AI;

[0084] A3. I have full confidence in the suggestions given by AI;

[0085] A4. I can rely on this AI;

[0086] A5. I believe AI will maintain a consistent operating method;

[0087] A6. I believe that every time I follow the AI's advice, the AI ​​will do its best.

[0088] After completing the initial trust questionnaire, participants were required to rate their trust in the AI ​​assistant, with a rating range of 0 to 100.

[0089] (23) Design a prior knowledge learning experiment to allow the subjects to understand and learn the prior knowledge that they will encounter in the experiment. The experiment continues until the subjects understand the prior knowledge.

[0090] Subjects read information about mineral deposits on an interface, including: an overhead view of the deposit, discovery time, geographical location, mineral elements, and landform features. They then had a preset time to determine whether the deposit was more likely to be a large or small deposit.

[0091] After each selection, participants are shown a feedback screen indicating whether their choice was correct. Participants will only proceed to the next stage of the experiment after answering four questions correctly in a row.

[0092] (24) Design a decision-making experiment, which is the formal experiment in the collaborative task experiment. In addition to the mineral point information from the prior knowledge learning experiment, the decision-making experiment also includes the AI's image and suggestion information. Furthermore, the binary decision involves consuming 100 energy points for deep exploration and consuming 10 energy points for shallow exploration. Participants must complete their selection within a preset time; otherwise, they will be considered to have exceeded the time limit and will be deducted 100 energy points. The decision-making experiment will run for 60 rounds. The bottom of the decision-making experiment interface displays a gray-background green progress bar to let participants know the progress of the experiment.

[0093] The decision-making interface displays an AI avatar and suggestions on the left, a selection box with two options in the middle, and mineral resource information on the right. To prevent accidental selections, a confirmation button appears below the selection box after a participant selects an option. Confirming the selection leads to a decision feedback interface informing the participant of their choice. This feedback interface lasts for a preset time before proceeding to a trust rating screen.

[0094] The trust rating interface features a blue background and a green slider with scale indicators below it, showing values ​​of 0, 25, 50, 75, and 100. Participants cannot proceed to the next screen if they do not trigger the slider.

[0095] After every 10 decision-making experiments, participants will be given their current energy point ranking and a curve showing the change in energy points over the past 10 experiments, which will serve as an incentive. Participants need to trigger the "Continue Experiment" button to proceed to the next experiment.

[0096] During the decision-making process, participants engage in a game with the AI ​​assistant based on prior knowledge provided by the interface. The game involves 12 possible scenarios, such as... Figure 7 As shown.

[0097] (25) Design a post-exposure trust quantification questionnaire to quantify the subject’s post-exposure trust. The subject can proceed to the next stage of the experiment only after completing all the questions.

[0098] This quantitative questionnaire is displayed on two pages. You can only proceed to the next page after completing the questions on the previous page.

[0099] The Trust Quantification Questionnaire consists of 12 questions, each rated from 1 to 7, with a score of 1 indicating strong disagreement and a score of 7 indicating strong agreement.

[0100] B1. This AI's suggestion is deceptive;

[0101] B2. The way this AI works is unpredictable;

[0102] B3. I have doubts about AI's suggestions;

[0103] B4. I am very cautious about the suggestions provided by this AI;

[0104] B5. I think using this AI's suggestions will lead to bad results;

[0105] B6. I have full confidence in the suggestions provided by this AI;

[0106] B7. I feel safer using the suggestions provided by this AI.

[0107] B8. I think this AI is trustworthy;

[0108] B9. I think this AI's suggestion is very reliable;

[0109] B10. I think this AI's suggestion is very credible;

[0110] B11. I can trust the advice provided by this AI;

[0111] B12. I am very familiar with this type of AI;

[0112] (3) Conduct human-machine collaborative mission experiments for lunar resource exploration using the experimental platform in step (1).

[0113] The experimental procedure clearly includes a pre-experiment questionnaire, prior knowledge learning experiments, practice experiments, and the formal human-computer collaborative task experiment. It also includes data collection from the experimental platform. The experimental procedure is as follows: Figure 3 .

[0114] Recruitment of participants: A total of 60 college students were recruited, including 30 males and 30 females (mean age = 23.86, age range = 2.63 years). These participants were recruited through a campus advertising network. All participants had experience using electronic devices. The research protocol was approved by the university ethics committee, and all participants signed consent forms before participating in the research.

[0115] The questionnaire survey statistically analyzed the participants' use of AI and their level of trust in AI assistants.

[0116] The experiment consists of prior knowledge learning and practice experiments. Prior knowledge learning experiments help participants familiarize themselves with the experimental background and understand relevant information about the mineral deposit, facilitating their subsequent decision-making by combining this information with AI suggestions, allowing them to choose between in-depth or shallow exploration. Practice experiments help participants become familiar with the formal experimental procedure, experiencing the entire decision-making process and developing a psychological expectation for the subsequent formal experiment.

[0117] Formal Experiment

[0118] After learning and practicing with prior knowledge, participants enter an initial trust rating interface where they rate their trust in the AI ​​assistant. This is followed by 60 rounds of decision-making experiments. In each decision-making experiment, a decision interface appears initially, displaying for 1000 milliseconds. Participants make a comprehensive decision based on the mineral point information and AI suggestions provided on the decision interface. After completing the decision, a confirmation button is triggered, leading to a result feedback interface that indicates whether the participant's choice was correct and specifies whether the mineral point is large or small. This feedback interface displays for 500 milliseconds before entering the trust rating interface. Participants rate their trust in the AI ​​assistant from 0 to 100, requiring them to provide a score to proceed to the next round. Every 10 rounds, after participants have rated their trust in the AI ​​assistant, a ranking feedback interface appears. This interface displays the participant's energy point change curve from the previous 10 rounds and their current energy point ranking. This ranking is calculated in real-time, and each participant's experimental information is dynamically updated in the ranking database after completion.

[0119] (4) Experimental Results

[0120] (41) Data processing of lunar resource exploration experiments, including experimental collection decision-making time, decision accuracy, degree of AI suggestion compliance, and strength of prior knowledge.

[0121] In this experiment, the decision-making time was acquired using a timer and defined as the interval between the appearance of the decision interface and the moment a participant triggers the confirmation button. The average trigger time for each participant was calculated using SPSS 23.0 statistical analysis software (IBM, New York, USA). To ensure data reliability, extreme values ​​in the search time (values ​​deviating from the mean by ±0.5 standard deviations) were removed. Then, a mixed-methods analysis was performed on the data for decision accuracy, AI suggestion adherence, and prior knowledge strength. Main effect analysis and interaction analysis were conducted for each indicator. In statistical analysis, the main effect refers to the influence of one independent variable on the dependent variable, ignoring the influence of other independent variables. Main effect analysis is used to determine the individual effect of each independent variable on the dependent variable. Interaction analysis means the mutual influence between two or more independent variables, resulting in a change in the dependent variable that is not merely the sum of the effects of individual independent variables. These two analyses allow for a better understanding of the combined impact of the designed independent factors on the experimental results. The proportion of participants who followed the AI ​​assistant's suggestions is shown in the results. Figure 4 The following conclusions can be drawn:

[0122] The main effect of AI assistant ability is significant. The higher the AI ​​assistant ability, the more the subjects follow the AI ​​assistant's advice, and the overall proportion of subjects following the AI ​​assistant's advice matches the AI ​​assistant ability.

[0123] The main effect of prior knowledge strength was not significant, but the interaction effect was significant. When the AI ​​assistant had high capabilities, prior knowledge did not modulate the participants' decisions; when the AI ​​assistant had low capabilities, prior knowledge significantly modulated the participants' decisions.

[0124] The percentage of participants who followed the AI ​​assistant's advice when it contradicted their prior knowledge was as follows: Figure 5 , Figure 5 (a) The smaller the value of the vertical axis, the more it indicates that the participants made decisions based on their prior knowledge of the mineral deposits and did not follow the AI ​​assistant's suggestions, reflecting a lower level of trust in the AI ​​assistant. Figure 5 The following analysis results can be obtained from the data analysis chart in the image:

[0125] The main effect of AI assistant capabilities is significant, which shows that the higher the AI ​​assistant's capabilities, the more the subjects follow the AI ​​assistant's suggestions and the more they trust the AI ​​assistant.

[0126] The main effect of prior knowledge strength was significant, showing that the stronger the prior knowledge of the participants, the smaller the proportion of decisions that followed the suggestions of the AI ​​assistant and the less they trusted the AI.

[0127] The interaction is significant, and the AI ​​assistant's ability effect is greatest when the prior knowledge strength is Medium.

[0128] When the AI ​​assistant's suggestion is the same as the subject's prior knowledge, the percentage of subjects who follow the AI ​​assistant's suggestion is as follows: Figure 6 As shown, when the AI ​​assistant's suggestion is the same as the subject's prior knowledge, the subject is very likely to make a decision according to the AI ​​assistant's suggestion, but not necessarily to make a decision according to the AI ​​assistant's suggestion.

[0129] (42) Dynamic trust data collection and statistical analysis of lunar resource exploration experiments.

[0130] The dynamic trust level of each participant was analyzed using the statistical analysis software JASP 0.19.1. Since trust changes are continuous, snapshot-based measurements cannot reflect this continuous characteristic. Therefore, the change in trust, the independent variable, was defined as the current trial trust level minus the previous trial trust level, i.e., Δtrust = trust. n -trust n-1 Then, the change in trust for each participant was categorized into eight groups based on trial conditions (initial expectation correct / incorrect × AI assistant suggestion correct / incorrect × decision outcome correct / incorrect). One-sample t-tests and paired-sample t-tests were performed for each trial type to explore the impact of different trial conditions on dynamic trust changes. For ease of identification and statistical analysis, the three trial conditions were coded with three-digit codes: 1 for correct and 0 for incorrect. For example, the coded code for correct initial expectation, incorrect AI assistant suggestion, and incorrect decision outcome was 100. Using this method, all trials were divided into eight groups (000, 001, 010, 011, 100, 101, 110, 111). Groups 001 and 110 were discarded due to insufficient data, thus retaining six groups (000, 010, 011, 100, 101, 111) of trust change data under different trial conditions. This method was used to explore and understand the underlying mechanisms of dynamic trust regulation.

[0131] One-sample t-tests of the changes in conditional confidence for each type of trial compared to the baseline (0) are shown in the following results. Figure 8 The following conclusions can be drawn:

[0132] The level of trust change corresponding to each type of trial condition varied significantly. At the same time, the AI ​​assistant's incorrect suggestions had a negative impact on trust change, while its correct suggestions had a positive impact on trust change.

[0133] The paired-samples t-test results for the change in confidence for the three types of trial conditions are as follows: Figure 9 From 10 and 11, we can draw the following conclusions:

[0134] The results of the analysis on the impact of the correctness of the initial expectation on the change in trust are as follows: Figure 9As shown, the correctness of the initial expectation has a significant impact on the change in trust. The change in trust when the initial expectation is wrong is significantly higher than when the initial expectation is correct. However, the correctness of the initial expectation does not significantly affect the asymmetry of the change in trust, and there is no significant difference in the absolute value of the change in trust.

[0135] The results of the analysis on the impact of the accuracy of AI assistant suggestions on the change in trust are as follows: Figure 10 As shown, the correctness of the AI ​​assistant's suggestion has a significant impact on the change in trust. The change in trust caused by the AI ​​assistant's incorrect suggestion is significantly lower than that caused by the initial expectation being correct. At the same time, when the decision result is wrong, the correctness of the AI ​​assistant's suggestion significantly affects the asymmetry of the change in trust. The absolute value of the change in trust when the suggestion is wrong is significantly higher than that when the suggestion is correct, but there is no significant difference when the result is wrong.

[0136] The analysis results regarding the impact of the correctness of decision outcomes on the change in trust are as follows: Figure 11 As shown, the correctness of the decision outcome did not significantly affect the change in trust. At the same time, the correctness of the outcome did not affect the symmetry of the change in trust. No outcome bias was observed in this decision-making task.

[0137] The above is the entire working process of the device, and all contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0138] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic trust measurement method based on human-machine collaborative tasks, characterized in that, Includes the following steps: S1. Construct a human-machine collaborative task experimental platform. The platform is driven by decision-making tasks and has a built-in intelligent auxiliary system to provide decision-making suggestions. S2. Resource exploration and decision-making are carried out on the human-computer collaborative task experimental platform in S1. The subjects make decisions by combining prior knowledge and suggestions from the intelligent system, and make multiple decision-making iterations with the goal of maximizing benefits. S3. Based on the human-computer collaboration task experimental platform in S1, recruit a predetermined number of subjects to perform human-computer interaction collaboration tasks and collect prior experimental data and interaction experimental data. S4. Based on the prior experimental data and interactive experimental data obtained in S3, we can obtain the rules of human compliance with AI suggestions under different conditions of human-computer collaborative tasks. The different conditions include different AI capabilities, different prior knowledge, and different experimental durations. S3 specifically includes the following processes: S31. Before the experiment, the participants fill in their personal information and complete a questionnaire to understand their knowledge and use of AI. After completing the questionnaire, they enter the experiment prompt interface. S32. Each participant reads the experimental instructions and then fills out the AI ​​Trust Initial Rating Questionnaire to quantify the participant’s prior trust. S33. After completing the prior trust test, participants will enter the practice phase. This phase aims to assist them in judging the suggestions of artificial intelligence by learning prior knowledge. In this phase, participants will be exposed to various types of prior knowledge. Participants need to evaluate and judge the provided prior knowledge within a specified time. The system will provide immediate feedback based on their judgments. Only if they answer correctly four times in a row will the participants be considered to have fully understood the connection between prior knowledge and resources and be eligible to enter the next stage of the experimental process. S34. After completing the pre-set exercises, the participants enter the decision-making experiment stage. After the decision-making experiment, the participants fill out the AI ​​trust end rating questionnaire to quantify the participants' post-event trust. S34 specifically includes the following processes: S341. The human-computer collaboration task experiment platform displays background information and AI suggestion information. The background information and AI suggestion information are calculated by the system. Participants need to choose between two options based on the information provided. This process will last for a preset period of time. The background information, i.e., prior knowledge, is calculated by the system through a weighted summation based on the strength of the participants' prior knowledge and the accuracy of their decisions in the previous rounds of experiments. The AI ​​suggestion information is adjusted based on the number of times the AI ​​suggestion matches the actual resource information in the previous rounds of experiments. In order to more realistically reflect the fluctuations in the participants' trust and more easily detect the patterns of trust, the system will change the AI ​​suggestion information after it has matched several times in a row, so that it no longer matches, thereby introducing the volatility of trust. S342. The human-computer collaborative task experimental platform displays selection feedback to let the subjects know whether their selection is correct. The subjects are ranked and given feedback every 10 experiments to provide motivation. The selection feedback interface lasts for the second duration, and the ranking feedback interface lasts for the third duration. S343. The human-computer collaborative task experimental platform displays a trust rating interface, and the subjects give a trust rating to the AI ​​after each experiment.

2. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: Specific in S1 The process includes the following steps: S11. Clearly define the types and complexity of human-machine collaboration tasks that the platform needs to support. Select binary decision-making tasks for human-machine collaboration tasks to more accurately assess the user's level of trust in the machine and its dynamic changes. S12. The platform's built-in intelligent assistance system dynamically generates decision suggestions based on system capability settings and external information constraints of decision rounds. When the platform's built-in intelligent assistance system is working, it adaptively adjusts its capability level through real-time assessment of user feedback and task complexity to cope with different decision-making environments. To ensure the accuracy and timeliness of decisions, the platform's built-in intelligent assistance system has the ability to manage external information for each round of decision-making, ensuring that decisions are always based on the latest data. Through thought chain technology, the platform's built-in intelligent assistance system can simulate human thought processes, providing logically rigorous and coherent reasoning and suggestions. At the same time, the platform's built-in intelligent assistance system uses prompt word technology to guide users to clearly express their needs, and combines content formatting functions to meet the diverse needs of different users and scenarios. S13. Analyze the specific needs of dynamic trust measurement, clarify the construction of measurement indicators and feedback mechanisms. The experimental platform focuses on three key trust assessment dimensions: pre-exposure trust, real-time trust, and post-exposure trust. Before the experiment begins, participants need to complete an initial trust rating questionnaire to assess their initial level of trust in the experiment. During the experiment, participants need to make a quantitative trust rating of 0 to 100 points after each decision to reflect their trust changes in real time. After the experiment, participants will fill out a trust end rating questionnaire to assess their trust feelings throughout the entire experimental process.

3. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: S2 specifically includes the following processes: S21, designing the materials required for the collaborative task experiment and developing the experimental task on the experimental platform to simulate the real experimental environment; S22, designing the human-computer collaborative task experimental script, and presetting the number of experiments to be completed and the experimental settings; S23. Construct an experimental paradigm to explore the level of trust participants have in AI assistants. The experiment will provide participants with prior knowledge to learn decision-making experience. In each actual decision-making trial, relevant external information will be generated for participants' reference, and the intelligent assistance system will also provide decision-making suggestions. The intelligent assistance system will dynamically adjust the accuracy of its suggestions based on prior knowledge conditions and participants' experimental feedback, thereby repairing participants' trust. A series of scenarios will be designed, varying the AI ​​assistant's ability level (high and low), the participants' prior knowledge strength (high, medium, and no knowledge), and binary decision-making suggestions provided by the AI ​​assistant. Participants will make choices in these 12 scenarios and evaluate their trust in the AI ​​assistant. By collecting and analyzing answer accuracy, decision-making time, compliance, and trust scores, the experiment will explore how humans establish and adjust their trust in AI assistants under different conditions.

4. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: Participants with experience using electronic devices and exposure to artificial intelligence products were given priority. Each participant underwent two rounds of experiments, each round consisting of 60 decision-making tasks. In these tasks, the accuracy of the AI's suggestions was designed to vary to simulate uncertainty in the real world. Each round of experiments was divided into three parts, each with 20 rounds, corresponding to high knowledge strength, medium knowledge strength, and no prior knowledge strength, respectively. This design aimed to assess the participants' trust in and reliance on AI suggestions under different knowledge backgrounds.

5. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: S4 specifically includes the following processes: S41. Prior Experiment Data Processing: The human-computer collaborative task experimental platform collects questionnaire data filled out by the participants before the experiment, and analyzes the participants' subject composition, understanding of AI, and trust in AI; S42. The human-computer collaborative task experimental platform collects prior information, answer accuracy rate, judgment time, whether AI suggestions are followed, and trust score data. The prior information is generated by the system based on previous answer feedback; the judgment time is obtained by a timer, representing the time interval between the appearance of the selection interface and the moment the participant triggers the selection button; the experimental data of participants who timed out are removed, and then a mixed method analysis is performed on the prior information, answer accuracy rate, judgment time, whether AI suggestions are followed, and trust score data, including main effect analysis and interaction analysis for each indicator.

Images