Dynamic trust measurement method based on man-machine cooperation task

Abstract

The invention discloses a dynamic trust measurement method based on a man-machine cooperation task, and belongs to the technical field of man-machine interaction trust evaluation, and the method comprises the steps: constructing a man-machine cooperation task experiment platform which is driven by a decision-making task, and is provided with a built-in intelligent auxiliary system for providing decision-making suggestions; s2, resource exploration decision making is carried out on the man-machine cooperation task experiment platform in the step S1, decision making is carried out by a subject in combination with priori knowledge and intelligent system suggestions, and decision iteration is carried out for multiple times by taking benefit maximization as a target; s2, recruiting a preset number of testees based on the man-machine cooperation task experiment platform in S1, executing a man-machine interaction cooperation task, and collecting prior experiment data and interaction experiment data; and obtaining man-machine cooperation tasks under different conditions according to the prior experiment data and the interaction experiment data obtained in the step S3. According to the dynamic trust measurement method based on the man-machine cooperation task, constraints can be provided for design specifications of an intelligent system in the related man-machine cooperation field, and prior early warning can be provided for trust repair.

Description

Technical Field

[0001] The present invention relates to the technical field of trust evaluation in human-computer interaction, and in particular to a dynamic trust measurement method based on human-computer collaborative tasks. Background Art

[0002] Against the backdrop of the rapid development of artificial intelligence (AI) technology, AI has been widely applied across various industries, significantly improving work efficiency and content quality. However, as AI and humans collaborate increasingly closely, trust issues in AI systems have gradually emerged. Trust is not only the cornerstone of the successful application of AI technology but also a crucial prerequisite for the smooth development of human-machine collaboration. To effectively establish and maintain human trust in AI systems, it is necessary to find appropriate methods to calculate human-machine trust indicators and summarize the changing patterns of AI system capabilities and trust levels.

[0003] For example, the Chinese patent application with application number 202410615849.3 and application date 2024-05-17 is a human-computer collaborative platform interface design and evaluation method based on transparency hierarchy. It optimizes the efficiency and quality of information transmission by fine-tuning the display transparency of information on the interface, thereby improving the operator's trust in the platform, the accuracy and safety of task operations, and reducing the operator's cognitive load.

[0004] With the continuous emergence of new human-machine collaborative systems, different users have different trust thresholds, system capabilities, and acceptable trust ranges. Therefore, before the system is put into operation, it is crucial to conduct a mapping test between capability and trust to ensure system reliability and user trust. On this basis, it can further guide the optimization of system design and the constraint specifications of capability performance. Currently, there is a lack of effective system testing methods around dynamic human-machine trust. For this reason, we propose a new experimental paradigm. Summary of the Invention

[0005] The purpose of the present invention is to provide a dynamic trust measurement method based on human-computer collaborative tasks to solve the problem raised in the above background technology that there is a lack of effective system testing methods around dynamic human-computer trust.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a dynamic trust measurement method based on human-machine collaborative tasks, comprising:

[0007] S1. Build a human-machine collaborative task experimental platform. The platform is driven by decision-making tasks and has a built-in intelligent assistance system to provide decision-making suggestions.

[0008] S2: Conduct resource exploration and decision-making on the human-machine collaborative task experimental platform in S1. The subjects make decisions by combining prior knowledge and intelligent system recommendations, and perform multiple decision iterations with the goal of maximizing benefits.

[0009] S3, based on the human-computer collaborative task experimental platform in S1, recruit a preset number of subjects to perform human-computer interactive collaborative tasks and collect prior experimental data and interactive experimental data;

[0010] S4. Based on the prior experimental data and interactive experimental data obtained in S3, obtain the rules of human compliance with AI suggestions under different conditions of human-computer collaborative tasks. The different conditions include different AI capabilities, different prior knowledge, and different experimental durations.

[0011] Preferably, S1 specifically includes the following process:

[0012] S11. Clarify the types and complexity of human-machine collaborative tasks that the platform needs to support. Select binary decision-making tasks for human-machine collaborative tasks. This approach can simplify the decision-making process, facilitate quantitative analysis and control of experimental variables, and simulate real-world scenarios, improving user acceptance and data reliability. This makes the research results more repeatable and practical, thereby more accurately assessing the level of user trust in machines and its dynamic changes.

[0013] S12. The platform's built-in intelligent assistance system dynamically generates decision recommendations based on system capability settings and external information constraints of decision-making rounds. While operating, the platform's built-in intelligent assistance system adaptively adjusts its capability level through user feedback and real-time assessment of task complexity to flexibly respond to different decision-making environments. To ensure the accuracy and timeliness of decisions, the platform's built-in intelligent assistance system has the ability to manage external information for each decision round, ensuring that decisions are always based on the latest data. Through advanced thought chain technology, the platform's built-in intelligent assistance system can simulate the human thought process and provide logically rigorous and coherent reasoning and suggestions. At the same time, the platform's built-in intelligent assistance system uses prompt word technology to guide users to clearly express their needs, and combines content formatting functions to ensure that the output information is both professional and standardized, and easy to understand, meeting the diverse needs of different users and scenarios. The synergy of these functions makes the intelligent assistance system a powerful tool for supporting efficient and accurate decision-making;

[0014] S13. Analyze the specific needs of dynamic trust measurement and clarify the construction of measurement indicators and feedback mechanisms. The experimental platform pays special attention to three key trust assessment dimensions: prior trust, real-time trust, and post-trust. Before the experiment begins, participants need to complete an initial trust rating questionnaire to assess their initial trust level in the experiment. During the experiment, participants need to make a trust quantitative rating from 0 to 100 points after each decision to reflect their trust changes in real time. After the experiment, participants will fill out a final trust rating questionnaire to assess their trust feelings throughout the entire experimental process.

[0015] Preferably, S2 specifically includes the following process:

[0016] S21. Design the materials required for the collaborative task experiment and develop the experimental tasks on the experimental platform to simulate the real experimental environment;

[0017] S22. Design the human-machine collaborative task experiment script, presetting the number of experiments to be completed and relevant parameters such as the experimental settings;

[0018] S23. Construct an experimental paradigm to explore the degree of trust that subjects have in artificial intelligence assistants. The experiment will provide subjects with certain prior knowledge in advance so that they can learn decision-making experience from it. In each actual decision-making trial, relevant external information will be generated for the subjects' reference. At the same time, the intelligent assistance system will also give decision-making opinions. The intelligent assistance system will dynamically adjust the accuracy of the suggestions based on the prior knowledge conditions and the subjects' experimental feedback, thereby repairing the subjects' trust. A series of scenarios will be designed. By changing the ability level of the AI ​​assistant (high ability level / low ability level), the prior knowledge intensity of the participants (high knowledge intensity / medium knowledge intensity / no knowledge), and the binary decision suggestions provided by the AI ​​assistant, participants will be asked to make choices in the game of these 12 situations and evaluate their trust in the AI ​​assistant. By collecting and analyzing data such as the accuracy of answering questions, decision-making time, compliance, and trust scores, we will deeply explore how humans establish and adjust their trust in AI assistants under different conditions.

[0019] Preferably, S3 specifically includes the following process:

[0020] S31. Before the experiment, the subjects filled in their personal information and completed a questionnaire to understand their understanding and use of AI. After completing the questionnaire, they entered the experimental prompt interface.

[0021] S32. Each subject read the experimental instructions and then filled out the AI ​​trust initial scoring questionnaire to quantify the subject's prior trust;

[0022] S33. After completing the prior trust test, participants will enter the practice phase, which aims to assist their judgment of AI recommendations by learning prior knowledge. In this phase, participants will be exposed to various types of prior knowledge, with the goal of enabling them to understand and master the relationship between this knowledge and available resources. Participants will be required to evaluate and judge the prior knowledge provided within a specified timeframe. The system will provide immediate feedback based on their judgments. Only after four consecutive correct answers will participants be deemed to have fully understood the connection between prior knowledge and resources and be eligible to enter the next phase of the experimental process.

[0023] S34. After completing the preset exercises, the subjects entered the decision-making experiment stage. After the decision-making experiment, the subjects filled out the AI ​​trust end score questionnaire to quantify the subjects' post-event trust.

[0024] Preferably, S34 specifically includes the following process:

[0025] S341. The human-machine collaborative task experiment platform displays background information and AI recommendations. These are calculated by the system. Participants are required to choose between two options based on this information. This process lasts for a preset period of time. Background information, or prior knowledge, is calculated by the system through weighted summation based on the strength of the participant's prior knowledge and the accuracy of their decision-making in previous rounds of experiments. AI recommendations are adjusted based on the number of matches between AI recommendations and actual resource information in previous rounds of experiments. To more realistically reflect participants' trust fluctuations and facilitate the detection of trust patterns, the system will change the AI ​​recommendations after several consecutive matches, so that they no longer match, thereby introducing trust volatility.

[0026] S342: The human-computer collaborative task experimental platform displays selection feedback to let the subjects know whether their choices are correct. Ranking feedback is also provided to the subjects every 10 trials to motivate them. The selection feedback interface lasts for the second duration, and the ranking feedback interface lasts for the third duration.

[0027] S343. The trust scoring interface is displayed on the human-machine collaborative task experimental platform, and the subjects give the AI ​​a trust score after each experiment.

[0028] Preferably, participants with experience in using electronic devices and a history of exposure to artificial intelligence products will be selected. Each participant will undergo two rounds of experiments, each round of which contains 60 decision-making tasks. In these tasks, the accuracy of artificial intelligence suggestions will be designed to be different to simulate the uncertainty in the real world. Specifically, each round of experiments will be divided into three parts, with 20 times in each part, corresponding to high, medium, and low levels of prior knowledge. This design aims to evaluate participants' trust and reliance on AI suggestions under different knowledge backgrounds.

[0029] Preferably, S4 specifically includes the following process:

[0030] S41. Prior Experimental Data Processing: The human-machine collaborative task experimental platform collects questionnaire data filled out by the subjects before the experiment, and analyzes the subjects' subject composition, degree of understanding of AI, and degree of trust in AI;

[0031] S42. The human-computer collaborative task experimental platform collects prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data, wherein the prior information is generated by the system based on previous answer feedback; the judgment time is obtained by a timer, which represents the time interval between the appearance of the selection interface and the moment when the subject triggers the selection button; the experimental data of the subjects with judgment timeout are removed, and then the prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data are subjected to a mixed method analysis, and main effect analysis and interaction analysis are performed under each indicator.

[0032] Preferably, a professional team of user experience experts will be formed. The team members include two interaction designers with more than five years of industry experience and four experts with deep expertise in the field of human-computer trust. The team will be responsible for planning and executing each round of experiments, ensuring that the number of decision-making experiments is set reasonably. In addition, the team will also conduct a comprehensive evaluation of the performance of the AI ​​assistant under different capability values ​​and determine its performance standards accordingly.

[0033] A dynamic trust measurement method based on human-machine collaborative tasks, the dynamic trust measurement method using a dynamic trust evaluation system based on human-machine collaborative tasks, including the collaborative task experimental platform;

[0034] The collaborative task experiment platform is provided with a user interaction interface for simulating collaborative tasks, and further includes:

[0035] Parameter setting module, used to set AI capabilities, feedback duration, selection duration, and prior knowledge strength in collaborative tasks;

[0036] The experiment execution module is used to execute human-machine collaborative task experiments and collect prior experimental data and interactive experimental data;

[0037] The data analysis module is used to analyze the dynamic trust changes and patterns in the human-computer collaboration process based on the prior experimental data and interactive experimental data obtained by the experiment execution module.

[0038] Compared with the existing technology, the beneficial effects of the present invention are: it can use a human-computer collaborative task simulation system based on AI suggestions and prior knowledge to make decisions, use this system to carry out user experiments, design collaborative task experiments based on AI suggestions and prior knowledge, explore the changing laws of dynamic trust between humans and computers, and provide examples for evaluating user trust intervals and system capability specifications in human-computer collaborative systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of a dynamic evaluation system based on human-machine collaborative tasks according to the present invention;

[0040] Figure 2 This is a schematic diagram of a dynamic trust assessment experiment using a resource simulation exploration task as an example in the present invention;

[0041] Figure 3 Schematic diagram of the experimental process of the present invention;

[0042] Figure 4 A schematic diagram of the expected interactive experiment results of the present invention without considering prior knowledge;

[0043] Figure 5 A schematic diagram of the results of an interactive experiment of the present invention in which the AI ​​assistant's suggestions are contrary to the subjects' prior knowledge expectations;

[0044] Figure 6 A schematic diagram of the interactive experiment results of the present invention when the AI ​​assistant's suggestions are consistent with the subject's prior knowledge expectations;

[0045] Figure 7 Schematic diagram of 12 situations of the game of the present invention;

[0046] Figure 8 This is a schematic diagram of the single sample t-test of the present invention;

[0047] Figure 9 This is a schematic diagram of the initial expected situation of the T-Test of the present invention;

[0048] Figure 10 This is a schematic diagram of the T-Test results of the present invention;

[0049] Figure 11 This is a schematic diagram of the T-TestAI recommendation situation of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0051] The present invention provides the following technical solution: a dynamic trust measurement method based on human-machine collaborative tasks. The method comprises the following steps:

[0052] (1) Build a human-machine collaborative task experiment platform to achieve customized adjustment of prior information strength and AI capabilities, and adjust platform parameters to achieve efficient and diverse collaborative experiments;

[0053] (2) For human-machine collaboration tasks, a future mission was designed in which humans and AI jointly conduct lunar resource exploration. Given certain mineral point information and AI recommendation information, the exploration intensity was selected and points were given based on the selected results.

[0054] (3) Combine the experimental platform in step (1) to carry out lunar resource exploration simulation experiments, clarify the experimental process of human-machine collaboration tasks including recruiting subjects, pre-experiments, pilot learning, practice experiments, and formal experiments, and collect experimental platform data;

[0055] (4) Summarize the changing patterns of human-computer dynamic trust based on the prior experimental data and interactive experimental data obtained in step (3).

[0056] The specific process is as follows:

[0057] (1) Developing a human-machine collaborative task experimental platform

[0058] (11) The experimental platform was developed using C# and Unity engines and ran on a Surface pro6 (Intel Corei5-8350U, 8GB). The Unity engine was used to simulate the real experimental environment, and the C# language was used to implement task logic and data processing. GPT-3.5 was integrated into the platform through the API interface to provide real-time decision support, such as Figure 1 shown.

[0059] (12) Set the experimental prior information intensity. The prior information intensity indicates the reliability of the prior information. The higher the prior information intensity, the greater the probability that the prior information is correct; conversely, the lower the probability that the prior information is correct.

[0060] The setting of the prior information strength is dynamic, and its value is related to the results of the previous consecutive decision-making experiments. The calculation formula of the prior information strength can be simplified as follows:

[0061]

[0062] ACC indicates whether the choice is correct, 1 indicates correct, and 0 indicates wrong; the prior information strength of the current round of experiments is the weighted sum of the decision accuracy of the previous k rounds of experiments and the prior information strength.

[0063] (13) Set the AI ​​assistant capability value of the experiment and the correct probability of the AI ​​assistant's suggestions for each selection experiment: The AI ​​assistant capability value represents the accuracy of the AI ​​assistant for the entire round of experiments. For example, if there are 60 selection experiments in a round of experiments, then when the AI ​​assistant capability value is 80%, it means that in the 60 selection experiments, 48 ​​AI assistant suggestions were correct and the other 12 suggestions were incorrect.

[0064] The setting of the AI ​​assistant's suggestion accuracy is also dynamic, related to the previous several consecutive selection experiments and the AI ​​assistant's ability value. Its calculation formula can be simplified as follows:

[0065]

[0066] ACC indicates whether the choice is correct, 1 indicates correct, and 0 indicates incorrect; the accuracy of the AI ​​assistant's suggestions in the current round of experiments is the weighted sum of the decision accuracy of the previous k rounds of experiments, the strength of prior information, and the accuracy of the AI ​​assistant's suggestions.

[0067] In addition, the system will also adjust according to the ability value of the AI ​​assistant. If the accuracy of the AI ​​assistant's suggestions reaches the AI ​​assistant's ability value, the subsequent AI assistant will give reverse suggestions.

[0068] (14) Construct a prompt word template and an LLM model thinking chain, so that the AI ​​assistant can provide suggestions and explanations when the subject's trust in it decreases, so as to improve the subject's trust in it.

[0069] Design a prompt word template, which includes mineral information, user decision information, user rating information, etc. Construct prompt word instances based on the prompt word template to guide the LLM model to think in a chain. The model will learn how to gradually derive conclusions from the input information and generate a coherent explanation.

[0070] The prompt word template is shown in the following table:

[0071]

[0072]

[0073] The following table shows examples of prompt word templates:

[0074]

[0075]

[0076] (2) Designing human-machine collaboration tasks and creating experimental scripts

[0077] (21) Design the experimental instructions, which introduce the experimental background information and some prior knowledge to the subjects. In this example, the instructions will introduce the mineral information, the relationship between lunar landform information and large and small mineral points, and the accuracy of AI suggestions;

[0078] Mineral element Chang'e Stone A and lunar landform Dragon Pattern 2 are both highly likely to be associated with large mineral sites; mineral element Lunar Basalt D and lunar landform Lowland 3 are both highly likely to be associated with small mineral sites; Mineral element Nodularite B or Lunar Breccia C and lunar landform Highland 1 or Crater 4 are unrelated to mineral site size and are considered interference information. There are a total of 16 combinations of mineral elements and lunar landforms: A1-4, B1-4, C1-4, and D1-4.

[0079] The subject's initial mineral energy points were 1000. Deep exploration consumed 100 mineral energy points, while shallow exploration consumed 10. Large mineral deposits, located at depths, gained 200 energy points from deep exploration, while shallow exploration yielded none. Small mineral deposits, located at shallow depths, gained 20 energy points from shallow exploration, while deep exploration yielded none.

[0080] (22) Design an initial trust quantification questionnaire, which is used to evaluate and quantify the subjects’ attitudes towards AI assistants. The subjects can proceed to the next stage of the experiment only after they have answered all the questions.

[0081] The initial trust quantification questionnaire consists of 6 questions, each of which is scored on a scale of 1 to 5, with 1 indicating complete disagreement and 5 indicating complete agreement:

[0082] A1. I believe this AI is capable of performing its task.

[0083] A2. I trust this AI;

[0084] A3. I have full confidence in the suggestions given by AI;

[0085] A4. I can rely on this AI;

[0086] A5. I believe AI will continue to operate in a consistent manner.

[0087] A6. I believe that every time I follow the AI's advice, the AI ​​will do its best.

[0088] After completing the initial trust scale, participants were asked to rate their trust in the AI ​​assistant on a scale of 0 to 100.

[0089] (23) Design a prior knowledge learning experiment to allow the subjects to understand and learn the prior knowledge that they will encounter in the experiment. The experiment continues until the subjects understand the prior knowledge.

[0090] Participants read the mineral site information on the interface, including an overhead image of the mineral site, discovery date, geographic location, mineral elements, and geomorphic features. They then made a judgment within a preset timeframe, choosing whether the site was likely a large or small deposit.

[0091] After each selection, the subject will enter the result feedback interface to tell the subject whether the selection is correct. Only after the subject correctly answers 4 questions in a row will they enter the next stage of the experiment.

[0092] (24) Design a decision-making experiment. The decision-making experiment is a formal experiment in the collaborative task experiment. In addition to the mineral point information in the prior knowledge learning experiment, the decision-making experiment also contains the image of the AI ​​and suggestion information. In addition, the binary decision is to consume 100 energy points for deep exploration and 10 energy points for shallow exploration. The subjects must complete the choice within the preset time, otherwise it will be considered a timeout and 100 energy points will be deducted. The decision-making experiment will be carried out for 60 rounds. The bottom of the decision-making experiment interface displays a green progress bar with a gray background to let the subjects know the progress of the experiment.

[0093] The left side of the decision-making interface features an AI image and its recommendations. In the center is a selection box with two options. On the right is information about the mineral deposit. To prevent subjects from accidentally selecting an option, a confirmation button appears directly below the selection box. After the user confirms an option, the decision-making feedback interface appears, informing them of their selection. This feedback interface lasts for a preset period of time, after which the user enters the trust score interface.

[0094] The trust score interface features a green slider with a blue background. Below the slider is a scale indicating 0, 25, 50, 75, and 100. Participants were unable to proceed to the next interface if they did not trigger the slider.

[0095] After every 10 decision-making experiments, the subject will be given information about their current energy point ranking and a graph of their energy point changes over the past 10 times to motivate them. The subject must trigger the Continue button to proceed to the next experiment.

[0096] During the decision-making process, the subjects played a game with the AI ​​assistant based on the prior knowledge provided by the interface. The 12 game scenarios are as follows: Figure 7 shown.

[0097] (25) Design a final trust quantification questionnaire to quantify the subjects’ post-test trust. Subjects can enter the next stage of the experiment only after they have answered all the questions.

[0098] This quantitative questionnaire is presented on two pages. You can proceed to the next page only after completing the questions on the previous page.

[0099] The trust quantification questionnaire consists of 12 questions, each of which is scored on a scale of 1 to 7, with a score of 1 indicating completely disagreement and a score of 7 indicating completely agreement.

[0100] B1. This AI's advice is deceptive.

[0101] B2. The operation of this AI is difficult to understand;

[0102] B3. I doubt AI's suggestions;

[0103] B4. I am very cautious about the advice provided by this AI;

[0104] B5. I think using this AI's advice will lead to bad results.

[0105] B6. I have full confidence in the advice provided by this AI;

[0106] B7. I feel safe using the advice provided by this AI.

[0107] B8. I think this AI is honest;

[0108] B9. I think the AI's advice is very reliable.

[0109] B10. I think the AI's advice is very credible.

[0110] B11. I can trust the advice provided by this AI;

[0111] B12. I am very familiar with this type of AI;

[0112] (3) Combine the experimental platform in step (1) to carry out human-machine collaborative mission experiments for lunar resource exploration.

[0113] Clearly define the experimental process of human-computer collaboration tasks, including pre-experimental questionnaires, prior knowledge learning experiments, practice experiments, and formal experiments, and collect data collected by the experimental platform. The experimental process is as follows: Figure 3 .

[0114] Recruitment: A total of 60 university students (30 males and 30 females) (mean age = 23.86, age range = 2.63 years) were recruited through a campus advertisement posting network. All participants had experience using electronic devices. The study protocol was approved by the university ethics committee, and all participants signed a consent form before participating in the study.

[0115] The questionnaire survey collected data on the subjects’ use of AI and their trust in AI assistants.

[0116] Prior knowledge learning experiments and practice experiments. Prior knowledge learning experiments help participants familiarize themselves with the experimental context and understand relevant information about mineral deposits. This facilitates subsequent decision-making based on mineral deposit information and AI recommendations, allowing participants to choose between deep and shallow exploration. Practice experiments help participants familiarize themselves with the formal experimental process. In practice experiments, participants will fully experience the decision-making process, which helps them prepare for the subsequent formal experiment.

[0117] Formal experiment

[0118] After completing the prior knowledge learning and practice experiments, participants entered an initial trust scoring interface, where they rated their trust in the AI ​​assistant. They then engaged in 60 rounds of decision-making experiments. During the decision-making experiment, a decision-making interface first appeared for 1000 milliseconds. Participants made a comprehensive decision based on the mineral site information provided and the AI's recommendations. After completing their decision, the participant clicked the confirmation button, which prompted a feedback screen indicating whether their choice was correct and whether the mineral site was large or small. The feedback screen remained for 500 milliseconds, after which the trust scoring interface was entered. Participants rated their trust in the AI ​​assistant on a scale of 0 to 100, requiring them to provide a score before proceeding to the next round. After every 10 rounds, participants completed their trust ratings and entered a ranking feedback interface, displaying the participant's energy point performance over the previous 10 rounds and their current energy point ranking. This ranking was calculated in real time, and after each round, their experimental information was dynamically added to the ranking database.

[0119] (4) Experimental results

[0120] (41) Data processing of lunar resource exploration experiments. The experiments collected information on decision-making time, decision accuracy, degree of compliance with AI recommendations, and strength of prior knowledge.

[0121] Among them, the decision-making time in the experiment was obtained by a timer, which was defined as the interval between the appearance of the decision interface and the moment when the participant triggered the confirmation button. The statistical analysis software SPSS23.0 (IBM Corporation, New York, USA) was used to calculate the average trigger time of each participant. In order to ensure the reliability of the data, the extreme values ​​in the search time, that is, the values ​​that deviated from the mean ±0.5 standard deviations, were removed. Then, a mixed method analysis was performed on the three parts of data: decision accuracy, degree of compliance with AI suggestions, and strength of prior knowledge. Main effect analysis and interaction analysis were performed for each indicator. In statistical analysis, the main effect refers to the influence of an independent variable on the dependent variable, while ignoring the influence of other independent variables. The main effect analysis is used to determine the individual influence of each independent variable on the dependent variable. The interaction means the mutual influence between two or more independent variables, resulting in the change of the dependent variable being more than just the sum of the effects of a single independent variable. Through these two analyses, we can better understand the comprehensive impact of the designed independent factors on the experimental results. The proportion of subjects who followed the suggestions of the AI ​​assistant, the results are as follows Figure 4 , we can draw the following conclusions:

[0122] The main effect of AI assistant ability was significant. The higher the AI ​​assistant's ability, the more the subjects followed the AI ​​assistant's suggestions. The overall proportion of subjects following the AI ​​assistant's suggestions matched the AI ​​assistant's ability.

[0123] The main effect of prior knowledge strength was not significant, but the interaction was significant. When the AI ​​assistant had high ability, prior knowledge would not regulate the subjects' decision-making; when the AI ​​assistant had low ability, prior knowledge would significantly regulate the subjects' decision-making.

[0124] The proportion of subjects who followed the AI ​​assistant's suggestions when they were contrary to their prior knowledge was as follows: Figure 5 , Figure 5 The smaller the vertical axis value in (a), the more likely the subject made an expected decision based on the prior knowledge of the mineral deposits and did not follow the AI ​​assistant's advice, reflecting the lower the subject's trust in the AI ​​assistant. Figure 5 The following analysis results can be obtained from the data analysis diagram:

[0125] The main effect of AI assistant ability was significant, showing that the higher the AI ​​assistant's ability, the more the subjects followed the AI ​​assistant's suggestions and the more they trusted the AI ​​assistant;

[0126] The main effect of prior knowledge strength was significant, showing that the stronger the subject's prior knowledge, the smaller the proportion of decisions made in accordance with the AI ​​assistant's suggestions, and the less trust in AI;

[0127] The interaction effect is significant, and the AI ​​assistant capability effect is the largest when the prior knowledge intensity is medium.

[0128] The proportion of subjects who followed the AI ​​assistant's suggestions when the AI ​​assistant's suggestions were the same as the subjects' prior knowledge was as follows: Figure 6 When the AI ​​assistant's suggestions are consistent with the subject's prior knowledge, the subject is very likely to make a decision according to the AI ​​assistant's suggestions, but will not make a decision according to the AI ​​assistant's suggestions.

[0129] (42) Dynamic trust data collation and statistical analysis of lunar resource exploration experiments.

[0130] The statistical analysis software JASP 0.19.1 was used to analyze the dynamic trust level of each participant. Since trust changes are continuous, snapshot measurements cannot reflect its continuity. Therefore, the independent variable trust change was defined as the current trial trust level minus the previous trial trust level, that is, Δtrust = trust n -trust n-1 Each participant's trust change was then divided into eight categories based on trial condition (initial expectation correct / incorrect × AI assistant suggestion correct / incorrect × decision result correct / incorrect). Single-sample t-tests and paired-sample t-tests were conducted for different trial types to explore the impact of different trial conditions on dynamic trust change. To facilitate identification and statistics, the three trial conditions were coded into three-digit codes, with correct as 1 and incorrect as 0. For example, the initial expectation correct, AI assistant suggestion incorrect, and decision result incorrect were coded as 100. Using this method, all trials were divided into eight groups (000, 001, 010, 011, 100, 101, 110, 111). Due to the small amount of data in groups 001 and 110, they were discarded during analysis. Therefore, a total of six groups (000, 010, 011, 100, 101, 111) of trust change data for different trial conditions were retained. This method allows us to explore and understand the internal mechanisms of dynamic trust regulation.

[0131] The results of the one-sample t-test of the trust changes in each type of trial compared with the baseline (0) are as follows: Figure 8 , we can draw the following conclusions:

[0132] The level of trust change corresponding to each type of trial condition changed significantly. At the same time, when the AI ​​assistant made incorrect suggestions, it had a negative impact on trust change, and when the suggestions were correct, it had a positive impact on trust change.

[0133] The paired sample t-test on the change in trust between the three types of trial conditions showed the following results: Figure 9 , 10, 11, we can draw the following conclusions:

[0134] The analysis results of the impact of initial expected correctness on trust change are as follows: Figure 9As shown in the figure, the correctness of initial expectations has a significant impact on the amount of trust change. The trust change caused by incorrect initial expectations is significantly higher than that caused by correct initial expectations. However, the correctness of initial expectations does not significantly affect the asymmetry of trust change, and the absolute value of trust change does not differ significantly.

[0135] The results of the analysis on the impact of the correctness of AI assistant suggestions on the change in trust are as follows: Figure 10 As shown in the figure, the correctness of the AI ​​assistant's suggestions has a significant impact on the change in trust. The change in trust when the AI ​​assistant's suggestions are wrong is significantly lower than when the initial expectation is correct. At the same time, when the decision result is wrong, the correctness of the AI ​​assistant's suggestions significantly affects the asymmetry of the trust change. The absolute value of the trust change when the suggestions are wrong is significantly higher than the absolute value of the trust change when the suggestions are correct, but there is no significant difference when the result is wrong.

[0136] The results of the analysis on the impact of the correctness of decision results on the change of trust are as follows: Figure 11 As shown in the figure, the correctness of the decision result did not significantly affect the change in trust. At the same time, the correctness of the result did not affect the symmetry of the trust change. No result bias occurred in this decision-making task.

[0137] The above is the working process of the entire device, and the contents not described in detail in this specification belong to the existing technology known to professional and technical personnel in this field.

[0138] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A dynamic trust measurement method based on human-computer collaborative tasks, characterized in that: The following steps are involved: S1. Build a human-machine collaborative task experimental platform. The platform is driven by decision-making tasks and has a built-in intelligent assistance system to provide decision-making suggestions. S2: Conduct resource exploration and decision-making on the human-machine collaborative task experimental platform in S1. The subjects make decisions by combining prior knowledge and intelligent system recommendations, and perform multiple decision iterations with the goal of maximizing benefits. S3, based on the human-computer collaborative task experimental platform in S1, recruit a preset number of subjects to perform human-computer interactive collaborative tasks and collect prior experimental data and interactive experimental data; S4. Based on the prior experimental data and interactive experimental data obtained in S3, obtain the rules of human compliance with AI suggestions under different conditions of human-computer collaborative tasks. The different conditions include different AI capabilities, different prior knowledge, and different experimental durations.

2. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: Specific in S1 The following processes are included: S11. Clarify the types and complexity of human-machine collaborative tasks that the platform needs to support. Select binary decision-making tasks for human-machine collaborative tasks. This approach can simplify the decision-making process, facilitate quantitative analysis and control of experimental variables, and simulate real-world scenarios, improving user acceptance and data reliability. This makes the research results more repeatable and practical, thereby more accurately assessing the level of user trust in machines and its dynamic changes. S12. The platform's built-in intelligent assistance system dynamically generates decision recommendations based on system capability settings and external information constraints of decision-making rounds. While operating, the platform's built-in intelligent assistance system adaptively adjusts its capability level through user feedback and real-time assessment of task complexity to flexibly respond to different decision-making environments. To ensure the accuracy and timeliness of decisions, the platform's built-in intelligent assistance system has the ability to manage external information for each decision round, ensuring that decisions are always based on the latest data. Through advanced thought chain technology, the platform's built-in intelligent assistance system can simulate the human thought process and provide logically rigorous and coherent reasoning and suggestions. At the same time, the platform's built-in intelligent assistance system uses prompt word technology to guide users to clearly express their needs, and combines content formatting functions to ensure that the output information is both professional and standardized, and easy to understand, meeting the diverse needs of different users and scenarios. The synergy of these functions makes the intelligent assistance system a powerful tool for supporting efficient and accurate decision-making; S13. Analyze the specific needs of dynamic trust measurement and clarify the construction of measurement indicators and feedback mechanisms. The experimental platform pays special attention to three key trust assessment dimensions: prior trust, real-time trust, and post-trust. Before the experiment begins, participants need to complete an initial trust rating questionnaire to assess their initial trust level in the experiment. During the experiment, participants need to make a trust quantitative rating from 0 to 100 points after each decision to reflect their trust changes in real time. After the experiment, participants will fill out a final trust rating questionnaire to assess their trust feelings throughout the entire experimental process.

3. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: S2 specifically includes the following processes: S21. Design the materials required for the collaborative task experiment and develop the experimental tasks on the experimental platform to simulate the real experimental environment; S22. Design the human-machine collaborative task experiment script, presetting the number of experiments to be completed and relevant parameters such as the experimental settings; S23. Construct an experimental paradigm to explore the degree of trust that subjects have in artificial intelligence assistants. The experiment will provide subjects with certain prior knowledge in advance so that they can learn decision-making experience from it. In each actual decision-making trial, relevant external information will be generated for the subjects' reference. At the same time, the intelligent assistance system will also give decision-making opinions. The intelligent assistance system will dynamically adjust the accuracy of the suggestions based on the prior knowledge conditions and the subjects' experimental feedback, thereby repairing the subjects' trust. A series of scenarios will be designed. By changing the ability level of the AI ​​assistant (high ability level / low ability level), the prior knowledge intensity of the participants (high knowledge intensity / medium knowledge intensity / no knowledge), and the binary decision suggestions provided by the AI ​​assistant, participants will be asked to make choices in the game of these 12 situations and evaluate their trust in the AI ​​assistant. By collecting and analyzing data such as the accuracy of answering questions, decision-making time, compliance, and trust scores, we will deeply explore how humans establish and adjust their trust in AI assistants under different conditions.

4. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: S3 specifically includes the following processes: S31. Before the experiment, the subjects filled in their personal information and completed a questionnaire to understand their understanding and use of AI. After completing the questionnaire, they entered the experimental prompt interface. S32. Each subject read the experimental instructions and then filled out the AI ​​trust initial scoring questionnaire to quantify the subject's prior trust; S33. After completing the prior trust test, participants will enter the practice phase, which aims to assist their judgment of AI recommendations by learning prior knowledge. In this phase, participants will be exposed to various types of prior knowledge, with the goal of enabling them to understand and master the relationship between this knowledge and available resources. Participants will be required to evaluate and judge the prior knowledge provided within a specified timeframe. The system will provide immediate feedback based on their judgments. Only after four consecutive correct answers will participants be deemed to have fully understood the connection between prior knowledge and resources and be eligible to enter the next phase of the experimental process. S34. After completing the preset exercises, the subjects entered the decision-making experiment stage. After the decision-making experiment, the subjects filled out the AI ​​trust end score questionnaire to quantify the subjects' post-event trust.

5. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 4, characterized in that: S34 specifically includes the following processes: S341. The human-machine collaborative task experiment platform displays background information and AI recommendations. These are calculated by the system. Participants are required to choose between two options based on this information. This process lasts for a preset period of time. Background information, or prior knowledge, is calculated by the system through weighted summation based on the strength of the participant's prior knowledge and the accuracy of their decision-making in previous rounds of experiments. AI recommendations are adjusted based on the number of matches between AI recommendations and actual resource information in previous rounds of experiments. To more realistically reflect participants' trust fluctuations and facilitate the detection of trust patterns, the system will change the AI ​​recommendations after several consecutive matches, so that they no longer match, thereby introducing trust volatility. S342: The human-computer collaborative task experimental platform displays selection feedback to let the subjects know whether their choices are correct. Ranking feedback is also provided to the subjects every 10 trials to motivate them. The selection feedback interface lasts for the second duration, and the ranking feedback interface lasts for the third duration. S343. The trust scoring interface is displayed on the human-machine collaborative task experimental platform, and the subjects give the AI ​​a trust score after each experiment.

6. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: Priority will be given to participants with experience in using electronic devices and a history of exposure to artificial intelligence products. Each participant will undergo two rounds of experiments, each containing 60 decision-making tasks. In these tasks, the accuracy of artificial intelligence suggestions will be designed to be different to simulate the uncertainty in the real world. Specifically, each round of experiments will be divided into three parts, with 20 times in each part, corresponding to high, medium, and low levels of prior knowledge. This design aims to evaluate participants' trust and reliance on AI suggestions under different knowledge backgrounds.

7. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: S4 specifically includes the following processes: S41. Prior Experimental Data Processing: The human-machine collaborative task experimental platform collects questionnaire data filled out by the subjects before the experiment, and analyzes the subjects' subject composition, degree of understanding of AI, and degree of trust in AI; S42. The human-computer collaborative task experimental platform collects prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data, wherein the prior information is generated by the system based on previous answer feedback; the judgment time is obtained by a timer, which represents the time interval between the appearance of the selection interface and the moment when the subject triggers the selection button; the experimental data of the subjects with judgment timeout are removed, and then the prior information, answer accuracy, judgment time, whether AI suggestions are followed, and trust score data are subjected to a mixed method analysis, and main effect analysis and interaction analysis are performed under each indicator.

8. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, characterized in that: A professional team of user experience experts will be assembled, including two interaction designers with more than five years of industry experience and four experts with deep expertise in human-computer trust. The team will be responsible for planning and executing each round of experiments, ensuring the appropriate number of decision-making experiments. In addition, the team will conduct a comprehensive evaluation of the AI ​​assistant's performance at different capability values ​​and determine its performance standards accordingly.

9. The dynamic trust measurement method based on human-machine collaborative tasks according to claim 1, wherein the dynamic trust measurement method uses a dynamic trust evaluation system based on human-machine collaborative tasks, characterized in that: Including the collaborative task experimental platform; The collaborative task experiment platform is provided with a user interaction interface for simulating collaborative tasks, and further includes: Parameter setting module, used to set AI capabilities, feedback duration, selection duration, and prior knowledge strength in collaborative tasks; The experiment execution module is used to execute human-machine collaborative task experiments and collect prior experimental data and interactive experimental data; The data analysis module is used to analyze the dynamic trust changes and patterns in the human-computer collaboration process based on the prior experimental data and interactive experimental data obtained by the experiment execution module.

Images