Self-adaptive man-machine interaction control method and system based on dynamic trust estimation

Abstract

The invention provides a self-adaptive man-machine interaction control method and system based on dynamic trust estimation, and belongs to the field of man-machine interaction. The dynamic trust estimation method based on the partial observable Markov decision process (POMDP) is provided for solving the problems that in the prior art, trust modeling is static and single-dimensional, a dynamic matching mechanism with system reliability is lacked, and a system is prone to being abandoned or misused. According to the method, human trust is used as a hidden state, and real-time trust estimation is realized through state transition in combination with multi-dimensional observation data such as user compliance, response time and physiological signals. The system comprises a behavior observation module, a trust estimation module, a strategy decision module, an interactive execution module and a data updating module to form closed-loop control. The method can respond to the trust change of the user in real time, improves the man-machine cooperation efficiency and safety, and is suitable for intelligent prostheses, collaborative robots, man-machine cooperation systems and other scenes.

Description

TECHNICAL FIELD The present application relates to the field of human-computer interaction, and in particular to an adaptive human-computer interaction control method and system based on dynamic trust estimation. BACKGROUND

[0001] With the development of artificial intelligence, robotics and wearable devices, the interaction between humans and intelligent systems is becoming increasingly complex, and the collaboration relationship is becoming increasingly close. In typical applications such as intelligent prosthetics, collaborative robots, and assisted driving, the system not only needs to complete task control, but also needs to make adaptive adjustments according to the psychological state and behavior intention of the human user. In this context, human-computer interaction has evolved from the traditional command-response mode to the human-computer fusion mode based on cognitive state recognition. In this process, "trust" as a subjective evaluation of the system's ability, safety and intention understanding by humans, becomes a key factor affecting whether users will adopt suggestions and whether they will continue to use the system. If the trust level is too low, it may lead to users frequently denying suggestions, refusing to collaborate, or even completely abandoning the system; while if the trust level is too high, it may lead to misuse or over-reliance, resulting in risk out of control when the system makes a mistake or the environment changes suddenly.

[0002] Currently, traditional interaction systems usually model trust as a static variable, or only evaluate it based on fixed questionnaires and single behavior data, lacking the ability to model the time-varying, uncertain and task-dependent characteristics of trust. In addition, existing systems lack consideration of matching trust with the actual reliability of the system, i.e., lack of trust calibration mechanism. This mismatch can easily cause trust bias, thereby seriously affecting the effectiveness and safety of human-machine collaboration.

[0003] Therefore, it is necessary to build a method framework that can dynamically estimate the trust state of users and incorporate it into the interaction control process, optimize the control strategy based on trust, and improve the efficiency and safety of human-machine collaboration to achieve better human-machine fusion. SUMMARY

[0004] The present application proposes an adaptive human-computer interaction control method and system based on dynamic trust estimation. By modeling the trust state as the core of the hidden variable, using the Bayesian filtering principle for state estimation, and introducing the Q-MDP algorithm to realize the control strategy optimization for the belief state, the adaptive evolution of the human-machine system in the perception, estimation, and control loop is realized. The present application provides an adaptive human-computer interaction control method based on dynamic trust estimation, comprising the following steps:

[0005] Step 1, defining a human-computer interaction state space S, the state space including a trust state s1, a workload state s2 and an environmental risk state s3, jointly constituting s∈S, wherein the trust state is a hidden state that cannot be directly observed;

[0006] Step 2, define the action space A, which includes control strategy a1, information transparency a2, and suggestion mode a3, jointly constituting a∈A;

[0007] Step 3, define the observation space O, which includes compliance c, response time tr, and physiological signal ps, jointly constituting o∈O;

[0008] Step 4, construct state transition probability P(s'| s, a) and observation probability P(o| s'), and obtain them through actual interaction data training;

[0009] Step 5, given the prior belief b(s), update the posterior estimate b'(s') of the trust state according to the current action a and observation o through the Bayes update formula: where η is the normalization factor, and the posterior estimate b'(s') of the trust state is updated in real time;

[0010] Step 6, based on the belief b'(s') and the expected reward function R(s, a), calculate the expected value of each action using the Q-MDP method: where , is the discount factor;

[0011] Step 7, select the optimal action that satisfies the maximum expected value and drive the interaction execution module to output the corresponding control strategy;

[0012] Step 8, collect new observations and repeat steps (4) to (7) to form a closed-loop adaptive human-machine interaction control process.

[0013] Further, the trust state s1 is a finite discrete state, including at least three levels: low trust, medium trust, and high trust, and the transition probability of the trust state is adjusted dynamically with system reliability, transparency level, and user response.

[0014] Further, the compliance c in the observation is a binary variable, the response time tr is a positive real number, and the physiological signal ps includes but is not limited to electroencephalogram, electromyogram, heart rate, and galvanic skin response.

[0015] Further, the state transition probability P(s'| s, a) and the observation probability P(o| s') are fitted through human-machine interaction experimental data, and the Baum-Welch algorithm or genetic algorithm is used for parameter learning.

[0016] Further, the control strategy a1 includes action amplitude, execution intensity, automatic / manual switching, etc., the information transparency a2 includes three levels of low, medium and high, the higher the level, the more information provided by the agent, and the suggestion mode a3 includes explanatory prompts, graphical assistance, language or vibration feedback, etc.

[0017] In another aspect, the application also provides an adaptive human-computer interaction control system based on dynamic trust estimation, comprising:

[0018] (1) a behavior observation module for observing and measuring user data, including compliance, response time, physiological signals, etc.;

[0019] (2) a trust estimation module for realizing real-time estimation of the user's trust state based on POMDP structure or Bayesian network;

[0020] (3) a strategy decision module for outputting optimal control actions according to the current belief state by using Q-MDP or belief-driven model;

[0021] (4) an interaction execution module for adjusting the control behavior of the agent, suggestion content, information transparency or interaction mode according to the selected action, so as to realize the response and guidance to the human operation intention;

[0022] (5) a data updating module for storing the interaction data of each round and updating the state transition and observation model parameters.

[0023] Further, the behavior observation module includes mechanical keys, kinematic sensors, electroencephalogram head rings, electromyography arm / leg rings, and heart rate bands.

[0024] Further, the trust estimation module is realized by using particle filtering, hidden Markov model (HMM), or dynamic Bayesian network (DBN).

[0025] Further, the strategy decision module supports integration with neural networks and reinforcement learning modules for strategy optimization in long-term interaction.

[0026] The application has the following beneficial effects:

[0027] Firstly, the application realizes dynamic estimation of trust. By modeling human trust as a dynamic hidden state that cannot be directly observed, and combining multi-modal behavior and physiological information such as user compliance, response time and physiological signals, real-time and multi-dimensional perception and estimation of trust state are realized. The multi-modal fusion perception mechanism effectively overcomes the limitations of static and single trust evaluation in traditional methods, can dynamically reflect the trust fluctuations of users in different task stages and situations, and more accurately capture the changes of user psychology and behavior.

[0028] Secondly, the application establishes a trust-based control strategy optimization mechanism with closed-loop regulation capability. Based on the Partially Observable Markov Decision Process (POMDP) and Q-MDP method, the system dynamically calculates the optimal configuration of control strategy, information transparency and interactive feedback mode according to the real-time updated trust belief state. Through the trust calibration mechanism, the consistency between user trust level and actual system reliability is ensured, avoiding abandonment or misuse due to trust bias, realizing the full-link closed-loop regulation from perception, estimation to control decision, and effectively improving the adaptability and safety of human-machine collaboration.

[0029] Thirdly, the application forms a unified system architecture suitable for multiple types of human-machine integration scenarios, supporting cross-platform deployment and customized extension. The proposed dynamic trust estimation and strategy optimization framework is not only suitable for specific applications such as intelligent prostheses, collaborative robots, and assisted driving, but also can be flexibly adapted to other complex human-machine interaction systems through modular design and parameter adjustment. The system structure is open, which can combine different types of sensors, control strategies and feedback mechanisms, facilitating rapid integration and functional expansion in different industries and scenarios, and has good universality, portability and practical value. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0031] Figure 1 A structural schematic diagram of the adaptive human-machine interaction control system based on dynamic trust estimation in the embodiment of the present application;

[0032] Figure 2 A scene schematic diagram of the adaptive human-machine interaction control method based on dynamic trust estimation in the embodiment of the present application in prosthesis control;

[0033] Figure 3 A logic flow chart of the adaptive human-machine interaction control method based on dynamic trust estimation in the embodiment of the present application in prosthesis control. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0035] The terms "including", "containing", "having" and "including" and any variations thereof in the DETAILED DESCRIPTION of the Invention and in the claims herein are intended to cover both express and implicit incorporating of the stated features. For example, a process, method, system, product, or apparatus that comprises a list of steps or units are not necessarily limited to those specifically listed but can include other steps or units that are inherent to such processes, methods, products, or apparatus.

[0036] Reference to "an embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all directed to the same embodiment, or to a single alternative embodiment. It is expressly understood that any of the features, structures, or characteristics described in connection with an embodiment can be included in at least one implementation of the application. The application proposes an adaptive human-computer interaction control method and system based on dynamic trust estimation, which is based on a partially observable Markov decision process (POMDP) modeling, takes the human trust state as an unobservable dynamic hidden state, combines human behavior characteristics and physiological signals and other multi-modal observation information, and realizes dynamic estimation of trust and optimal selection of control strategies through Bayesian filtering and Q-MDP method.

[0037] Referring to Figure 1 The embodiment provides an adaptive human-computer interaction control system based on dynamic trust state estimation, which comprises a behavior observation module 110, a trust estimation module 120, a strategy decision module 130, an interaction execution module 140, and a data updating module 150.

[0038] The behavior observation module 110 is responsible for observing and measuring the user's compliance, response time, electromyographic signal, electroencephalogram, galvanic skin response, heart rate and other data, and then sending the processed data to the trust estimation module 120. The trust estimation module 120 extracts and preprocesses these data, combines the preset state transition function and observation function in the system, dynamically estimates the user's trust level, workload and risk state through the Bayesian filtering method, and sends the estimated state parameters to the strategy decision module 130. The strategy decision module 130 calculates the expected value of different actions (such as control strategy, information transparency, and suggestion method) based on various state parameters, applies the Q-MDP method, generates the optimal strategy, and sends the strategy data to the interaction execution module 140. The interaction execution module 140 controls the intelligent prosthesis or robot to adjust the execution mode, information transparency and suggestion method according to the strategy decision output, and feeds back to the user. The data updating module 150 continuously records new data in the human-computer interaction process, updates the transition function, observation function and reward function parameters of the model, and supports the dynamic learning and evolution of the model.

[0039] The system forms a closed loop through perception, estimation and control, realizes the real-time monitoring and adaptive evolution of human-computer interaction system.

[0040] Specifically, the behavior observation module 110 includes a key 111, a kinematics sensor 112, an electroencephalogram head ring 113, an electromyography arm / leg ring 114, and a heart rate band 115. In the present application, more types of sensors can be added according to the observation range to further enrich the data dimension, such as an eye tracker, a hygrometer, etc.

[0041] The trust estimation module 120 is responsible for feature extraction and preprocessing of observation data, trust estimation, and the specific steps are as follows: first, denoising and normalization of multi-modal signals such as electroencephalogram, electromyography, heart rate, etc. are performed; then, features such as frequency domain power, time domain statistics and variation index are extracted; finally, the preset state transition and observation function are combined, and the current trust state, workload and risk level are dynamically inferred through Bayesian filtering, providing belief state input for subsequent strategy decision.

[0042] The policy decision module 130 includes a policy calculation unit 131 and a policy optimization unit 132. The policy calculation unit 131 calculates the expected value of all candidate actions based on the current belief state using the Q-MDP method, and preliminarily filters out a set of actions with higher potential according to the maximum expected criterion. The policy calculation unit 131 ensures that the current optimal decision reference is quickly obtained under the condition that the belief state is known. The policy optimization unit 132 further introduces a neural network or reinforcement learning mechanism to further optimize the action set. The policy optimization unit 132 can dynamically learn the long-term correlation between state-action-reward through algorithms such as policy gradient, deep Q network (DQN), proximal policy optimization (PPO), etc., to mine better action sequences or combinations, which is suitable for scenarios with complex state space and dynamic changes in interactive situations. Through continuous iterative training of the optimization unit, the system can continuously adjust and improve the global optimality and generalization ability of the policy. The policy calculation unit 131 and the policy optimization unit 132 work together, with the former responsible for efficient filtering and the latter responsible for deep optimization, to ensure that the optimal policy is quickly found in a complex state-action space. The optimal policy output by this module includes control policy, transparency level, and suggested expression method.

[0043] The interaction execution module 140 includes a control signal execution unit 141, an information presentation unit 142, and a multi-modal feedback interface 143. The control signal execution unit 141 is used to convert the control instructions output by the policy decision module into specific power control, position adjustment, or execution actions of the agent. The information presentation unit 142 is responsible for feeding back the agent state, control intention, or suggestion information to the user in the form of vision, hearing, etc. The multi-modal feedback interface 143 integrates vibration, sound, visual cues, and other feedback forms to enhance user perception and interaction experience. It should be noted that the interaction execution module 140 not only includes control of physical devices, but also integrates intelligent interaction interface design, such as interface assistance based on augmented reality (AR) or virtual reality (VR), which can intuitively present interaction information in a virtual and real combined environment, further improving the naturalness and immersion of human-machine integration.

[0044] The data update module 150 is used to dynamically record behavior data, belief state, executed policy, and corresponding interaction results during human-computer interaction. Based on the continuously collected interaction samples, the module periodically or on demand calls the model update mechanism, uses the expectation maximization (EM) algorithm or genetic algorithm to iteratively update the state transition function, observation probability function, and reward model parameters in the POMDP model. Through continuous parameter updating and model optimization, the system can continuously optimize its perception and response ability to trust dynamics based on interaction history, ensuring the accuracy, adaptability, and robustness of the model in long-term application, thereby supporting reliable operation of the human-computer interaction system in different users and variable environments.

[0045] It should be noted that in order to obtain the initial parameters of the model, the system needs to design a standardized human-computer interaction task scenario in advance, and carry out a systematic human-computer interaction experiment by recruiting subjects. During the experiment, the system will collect the behavioral responses, physiological signals and task feedback of the subjects in real time, and establish a multi-modal sample database. Subsequently, based on the sample database, the EM algorithm or genetic algorithm is used to train and fit the state transition function, observation function and reward function parameters in the POMDP model, forming the initial parameter set of the model.

[0046] In specific applications, such as the interactive control of intelligent prostheses, please refer to the scene diagram Figure 2 , the system can dynamically adjust the control mode of the prosthesis (conservative, normal, aggressive), the recommended transparency (low, medium, high) and the prompt method (explanation, graphics, voice) according to the changes in the user's trust state, improving the collaboration efficiency and user experience while ensuring safety. The running steps of the system are shown in Figure 3 , including: Step 1, define the state space (trust, workload, risk level, etc.), action space (control strategy, information transparency, interaction method) and observation space (user compliance, response time, physiological signals, etc.). Step 2, design and implement a human-computer interaction experiment, recruit subjects to participate, collect multi-modal behavior and physiological data through continuous interaction tasks, and build a sample database. Based on this database, methods such as EM algorithm and genetic algorithm are used to fit the state transition function and observation probability function of the model, complete the initial parameter training and loading of the POMDP model. Step 3, in the formal running phase, collect the user's current behavior and physiological signals to form the observation data o_t. Step 4: Feature extraction is performed on the observation data o_t, and the belief state b_t is updated based on the current action and state transition probability, realizing the dynamic estimation of trust, load and risk. Step 5, based on the belief state b_t, use the Q-MDP method to calculate the expected value Q(b_t, a) of each action in the action space, and select the optimal action a*. Step 6, execute the optimal action a*, which includes adjusting the control mode, transparency level and feedback method of the prosthesis, and applying the results to the user. Step 7, record the observation, belief change, selected strategy and user feedback information in this round of interaction, dynamically optimize the model parameters according to the set update mechanism, and continuously improve the model performance and interaction adaptability. Step 8, return to step 3, start a new round of interactive cycle, the system forms a complete closed-loop human-machine fusion interaction process through continuous sensing, estimation, decision-making, execution and updating.

[0047] Through the above embodiments, the application can realize dynamic estimation and calibration of trust in human-computer interaction, adapt human-computer collaborative control strategy to user trust level and agent reliability, guarantee the safety, stability and intelligence of interaction, and realize efficient human-machine fusion collaboration.

[0048] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present application.

[0049] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0050] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented by other means. For example, the device embodiments described above are only illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical or other forms.

[0051] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme. In addition, the function units in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software program module.

[0052] If the integrated unit is realized in the form of a software program module and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0053] A person of ordinary skill in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer readable memory, and the memory can include a flash disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0054] The embodiments of the present application are described in detail above, and the specific examples are applied to the principles and implementation modes of the present application. The above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description should not be understood as a limitation of the present application.

Claims

1. An adaptive human-machine interaction control method based on dynamic trust estimation, characterized in that, Comprising the following steps: Step 1, define the human-computer interaction state space S, which includes trust state s1, workload state s2 and environmental risk state s3, jointly constitute s∈S, wherein the trust state is an unobservable hidden state; Step 2, define the action space A, which includes control strategy a1, information transparency a2, and suggestion mode a3, jointly constitute a∈A; Step 3, define the observation space O, which includes compliance c, response time tr, and physiological signal ps, jointly constitute o∈O; Step 4, Construct state transition probability P(s' | s, a) | s, a) with observation probability P(o ), and is trained by actual interaction data; Step 5, given the prior belief b(s), according to the current action a and observation o, update the posterior belief b'(s') by the Bayes update formula: Where η is the normalization factor, and the posterior belief b'(s') of the trust state is updated in real time; Step 6, based on the belief b'(s') and the expected reward function R(s, a), the expected value of each action is calculated by the Q-MDP method: wherein , is a discount factor; Step 7, selecting the optimal action that satisfies the maximum expected value and drive the interactive execution module to output the corresponding control strategy; Step 8, collect new observations and repeat steps (4) to (7) to form a closed-loop adaptive human-computer interaction control process.

2. The method of claim 1, wherein, The trust state s1 is a finite discrete state, including at least three levels: low trust, medium trust and high trust, and the transition probability of the trust state is adjusted dynamically with system reliability, transparency level and user reaction.

3. The method of claim 1, wherein, The compliance c in the observation is a binary variable, the response time tr is a positive real number, and the physiological signal ps includes but is not limited to electroencephalogram, electromyogram, heart rate, galvanic skin response, etc.

4. The method of claim 1, wherein, The state transition probability P(s'| s, a) and the observation probability P(o| s') are fitted by human-computer interaction experimental data, and the Baum-Welch algorithm or genetic algorithm is used for parameter learning.

5. The method of claim 1, wherein, The control strategy a1 includes action amplitude, execution intensity, automatic / manual switching, etc., the information transparency a2 includes low, medium and high levels, and the higher the level, the more information the agent provides, and the suggestion mode a3 includes explanatory prompts, graphical assistance, language or vibration feedback, etc.

6. An adaptive human-machine interaction control system based on dynamic trust estimation, characterized in that, Comprising: (1) Behavior observation module, for collecting user behavior data, including compliance, reaction time, physiological signal, etc.; (2) Trust estimation module, based on POMDP structure or Bayesian network to realize real-time estimation of user trust state; (3) Strategy decision module, using Q-MDP or confidence-driven model to output the optimal control action according to the current belief state; (4) Interaction execution module, adjusting the control behavior, suggestion content, information transparency or interaction mode of the agent according to the selected action, to realize the response and guidance to human operation intention; (5) Data update module, for storing each round of interaction data and updating the state transition and observation model parameters.

7. The system of claim 6, wherein, The behavior observation module includes: key, kinematics sensor, electroencephalogram head ring, electromyogram arm / leg ring, heart rate band.

8. The system of claim 6, wherein, The trust estimation module uses particle filtering, hidden Markov model or dynamic Bayesian network to realize.

9. The system of claim 6, wherein, The strategy decision module supports integration with neural network and reinforcement learning module for long-term interaction strategy optimization.

10. The system of claim 6, wherein, The system is applied to the fields of prosthesis control, collaborative robot, human-artificial intelligence collaborative system, etc.

Images