Human-intelligent interaction process safety assessment method considering transparency influence

Abstract

The invention provides a human-intelligent interaction process security assessment method considering transparency influence. The method comprises the following steps: modeling scene awareness, trust update and takeover decision in a human-intelligent interaction process through a DBN; monolithic network modeling in the DBN is based on a single takeover decision of scene awareness and trust, and connection between adjacent networks models multiple possible trust updates driven by human-intelligent interaction paths. According to the method, the influence of transparency on scene awareness, trust dynamic evolution and takeover decision can be systematically analyzed, and a multi-level dynamic model of a human-intelligent interaction process is constructed; interaction links such as intelligent system decision errors, personnel takeover decisions and trust updating can be described, comprehensive evaluation of the safety of the human-intelligent interaction process is achieved, and accurate safety indexes and dynamic trust change sequences are output. The evaluation method is scientific and rigorous, the modeling process is efficient, the applicability is high, and the method has wide popularization and application value.

Description

Technical Field

[0001] The present invention proposes a new method for assessing the security of human-intelligence interaction processes taking into account the impact of transparency. During the human-intelligence interaction process, the impact of transparency on situational awareness and trust is considered to assess the security of the human-intelligence interaction process, which belongs to the field of human-intelligence interaction security modeling and analysis. Background Art

[0002] Nowadays, intelligent systems have been widely used in various fields and have achieved remarkable results in reducing workload and improving efficiency. However, their application has not effectively prevented accidents, such as the Tesla crash in Florida and the Lion Air Boeing 737MAX crash in May 2016. This is because the intelligent system technology is not yet mature, and humans have not yet adapted to the new human-intelligence interaction mode, which requires them to change from operators to supervisors and take over when the intelligent system decision-making is obviously wrong. Correct human takeover is crucial to the safety of the human-intelligence interaction process and is usually the last barrier to prevent accidents. Intelligent system transparency is seen as an effective factor in supporting takeover decisions.

[0003] The takeover decision based on transparency depends on situation awareness (SA) of the correctness of the intelligent system's decision and trust in the intelligent system. Transparency refers to the ability of the interface to provide information about the intelligent system's intentions, performance, future plans, and reasoning process. The realization of situation awareness is based on the correctness of transparency information, and any obvious errors will lead to takeover. If there is no evidence that the intelligent system's decision is obviously wrong, trust will guide human takeover behavior. Obviously, compared with the human-intelligence interaction process without transparency information and only based on trust as the main basis for takeover decisions, the human-intelligence interaction supported by transparency information has a more complex interaction process. The former has only two possible paths (the interaction results are success / failure), while the latter has multiple possible paths, which are distinguished by different situation awareness states (i.e., whether the intelligent system decision is obviously wrong), trust states (trust / distrust), and interaction results (success / failure). These paths will bring different interaction experiences and update trust in different directions or degrees.

[0004] Improved transparency can promote the realization of correct situational awareness and the evolution of trust. Higher levels of transparency expose more details about the decision-making process of intelligent systems, which makes errors in intelligent system decisions easier to detect and increases the possibility of achieving correct situational awareness. In addition, the more details provided by higher transparency also increase human expectations of the functions of intelligent systems, thereby enhancing the impact of positive and negative human-intelligence interaction paths on trust. Thanks to higher situational awareness reliability, higher transparency helps to take over correctly and reduce negative experiences. Therefore, increased transparency can promote the trust formation process, and trust will gradually stabilize after multiple updates. At present, the broad prospects of transparency in improving the performance of human-intelligence interaction have attracted widespread attention in many industries, including smart shipping. The present invention aims to develop a safety assessment method for human-intelligence interaction processes that considers the impact of transparency on situational awareness and trust.

[0005] Some studies have attempted to model and evaluate the safety of human-intelligence interaction processes through trust-based takeover decision models. These models regard trust as the basis for takeover decisions and as a mediating variable, that is, the characteristics of intelligent systems affect the human-intelligence interaction process by affecting trust. Some of them regard trust as the probability of accepting an intelligent system. For example, scholars at the University of Michigan proposed a trust formation model based on Beta distribution, which describes trust affected by the reliability of intelligent systems. Based on this model, they used hidden Markov models to evaluate the impact of intelligent system reliability on the safety of sequential decision-making human-intelligence interaction tasks. Similarly, Fooladi Mahani et al. of Clemson University used a dynamic Bayesian network (DBN) to build a model that regards trust as a mediating variable to predict human takeover tendencies under the influence of intelligent system performance. Other studies use the gap between trust and confidence to measure acceptance preferences. Takeover occurs when the preference evolves below a given threshold. For example, Gao and Lee of the University of Iowa applied decision field theory to model trust and confidence updates based on the past performance of intelligent systems and humans. At different levels of transparency, they used amplification factors to adjust the degree of influence of interaction experience on trust updates. This approach has been used to model takeover behavior in many studies related to human-human interaction processes.

[0006] Trust-based takeover decision models provide an effective way to evaluate the safety of human-intelligence interaction processes and successfully consider the role of transparency in enhancing the degree of trust in updating human-intelligence interaction paths. However, they use trust that relies on past interaction results as the main basis for takeover decisions, while ignoring situational awareness supported by transparency. Therefore, they only consider two human-intelligence interaction paths (i.e., the results are success / failure) without considering more paths. This also makes them fail to fully consider the diversity of human-intelligence interaction paths when modeling trust updates. Empirical and mechanistic studies qualitatively illustrate how situational awareness is combined with trust to support takeover decisions, and how situational awareness is combined with current trust and interaction results to update trust. They provide sufficient theoretical guidance to fill the above gaps. Summary of the invention

[0007] The present invention aims to propose a new method for safety assessment of human-intelligence interaction process, which fully considers the impact of transparency on situational awareness and trust formation, develops relevant models for quantitative analysis, builds an evaluation framework based on DBN, and more accurately evaluates the safety of human-intelligence interaction process, providing effective support for the safety design and optimization of intelligent systems.

[0008] Based on the impact of transparency on situational awareness and trust in the process of human-intelligence interaction, the present invention proposes a method for assessing the security of the human-intelligence interaction process considering the impact of transparency. The method models situational awareness, trust updates and takeover decisions in the process of human-intelligence interaction through DBN. Specifically, the single-chip network modeling in DBN is based on a single takeover decision of situational awareness and trust, and the connection between adjacent networks models the trust updates driven by multiple possible human-intelligence interaction paths. Figure 1 The four main steps of the human-intelligence interaction path safety assessment are presented and explained as follows:

[0009] Step 1: Intelligent system decision and transparency analysis, to obtain key knowledge required for modeling and evaluation, such as key input information, the generation process of intelligent system decisions, and transparency information at each level;

[0010] Step 2: Qualitative modeling, qualitatively describing the takeover decision process and trust update process through the nodes and connections in DBN;

[0011] Step 3: Quantitative modeling: quantify the parameters of each node based on the construction of DBN structure;

[0012] Step 4: Safety assessment of the human-intelligence interaction process. Based on the prior distribution of the root node and the conditional probability distribution of the remaining nodes, the safety of the human-intelligence interaction process is assessed.

[0013] Among them, in step one, it is necessary to fully study the operating principles of the intelligent system and the transparency information available at each level. After an abnormal situation occurs, the intelligent system uses the detected information (the content of the information depends on the specific analysis scenario) and various algorithms (the category of the algorithm depends on the intelligent system analyzed) to achieve situation reasoning, decision making and decision utility prediction, and will execute the decision with the best expected utility. While these algorithms are running, transparency information at each level is generated. Among them, key input information, the generation process of intelligent system decisions, and transparency information are important parts of the safety modeling and evaluation of the human-intelligence interaction process. The purpose of this preparatory step is to obtain this necessary knowledge.

[0014] This invention focuses on the human-intelligence interaction process in emergency situations because the purpose is to evaluate safety. We follow the theoretical framework of situational awareness-based agent transparency (SAT) and adapt it to meet the needs of takeover decision-making. Specifically, SAT1 provides inputs and action suggestions detected by the intelligent system; SAT2 provides information about the intelligent system's understanding of current risks; SAT3 provides information about the expected utility of the intelligent system's action suggestions. Then, the intelligent system transparency can be divided into three levels: TL1=SAT1, TL2=TL1+SAT2, TL3=TL2+SAT3.

[0015] The impact of transparency on situational awareness and trust in human-intelligence interactions under risk scenarios Figure 2 In the code, the numbers ① and ② are used to indicate the Figure 2 As shown in , transparency incorporates situational awareness into takeover decisions. Based on information from other sources, humans can verify SAT information at a given level, which helps humans achieve situational awareness of the correctness of the intelligent system's decision. Discovering any obvious errors in SAT information will make humans aware of the intelligent system's decision error and initiate takeover. Otherwise, trust will guide human takeover behavior. Second, as Figure 2 As shown in Figure 1, transparency introduces more human-intelligence interaction paths. The number of possible human-intelligence interaction paths increases from two (i.e., the interaction results are success or failure) to six, distinguished by different combinations of situational awareness, trust, and interaction results. These human-intelligence interaction paths can update trust in different directions and degrees. Whether the experience brought about by these paths is positive or negative depends on whether the initial expectations of the intelligent system's capabilities are violated. Therefore, Figure 2 Paths 1-3 bring positive experiences, which increase trust and are positive paths; paths 4-6 bring negative experiences, which reduce trust and are negative paths. Due to negative bias, intelligent systems are abused (i.e. Figure 2The trust loss caused by path 4) in the above example is much greater than that of other paths. Although paths 4 and 5 bring negative experiences to humans, they effectively support the takeover decision, resulting in very slight trust loss. Figure 2 As shown in Figure 1, higher levels of transparency enable personnel to more fully appreciate the complexity of intelligent system decisions and increase expectations of the capabilities of intelligent systems. This leads to greater changes in trust on both positive and negative paths. Finally, with increased transparency, more of the process by which intelligent system decisions are made is exposed. Errors in intelligent system decisions become more obvious and easier to detect. This increases the probability of correct takeover and reduces extreme negative paths (i.e., Figure 2 The emergence of path 4). Accordingly, the human-intelligence interaction process more frequently moves toward paths that bring positive experiences or slightly negative experiences, thereby accelerating the stabilization of trust. The combination of the above influences also makes the security of the human-intelligence interaction process converge to a higher level more quickly.

[0016] In step 2, considering the continuity of decision variables and the time dependence of trust, DBN is selected as an effective method to model the human-intelligence interaction process. In order to clarify the modeling concept, Figure 3 A schematic diagram is presented to explain how DBN models the security and evolution of the human-intelligence interaction path. As shown in the example, the human-intelligence interaction path including intelligent system decision-making, SAT information transmission, human takeover decision-making and interactive experience feedback is regarded as an important element in DBN. The process of establishing the network structure includes the following three sub-steps.

[0017] Step 1): Identify key nodes in DBN

[0018] In order to model the dynamic evolution of trust and security of human-intelligence interaction paths, this sub-step identifies DBN nodes to represent human-intelligence interaction paths. Figure 3 As shown, a set of nodes, namely environment, sensor error, and detection, are used to model the actual information detected by the intelligent system under the deviation caused by insufficient sensor function. The reasoning and prediction nodes represent the subsequent decision-making process based on the detection information. In addition, these nodes also represent the situational awareness transparency (SAT) information generated when the intelligent system makes decisions.

[0019] To model the takeover decision supported by transparency, the baseline information and correctness judgment of each SAT information are identified as a set of separate nodes that determine the correctness of each level of SAT, namely SAT correctness nodes. Other nodes, such as situation awareness, takeover decision, trust, and interaction result nodes, are identified to represent takeover decisions and results. In addition, interaction paths, α, β, and trust nodes are identified to describe the trust updates caused by positive and negative interaction paths.

[0020] Step 2): Establish causal relationships within a single human-intelligence interaction path

[0021] The causal relationship in a single time slice of DBN describes a single human-intelligence interaction path and interaction results. The establishment of the causal arc in the decision of the intelligent system starts from the node related to information detection, followed by the reasoning and prediction nodes. In the human takeover decision process, the development of the causal arc starts from the node related to the SAT information correctness judgment, followed by the nodes related to SAT correctness and SA, and finally ends with the node related to the takeover decision. After that, the takeover decision and human error nodes point to the interaction results. Finally, the nodes related to the takeover decision jointly point to the nodes related to the interaction experience, that is, the interaction path.

[0022] In order to simplify the network structure, the present invention proposes a Bayesian network submodule model. The proposed submodule model is a container of Bayesian nodes. Figure 4 Three forms of submodules are illustrated. First, an arrow between two submodules indicates that each node in the parent submodule is the parent node of any node in the child submodule. Second, an arrow from a single node to a submodule indicates that the single node is the parent node of all nodes in the submodule. Third, an arrow from a submodule to a single node means that all nodes in the submodule are the parent nodes of the single node.

[0023] Step 3): Establish causal relationships between adjacent human-intelligence interaction paths

[0024] This substep aims to establish connections between adjacent DBN slices, reflecting trust updates. The updates are based on the trust at the start of the previous human-intelligence interaction path and the interaction experience at the completion of the previous human-intelligence interaction path. The influence of previous trust is described by arcs from the α and β nodes to themselves. The influence of interaction experience is described by arcs from the interaction path nodes in the previous time slice to the α and β nodes in the current time slice. The above nodes and connections are combined with the nodes and connections in a single time slice to describe the dynamic evolution of trust and the security of the human-intelligence interaction path.

[0025] Among them, in step three, after completing the DBN structure, it is necessary to determine the parameters in the DBN, including the prior probability of the root node, the conditional probability distribution of nodes with and without time dependence, including the following steps:

[0026] Step 1): Assign a priori probability to the root node

[0027] like Figure 4As shown in Figure 1, the root nodes in DBN include environment, sensor failure function deficiency, α0 and β0, and human error. The environment node is the input variable for intelligent system decision-making and human cognition, which depends on the actual interaction scenario. The sensor function deficiency node is the deviation of the sensor. These deviations usually follow a normal distribution and can be determined by the performance parameters of the sensor or relevant industry standards. The α0 and β0 nodes are parameters that describe the initial trust distribution and can be determined according to the actual situation of the object being analyzed or different analysis purposes. The human error node represents the failure probability of manual operation, which can be determined by statistical data.

[0028] Step 2): Determine the conditional probability distribution of nodes within a single human-intelligence interaction path

[0029] The goal of this substep is to determine the conditional probability distribution of nodes that have no time dependency. The conditional probability distribution of nodes related to intelligent system decisions and transparency information depends on the algorithm applied by the intelligent system. The conditional probability distribution of nodes related to situational awareness implementation and takeover decisions can be found in Figure 2 The event tree in .

[0030] Among them, the takeover decision can be expressed as:

[0031]

[0032] In formula (1), is the conditional probability, indicating that the trust, SA, interaction results and transparency level in the previous human-intelligence interaction process jointly determine the current trust, t i-1 is the trust at the beginning of the i-1th human-intelligence interaction process, which is consistent with the SA (SA) supported by transparency. i-1 ) and interaction results (CO i-1 ) to determine the interaction path of the previous human-intelligence interaction process. This human-intelligence interaction path is based on t i-1 Update Trust, t i-1 The effect is enhanced by the Transparency Level (TL). It indicates that under the deviation space ξ and transparency level of the current input information error, humans believe that the decision of the intelligent system is not obviously wrong. This belief depends on the correctness judgment of the situational awareness SAT information at a given level, which is based on the information from other reliable sources. Each SAT information There is corresponding benchmark information The correctness of SAT information depends on their consistency, which is usually fuzzy. The present invention uses fuzzy logic to simulate the inherently fuzzy human thinking process. By defining the membership function of the difference between SAT information and its benchmark information, the human belief in the correctness of each SAT information can be quantified, that is, Figure 5 (a) and (b) respectively give examples of membership functions for the consistency between SAT information and benchmark information when it is continuous information and discrete information. The determination of membership functions can be based on relevant industry standards, or conducting questionnaire surveys on operators and using rule-based methods for expert judgment are also effective methods.

[0033] Step 3): Quantify the conditional probability distribution of nodes with time correlation

[0034] This substep aims to determine the conditional probability distribution of nodes with time correlation, that is, α describing the trust update i and β i Node. These conditional probability distributions can refer to formulas (2)-(4).

[0035] Humanity's current trust i Depends on past interaction experience and is subject to parameter α i and β i The Beta distribution, t i ~Beta(α i ,β i ). i and β i Represent the positive and negative experience accumulation respectively. Then, the trust at the completion of the i-1th human-intelligence interaction path can be obtained by t i The mean To estimate. Since t i ~Beta(α i ,β i ), then the trust can be calculated as:

[0036]

[0037] Positive experience accumulation i and negative experience accumulation β i Updated via:

[0038]

[0039] Among them, α i-1 and β i-1 They represent the positive and negative experience accumulation after the end of the i-1th interaction; and are used to represent the experience feedback caused by the positive and negative paths, respectively, and j represents Figure 2 In addition, the interpretation of the intelligent system will increase the expectation of the intelligent system's capabilities and enhance the impact of the interaction experience on the change of trust. The higher the transparency level, the more detailed the interpretation. Amplification factor b Lk(k=1, 2, 3) is used to represent the enhanced impact brought by the three transparency levels.

[0040] Under the guidance of relevant mechanism research and empirical research, the following principles can be set to determine the relative relationship of the impact of each path:

[0041] P1: The human-intelligence interaction path that causes the initial expectation of the intelligent system function to be violated will lead to negative experience. The functions of the intelligent system in emergency situations include intelligent system risk avoidance and takeover decision support based on transparency.

[0043] P2: The failure of two types of functions of the intelligent system leads to a greater reduction in trust than the failure of only one type of function or the failure of both types of functions.

[0045] P3: Providing transparency information to support intelligent systems’ error detection and takeover decisions can inhibit the reduction of trust after negative interactions.

[0047] P4: Due to outcome bias, the experience brought about by the human-intelligence interaction path with ideal outcomes is more positive than the experience brought about by the human-intelligence interaction path with unideal outcomes.

[0049] Then, the relative relationship of trust change for each path can be determined as:

[0050]

[0051] Among them, in step 4, based on the prior probability and conditional probability distribution of all nodes, the joint probability distribution of the DBN modeling the safety evolution of the human-intelligence interaction process is calculated as follows:

[0052]

[0053] Among them, P(Z 1:N ) represents the joint distribution; is the node in the i-th human-intelligence interaction path j ; Is a node The parent node set may contain nodes in the i-th or i-1-th human-intelligence interaction path; M is the number of nodes in a single time slice; N is the number of human-intelligence interaction paths. This calculation can be performed using a Gibbs sampler, or more generally using a Markov chain Monte Carlo simulation. Subsequently, the safety of the human-intelligence interaction process can be expressed as:

[0054]

[0055] Among them, S HAI To ensure the safety of the human-intelligence interaction process, It is the interaction result node in the 1st to Nth human-intelligence interaction process.

[0056] The advantages and beneficial effects of the present invention are:

[0057] The present invention provides a method for assessing the safety of the human-intelligence interaction process taking into account the influence of transparency. The method can systematically analyze the influence of transparency on situational awareness, dynamic evolution of trust and takeover decision, and construct a multi-level dynamic model of the human-intelligence interaction process; it can describe the interaction links such as intelligent system decision errors, personnel takeover decision and trust update, realize a comprehensive assessment of the safety of the human-intelligence interaction process, and output accurate safety indicators and dynamic trust change sequences. The assessment method described in the present invention is scientific and rigorous, the modeling process is efficient, the applicability is strong, and it has a wide range of promotion and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Flowchart for modeling human-intelligence interaction paths based on DBN and evaluating their safety.

[0059] Figure 2 It is a human-intelligence interaction process based on transparency.

[0060] Figure 3 Schematic diagram of DBN-based human-intelligence interaction path security assessment.

[0061] Figure 4 This is an example of a sub-model in DBN.

[0062] Figure 5 is an example of a membership function.

[0063] Figure 6 Transparency information for autonomous collision avoidance systems.

[0064] Figure 7 The DBN structure used to evaluate the safety of human-intelligence interaction paths in the case study.

[0065] Figure 8 is the conditional probability distribution of nodes related to SAT correctness.

[0066] Fig. 9 Trust iteration (a) and convergence speed (b).

[0067] Fig.10 To ensure the security of stable human-intelligence interaction paths at different transparency levels.

[0068] The serial numbers, symbols and codes in the figure are explained as follows:

[0069] Figure 3In the figure, the double-circle nodes with dotted inner circles are nodes related to intelligent system decision-making and transparency; the double-ring circular nodes are nodes related to personnel situational awareness establishment and takeover decision-making; the bold circular nodes are nodes with time correlation, which are used to describe the quantification and update of trust; the bold double-circle nodes are result-related nodes; the solid arrows are arrows between single nodes; the solid arrows with the number "1" are arrows with time dependence. DETAILED DESCRIPTION

[0070] The present invention provides a method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency. Figure 1 As shown; the method is carried out in the following four steps in sequence; the present invention takes a typical risk scenario - the human-intelligence interaction process of the collision avoidance of an intelligent ship at sea as an example. The final guarantee for the successful collision avoidance of an intelligent ship is that the operator takes over correctly when necessary. This depends on whether the transparency of the autonomous navigation system (ANS) can support the operator to correctly establish situational awareness of the correctness of the collision avoidance decision of the intelligent system and form a stable trust after the least human-intelligence interaction process. Therefore, this risk scenario is selected as the research object to demonstrate the effectiveness of the proposed method.

[0071] According to the International Regulations for Preventing Collisions at Sea (COLREGs), there are three situations when two ships encounter each other, considering the different positions of the target ship: overtaking, encountering, and crossing. In various encounter situations, the reliability of the collision avoidance decision of the autonomous navigation system is different, which affects the trust update of the operator. Especially when various encounter situations occur alternately, the evolution of trust becomes more complicated. Therefore, four typical encounter situations are designed in this case study. They are given the same probability of occurrence to represent the uncertainty of the encounter situation, and only one situation occurs in a single human-intelligence interaction process.

[0072] The shapes and initial motion parameters of the own ship (OS, i.e., smart ship) and the target ship (TS) are shown in Table 1. There is a collision risk between the own ship and the target ship. The ship territory is a generalization of the safety distance and is defined as a two-dimensional area around the ship that should not be violated. The situation in which the ship territory is violated is generally considered to be a conflict occurrence including close approach and collision scenarios. In related studies, a circular shape is recommended for open sea areas.

[0073] Table 1 Setting of two-ship encounter situation

[0074]

[0075] The specific implementation plan is described in detail as follows:

[0076] Step 1: Intelligent system decision-making and transparency analysis

[0077] Transparency information of autonomous navigation system at each level such as Figure 6 As shown in Figure 1. SAT1 includes the input data required to make a collision avoidance decision, such as the relative distance, position, speed and heading of the two ships. SAT2 describes the autonomous navigation system's assessment of severity and urgency. SAT3 explains the effectiveness of the collision avoidance decision. That is, if the autonomous navigation system's decision is implemented, the distance to the closest point of approach (DCPA) and time to the closest point of approach (TCPA) of the two ships.

[0078] Step 2: Qualitative Modeling

[0079] Step 1): Identify key nodes in DBN

[0080] In the collision avoidance scenario, the human-intelligence interaction process between the autonomous navigation system and the operator is identified as nodes in the DBN. First, the parameters reflecting the decision and transparency of the autonomous navigation system, such as input data, current collision situation evaluation, and utility evaluation of the collision avoidance decision, are represented by a set of nodes related to the autonomous navigation system. Then, the parameters used to judge the correctness of the transparency information are represented by a set of nodes related to the operator. Other intrinsic nodes are set to reflect takeover decisions, trust updates, and safety evaluations. Table 2 lists the above nodes and their descriptions.

[0081] Table 2 Description of nodes in DBN

[0082]

[0083] Step 2): Establish causal relationships within a single human-intelligence interaction path

[0084] For the establishment of causal relationships within a single human-intelligence interaction path, the key is to represent the autonomous navigation system decision and the operator's cognition. The generation of collision avoidance decisions and transparency information is represented by a causal arc from the motion data of the two ships to the risk level of the encounter situation, and then to the utility of the collision avoidance decision. The operator's cognitive process is represented by a causal arc from the perceived encounter information to each benchmark information, and then to the judgment of the correctness of the SAT information. Other causal arcs can be related to Figure 3 Consistent.

[0085] Step 3): Establish causal relationships between adjacent human-intelligence interaction paths

[0086] The causal relationship between adjacent human-intelligence interaction paths represents the update of trust, which can be compared with Figure 3 Combining the connections within and between human-intelligence interaction paths, the construction of the qualitative DBN model in the case study can be completed as follows: Figure 7As shown. First, the nodes and connections related to the environment, sensor error, and SAT1 information describe the information detection of ANS affected by sensor errors; on this basis, the nodes and connections related to SAT2 information and SAT3 information describe the ANS reasoning decision and decision effect prediction process. On the basis of the nodes related to the environment information, the personnel obtain the benchmark information of SAT1, SAT2, and SAT3, which is represented by the operator's perception, understanding, and prediction related nodes and connections; based on SAT and its benchmark information, the operator judges the correctness of SAT information at each level, which is represented by the SAT1, SAT2, and SAT3 information correctness and its overall correctness related nodes and connections; on this basis, the SA about the correctness of ANS decisions is established, and the takeover decision is made in combination with trust. The above interaction process and interaction results jointly determine the interaction experience brought to the operator by this interaction and update the trust, which is represented by trust-related nodes and connections.

[0087] Step 3: Quantitative Modeling

[0088] Step 1): Assign a priori probability to the root node

[0089] The prior probability of the root node in this case study is set as follows. First, the prior probability of the node related to the environmental data can be determined based on the encounter situation designed in Table 1. Second, the deviation space of the input information is shown in Table 3. It includes the deviation distribution of each detection information, which is determined by the resolution of the International Maritime Organization, namely MSC.192 (79). Third, the failure probability of manual collision avoidance is taken as 0.01. Finally, the operator's initial trust distribution parameters are set to α0 = 100 and β0 = 50.

[0090] Table 3 Error distribution of input information caused by insufficient sensor function

[0091]

[0092] Step 2): Determine the conditional probability distribution of nodes within a single human-intelligence interaction path

[0093] The conditional probability distribution of nodes related to the autonomous navigation system decision can be regarded as a function of the encounter parameters. The conditional probability distribution of nodes related to intelligent system prediction, human understanding, and human prediction are all based on DCPA and TCPA. Among them, the nodes related to human understanding, namely UDH and SDH, are based on the DCPA and TCPA of the current encounter parameters, and the other nodes are based on the DCPA and TCPA after the collision avoidance decision of the autonomous navigation system is executed.

[0094] The conditional probability distribution of nodes related to the correctness of transparency information can be determined by the membership function determined according to MSC.192 (79). Figure 8As shown in (a), the conditional probability distribution of SDC depends on the consistency of SDA and SDH determined by the current DCPA. Similarly, Figure 8 As shown in (b), the conditional probability distribution of UDC depends on the consistency of UDA and UDH determined by the current TCPA. The conditional probability distribution of other nodes related to the correctness of SAT information is a function of the difference between transparency information and benchmark information, and they have the same shape, as shown in Figure 8 As shown in Figure (c).

[0095] Step 3): Quantify the conditional probability distribution of nodes with time dependence

[0096] The time-dependent nodes in this case study are α and β, which are used to quantify trust updates. Their conditional probability distribution can be expressed as formula (3). The key point is to determine the trust updates caused by different human-intelligence interaction paths. and The value of b caused by different transparency levels Lk The impact of paths 1 to 6 is shown in Table 4, and other parameters are based on the assumptions of formula (4).

[0097] Table 4 is used to quantify α and β Variables of node conditional probability distribution

[0098]

[0099] Step 4: Safety assessment of the human-intelligence interaction process

[0100] After determining the prior probability and conditional probability distribution of all nodes, the safety of the human-intelligence interaction process can be calculated using formula (5) and formula (6). In the present invention, the safety of the human-intelligence interaction process is the probability that the autonomous navigation system or the operator successfully avoids the risk of collision. In order to simplify the calculation, MATLAB is used to calculate the joint distribution of DBN.

[0101] After comparison, the effect of transparency on trust formation is analyzed as follows. Fig. 9 As shown. Fig. 9 As can be seen in Figure (a), although the trust at different transparency levels gradually converges, the converged trust value increases with the increase of transparency level. In addition, the rate at which the trust variance interval shrinks indicates that the trust convergence speed increases with the increase of transparency level. In order to intuitively compare the rate of trust stabilization, we analyzed the number of human-intelligence interaction processes required for trust to stabilize, and used the standard error of trust <0.001 as the convergence standard, such as Fig. 9 As shown in Figure (b), it is clear that a higher level of transparency can promote faster trust stabilization.

[0102] The safety of the human-computer interaction process under different transparency levels is as follows: Fig.10 As shown. Obviously, the safety of the human-machine interaction process increases significantly with the increase of transparency level. In addition, at TL1, the safety of the human-machine interaction process is very close to the reliability of the autonomous navigation system. This means that the advantages of the human-machine interaction process in reducing the risk of autonomous ship conflicts cannot be effectively exerted. This is because for the operators in the shore-based control center, transparency information is the only source of situational awareness of the correctness of the collision avoidance decisions made by the autonomous navigation system. At lower transparency levels, less transparency information is provided to the operator. This makes it difficult for the operator to detect errors in the decisions of the autonomous navigation system, which means that the reliability of situational awareness is low. Therefore, when the collision avoidance decision of the autonomous navigation system is wrong, the operator cannot make the correct takeover decision, and their role in preventing conflicts will be reduced.

Claims

1. A method for assessing the safety of human-intelligence interaction processes considering the impact of transparency. The method models situational awareness, trust updates, and takeover decisions in the human-intelligence interaction process through DBN. The single-chip network model in the DBN is based on a single takeover decision based on situational awareness and trust, and the connection between adjacent networks models the trust updates driven by multiple possible human-intelligence interaction paths. It is characterized by: The steps include: Step 1: Intelligent system decision and transparency analysis, obtaining key input information, the generation process of intelligent system decisions, and the key knowledge required for modeling and evaluation of transparency information at each level; Step 2: Qualitative modeling, qualitatively describing the takeover decision process and trust update process through the nodes and connections in the DBN; Step 3: Quantitative modeling: quantify the parameters of each node based on the construction of DBN structure; Step 4: Security assessment of the human-intelligence interaction process. Based on determining the prior distribution of the root node and the conditional probability distribution of the remaining nodes, the security of the human-intelligence interaction process is assessed.

2. According to claim 1, a method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency is characterized in that: In step one, the operating principles of the intelligent system and the transparency information available at each level are studied; after an abnormal situation occurs, the intelligent system uses the detected information and various algorithms to achieve situation reasoning, decision making and decision utility prediction, and will execute the decision with the best expected utility; while these algorithms are running, transparency information at each level is generated; among them, key input information, the generation process of intelligent system decisions and transparency information are important parts of the safety modeling and evaluation of the human-intelligence interaction process.

3. A method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency according to claim 1 or 2, characterized in that: In the human-intelligence interaction process in emergency situations, SAT1 provides inputs and action recommendations detected by the intelligent system; SAT2 provides information about the intelligent system's understanding of current risks; SAT3 provides information about the expected utility of the intelligent system's action recommendations; the transparency of the intelligent system is divided into three levels: TL1=SAT1, TL2=TL1+SAT2, and TL3=TL2+SAT3.

4. According to claim 1, a method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency is characterized in that: In step 2, considering the continuity of decision variables and the time dependence of trust, DBN is selected as an effective method to model the human-intelligence interaction process; The human-intelligence interaction path, including intelligent system decision-making, SAT information transmission, human takeover decision-making and interactive experience feedback, is regarded as an important element in DBN; the process of establishing the network structure includes the following three sub-steps; Step 2.1): Identify key nodes in DBN In order to model the dynamic evolution of trust and the safety of the human-intelligence interaction path, DBN nodes are identified to represent the human-intelligence interaction path; a set of nodes, namely environment, sensor error and detection, are used to model the actual information detected by the intelligent system under the deviation caused by insufficient sensor function; reasoning and prediction nodes represent the subsequent decision-making process based on the detection information; in addition, these nodes also represent the situational awareness transparency (SAT) information generated when the intelligent system makes decisions; To model the takeover decision supported by transparency, the baseline information and correctness judgment of each SAT information are determined as a set of separate nodes that determine the correctness of each level of SAT, namely, SAT correctness nodes; Situational awareness, takeover decision, trust, and interaction outcome nodes are identified to represent takeover decisions and outcomes; In addition, interaction paths, α, β, and trust nodes are identified to describe the trust updates caused by positive and negative interaction paths; Step 2.2): Establish causal relationships within a single human-intelligence interaction path In the human takeover decision process, the development of the causal arc starts from the node related to the judgment of the correctness of SAT information, followed by the nodes related to the correctness of SAT and SA, and ends with the node related to the takeover decision; After that, the takeover decision and human error nodes point to the interaction results; finally, the nodes related to the takeover decision jointly point to the nodes related to the interaction experience, that is, the interaction path; Step 2.3): Establish causal relationships between adjacent human-intelligence interaction paths Establishing connections between adjacent DBN slices reflects trust updates; Update the trust at the beginning of the previous human-intelligence interaction path and the interaction experience at the completion of the previous human-intelligence interaction path; The influence of previous trust is described by arcs from α and β nodes to itself; the influence of interaction experience is described by arcs from the interaction path nodes in the previous time slice to the α and β nodes in the current time slice; The above nodes and connections are combined with the nodes and connections in a single time slice to describe the dynamic evolution of trust and security of human-intelligence interaction paths.

5. According to claim 1, a method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency is characterized in that: In step 3, after completing the DBN structure, it is necessary to determine the parameters in the DBN, including the prior probability of the root node, the conditional probability distribution of nodes with and without time dependence, including the following steps: Step 3.1): Assign a priori probability to the root node The root nodes in DBN include environment, sensor failure function deficiency, α0 and β0, and human error; the environment node is the input variable of intelligent system decision-making and human cognition, which depends on the actual interaction scenario; the sensor function deficiency node is the deviation of the sensor; these deviations follow the normal distribution and are determined by the performance parameters of the sensor or relevant industry standards; the α0 and β0 nodes are parameters describing the initial trust distribution, which are determined according to the actual situation of the analyzed object or different analysis purposes; the human error node represents the failure probability of manual operation, which is determined by statistical data; Step 3.2): Determine the conditional probability distribution of nodes within a single human-intelligence interaction path Determine the conditional probability distribution of nodes without time dependence; the conditional probability distribution of nodes related to the intelligent system decision and transparency information depends on the algorithm applied by the intelligent system; Step 3.3): Quantify the conditional probability distribution of nodes with time correlation Determine the conditional probability distribution of nodes with time correlation, i.e., α describing the trust update i and β i node.

6. The method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency according to claim 5, characterized in that: In step 3.2, the takeover decision is expressed as: In formula (1), is the conditional probability, indicating that the trust, SA, interaction results and transparency level in the previous human-intelligence interaction process jointly determine the current trust, t i-1 is the trust at the beginning of the i-1th human-intelligence interaction process, which is consistent with the SA (SA) supported by transparency. i-1 ) and interaction results (CO i-1 ) to determine the interaction path of the previous human-intelligence interaction process; this human-intelligence interaction path is based on t i-1 Update Trust, t i-1 The effect of transparency level is enhanced by TL; It indicates that under the deviation space ξ and transparency level of the current input information error, humans believe that the decision of the intelligent system is not obviously wrong; this belief depends on the correctness judgment of the situational awareness SAT information at a given level, which is based on the information from other reliable sources; each SAT information There is corresponding benchmark information 7. The method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency according to claim 6, characterized in that: Fuzzy logic is used to simulate the inherently fuzzy human thought process; human beliefs about the correctness of each SAT information are quantified by defining a membership function for the difference between the SAT information and its benchmark information, i.e.

8. The method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency according to claim 5, characterized in that: In step 3.3, the current trust t i Depends on past interaction experience and is subject to parameter α i and β i The Beta distribution, t i ~Beta(α i ,β i ); α i and β i Represent the positive and negative experience accumulation respectively; then, the trust at the completion of the i-1th human-intelligence interaction path is expressed by t i The mean To estimate; due to t i ~Beta(α i ,β i ), then the trust calculation is: Positive experience accumulation i and negative experience accumulation β i Updated via: Among them, α i-1 and β i-1 They represent the positive and negative experience accumulation after the end of the i-1th interaction; and are used to represent the experience feedback caused by positive and negative paths respectively; in addition, the interpretation of the intelligent system will increase the expectation of the intelligent system's capabilities and enhance the impact of the interactive experience on the change of trust; the higher the transparency level, the more detailed the interpretation; the amplification factor b Lk (k=1, 2, 3) is used to represent the enhanced impact brought by the three transparency levels.

9. The method for assessing the safety of a human-intelligence interaction process taking into account the influence of transparency according to claim 8, characterized in that: The following principles are set to determine the relative impact of each path: P1: The human-intelligence interaction path that causes the initial expectation of the intelligent system function to be violated will lead to negative experience; the functions of the intelligent system in emergency situations include intelligent system risk avoidance and takeover decision support based on transparency; P2: The failure of two types of functions of the intelligent system leads to a greater reduction in trust than the failure of only one type of function or the failure of both types of functions; P3: Provide transparency information to support intelligent systems’ error detection and takeover decisions to inhibit the reduction of trust after negative interactions; P4: Due to outcome bias, the experience brought by the human-intelligence interaction path with ideal outcomes is more positive than the experience brought by the human-intelligence interaction path with unideal outcomes; Then, the relative relationship of trust change for each path is determined as:

10. The method for assessing the safety of a human-intelligence interaction process considering the influence of transparency according to claim 1, characterized in that: In step 4, based on the prior probability and conditional probability distribution of all nodes, the joint probability distribution of the DBN modeling the safety evolution of the human-intelligence interaction process is calculated as follows: Among them, P(Z 1:N ) represents the joint distribution; is the node in the i-th human-intelligence interaction path j ; Is a node The parent node set of may include nodes in the i-th or i-1-th human-intelligence interaction path; M is the number of nodes in a single time slice; N is the number of human-intelligence interaction paths; Gibbs sampler is used, or Markov chain Monte Carlo simulation is used; then, the safety of the human-intelligence interaction process is expressed as: Among them, S HAI To ensure the safety of the human-intelligence interaction process, It is the interaction result node in the 1st to Nth human-intelligence interaction process.

Images