Human-machine interface evaluation method, device, equipment and medium

Abstract

The application provides a human-machine interface evaluation method, device, equipment and medium, the method comprises: obtaining human reliability analysis requirements for human-machine interface; the human reliability analysis requirements include a target task for evaluating the human-machine interface; mapping the operation behavior record of the operator when performing the target task with the knowledge graph corresponding to the human-machine interface, to determine a plurality of target operation paths corresponding to the target task; for each target operation path, based on the knowledge graph and the human-machine interface, determine the interface indicators corresponding to the target operation path; input the interface indicators and the operator type of the operator into the pre-trained probability prediction model to obtain the probability distribution of the target operation path under a plurality of preset operation state labels; based on the probability distribution and the interface indicators corresponding to each target operation path, determine the evaluation result of the human-machine interface. Through human reliability analysis, the interface evaluation is improved, and the explainability and prediction ability of human-induced failures of the human-machine interface are improved.

Description

Technical Field

[0001] This application relates to the field of computer software technology, and more specifically, to a human-computer interface evaluation method, apparatus, device, and medium. Background Technology

[0002] With the increasing digitalization and automation of industrial systems, human-machine interfaces (HMIs) have become a crucial element for operators in high-risk industries such as nuclear power, aviation, and chemicals to obtain information and perform control operations. Especially in the main control room of a digital nuclear power plant, operators need to maintain a high level of situational awareness amidst complex information displays and multifunctional interactive environments; their operational reliability is directly related to system safety.

[0003] In fully digital interface environments, abundant visual information often leads to perceptual overload, cognitive conflict, and operational delays, becoming major triggers for abnormal human operations. However, interface evaluations often focus on general usability or single psychological overload indicators, failing to effectively couple them with the probability of human error. Furthermore, while a large amount of operator behavior data can be collected, there is a lack of effective methods to combine interface structure indicators with behavioral data to achieve real-time risk identification and trend prediction in the operational process, resulting in insufficient refined diagnostic capabilities for interface-induced risks. Summary of the Invention

[0004] In view of this, this application provides a human-machine interface evaluation method, apparatus, device and medium, which maps operation behavior records to the knowledge graph corresponding to the human-machine interface, evaluates the interface through human factor reliability analysis, and improves the interpretability and predictability of human-machine interface-induced human-factor failures.

[0005] Specifically, this application is implemented through the following technical solution: According to a first aspect of this application, a human-computer interface evaluation method is provided, the method comprising: Obtain human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, the target tasks indicating the search for target elements from the human-machine interface; A knowledge graph corresponding to the human-machine interface is constructed, and operation behavior records of the operator when performing the target task are obtained. The operation behavior records are mapped to the knowledge graph, and multiple target operation paths corresponding to the target task are determined from multiple operation paths. The operation path is a connecting line between two knowledge graph nodes, and the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface. The operation path is used to indicate two consecutive operation behaviors. For each target operation path, based on the knowledge graph and the human-computer interface, an interface indicator corresponding to the target operation path is determined. The interface indicator is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path. Input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. The evaluation result of the human-machine interface is determined based on the probability distribution and interface indicators corresponding to each of the target operation paths.

[0006] In one optional implementation, the interface metrics include visual density, semantic interference density, and interaction span. The step of determining the interface metrics corresponding to the target operation path based on the knowledge graph and the human-computer interface includes: Based on the knowledge graph and the human-computer interface, determine the pointer movement distance when executing the target operation path, the maximum movement distance in the human-computer interface, and the number of elements, the number of first element pairs, and the number of second element pairs included in the human-computer interface; the first element pair includes any two elements in the human-computer interface, and the second element pair includes the target element and other elements whose semantic similarity to the target element is greater than a preset threshold. The visual density is determined based on the number of target elements and the number of elements. The semantic interference density is determined based on the number of the first element pairs and the number of the second element pairs; The interaction span is determined based on the pointer movement distance and the maximum movement distance.

[0007] In one optional implementation, the plurality of preset operation status labels include error-free execution status labels, omission status labels, execution deviation status labels, result deviation status labels, and timing deviation status labels; the probability prediction model is a hierarchical discriminative structure, and the probability prediction model includes an anomaly discrimination unit, a timing deviation discrimination unit, an error type classification unit, and an anomaly probability combination unit. The step of inputting the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels includes: The anomaly detection unit performs anomaly detection based on the interface indicators and the operator's operator type to obtain the probability of the target operation path under the error-free execution status label. The timing deviation discrimination unit, when the probability of the target operation path under the error-free execution state label is less than 1, performs timing deviation discrimination based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the timing deviation state. By using the error type classification unit, when the probability of the target operation path belonging to the timing deviation state is less than 1, the error type is classified based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the omission state, execution deviation state, and result deviation state, respectively. The anomaly probability combination unit determines the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively.

[0008] In an optional implementation, determining the probability of the target operation path being under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively, based on the probability of the target operation path being under the error-free execution state label and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, includes: Based on the probability that the target operation path is under the error-free execution status label, the probability that the target operation path belongs to an abnormal state is determined; Based on the probability that the target operation path belongs to a timing deviation state, the probability that the target operation path does not belong to a timing deviation state is determined. Based on the probability that the target operation path belongs to an abnormal state and the probability that it belongs to a timing deviation state, the probability that the target operation path is under the timing deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an omission state, the probability that the target operation path is under the omission state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an execution deviation state, the probability that the target operation path is under the execution deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to a result deviation state, the probability that the target operation path is under the result deviation state label is determined.

[0009] In one optional implementation, the probability prediction model is trained through the following steps: Obtain test operation behavior records when test operators execute test tasks, map the test operation behavior records to the knowledge graph, and determine multiple test operation paths corresponding to the test task from multiple operation paths; For each of the test operation paths, based on the knowledge graph and the human-computer interface, the test interface indicators corresponding to the test operation path are determined; Based on the test operation behavior record and the specified behavior requirements of the test task, a target operation status label that matches the test operation path is determined from multiple preset operation status labels. The test interface indicators and the operator type of the test operator are input into the neural network to obtain the test probability distribution of the test operation path under multiple preset operation state labels. Based on the target operation state label corresponding to the test operation path and the test probability distribution, the parameters of the neural network are adjusted until the training cutoff condition is met, and a trained probability prediction model is obtained.

[0010] In one optional implementation, the probability prediction model is a hierarchical discriminative structure, and the probability prediction model includes an anomaly discrimination unit, a time series deviation discrimination unit, an error type classification unit, and an anomaly probability combination unit. The following steps are used to determine if a neural network meets the cutoff condition: The neural network meets the cutoff condition when the negative log-likelihood function of the corresponding layer converges, the prediction probability of the neural network for the target operation state label is stable, the output probabilities of the anomaly discrimination unit and the timing deviation discrimination unit meet the consistency condition, and the value of the negative log-likelihood function of the layer does not decrease within a preset time window; the negative log-likelihood function of the layer is determined based on the test probability of the test operation path under the target operation state label.

[0011] In one optional implementation, the plurality of preset operation status labels include an error-free execution status label, an omission status label, an execution deviation status label, a result deviation status label, and a timing deviation status label. The step of determining the evaluation result of the human-computer interface based on the probability distribution and interface indicators corresponding to each of the target operation paths includes: For each target operation path, if the probability of the target operation path under the error-free execution state label is less than 1, the combined probability of the execution error corresponding to the target operation path is determined based on the probability of the target operation path under the omission state label, the execution deviation state label, and the result deviation state label, respectively. Based on the probability of the target operation path under the time deviation status label, the corresponding execution error synthesis probability, and the interface index, the human factor reliability analysis result corresponding to the target operation path is determined. Based on the human factor reliability analysis results corresponding to each of the target operation paths, the evaluation results of the human-machine interface are determined.

[0012] According to a second aspect of this application, a human-machine interface evaluation device is provided, the device comprising: The requirement acquisition module is used to acquire human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, and the target tasks indicate how to find target elements from the human-machine interface; The path mapping module is used to construct a knowledge graph corresponding to the human-machine interface, obtain the operation behavior records of the operator when performing the target task, map the operation behavior records to the knowledge graph, and determine multiple target operation paths corresponding to the target task from multiple operation paths; the operation path is a connecting line between two knowledge graph nodes, the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface, and the operation path is used to indicate two consecutive operation behaviors. The indicator determination module is used to determine the interface indicator corresponding to each target operation path based on the knowledge graph and the human-computer interface. The interface indicator is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path. The probability prediction module is used to input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. The result determination module is used to determine the evaluation result of the human-machine interface based on the probability distribution and interface indicators corresponding to each of the target operation paths.

[0013] According to a third aspect of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the human-computer interface evaluation method described in the first aspect above.

[0014] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the human-machine interface evaluation method described in the first aspect above.

[0015] The human-machine interface (HMI) evaluation method, apparatus, device, and medium provided in this application construct a knowledge graph corresponding to the HMI and map the operator's operational behavior records when performing a target task to the knowledge graph. This effectively determines multiple target operation paths corresponding to the target task. By combining the knowledge graph and the HMI, interface indicators corresponding to each target operation path are determined, thereby quantifying the influencing factors of the interface. The interface indicators and the operator's operator type are input into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. This effectively determines the impact of interface factors on the probability of human-caused failure. Based on human-caused reliability analysis, the HMI is evaluated, achieving real-time risk identification and trend prediction of the operation process. It can effectively identify high-risk operation paths and their corresponding interface-induced factors, achieving dynamic diagnosis and quantitative evaluation of operational behavior and key interaction nodes. This helps improve the interpretability and predictive ability of HMI-induced human-caused failures, providing a scientific and reliable basis for the optimized design of the HMI and risk warning for the operator.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a human-computer interface evaluation method according to an exemplary embodiment of this application; Figure 2 This is a schematic diagram illustrating multiple target operation paths corresponding to a target task in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating the structure of a probability prediction model according to an exemplary embodiment of this application; Figure 4 This is a schematic diagram illustrating a human-computer interface evaluation process according to an exemplary embodiment of this application; Figure 5 This is a schematic diagram of a human-machine interface evaluation device shown in an exemplary embodiment of this application; Figure 6 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0020] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0022] Research has revealed that in fully digital interface environments, design issues such as high visual information density, numerous semantically similar elements, and dispersed interaction paths often lead to perceptual load, cognitive conflict, and operational delays, becoming major contributing factors to layout omissions, misoperations, and execution deviations. However, interface evaluations often focus on general usability or single psychological load indicators, failing to effectively couple them with the probability of human error. Furthermore, while a large amount of operator behavior data can be collected, there is a lack of effective methods to combine interface structure indicators with behavioral data to achieve real-time risk identification and trend prediction during operation. Risk identification largely relies on human experience, lacking the ability to provide refined diagnosis of interface-induced risks. Existing human factor reliability analysis methods, such as the Technique for Human Error Rate Prediction (THERP), the Standardized Plant Analysis Risk-Human Reliability Analysis Method (SPAR-H), and the Integrated Human Event Analysis and Simulation System (IDHEAS), primarily rely on expert experience and static evaluation models, limiting their ability to represent dynamic changes in operational behavior and individual differences. These methods generally employ qualitative or semi-quantitative approaches, failing to accurately characterize the impact of interface factors on the probability of human-caused failures.

[0023] Based on the above research, this application provides a human-computer interface evaluation method. By constructing a knowledge graph corresponding to the human-computer interface and mapping the operator's operational behavior records when performing the target task to the knowledge graph, multiple target operation paths corresponding to the target task are effectively determined. By combining the knowledge graph and the human-computer interface, interface indicators corresponding to each target operation path are determined, thereby quantifying the influencing factors of the interface. The interface indicators and the operator's operator type are input into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels, effectively determining the impact of interface factors on the probability of human-caused failure. Based on human-caused reliability analysis, the evaluation of the human-computer interface is completed, realizing real-time risk identification and trend prediction of the operation process.

[0024] To facilitate understanding of this embodiment, a detailed description of the human-computer interface evaluation method disclosed in this application embodiment is provided first. The execution entity of the human-computer interface evaluation method provided in this application embodiment is generally an electronic device with a certain computing power. This electronic device can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. In some possible implementations, this human-computer interface evaluation method can be implemented by a processor calling computer-readable instructions stored in memory.

[0025] The following description, in conjunction with the accompanying drawings, illustrates a human-machine interface evaluation method provided in an embodiment of this application.

[0026] See Figure 1 The diagram shown is a flowchart illustrating a human-computer interface evaluation method according to an exemplary embodiment of this application. Figure 1 As shown in the figure, the human-machine interface evaluation method provided in this embodiment includes steps S101 to S105, wherein: S101: Obtain human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, the target tasks indicating the search for target elements from the human-machine interface.

[0027] Here, the human-machine interface can be an interface that enables human-machine interaction, and the operator can complete tasks by operating the human-machine interface. This embodiment of the disclosure does not limit the business type of the human-machine interface. For example, the human-machine interface can be a human-machine interface in the control room of a digital nuclear power plant.

[0028] The human-machine interface includes multiple visible interface elements. Taking the human-machine interface in the control room of a digital nuclear power plant as an example, the elements included in the human-machine interface are such as "Nuclear Island System", "2 LAB DW001", "0 KBE DW101", "Nuclear Island Auxiliary", "1 JET DW001", "Conventional Island System", and "0 PCB DW001". Here, "Nuclear Island System", "Nuclear Island Auxiliary" and "Conventional Island System" are at the same level. "2 LAB DW001" and "0 KBE DW101" are at the next level below "Nuclear Island System", "1 JET DW001" is at the next level below "Nuclear Island Auxiliary", and "0 PCB DW001" is at the next level below "Conventional Island System".

[0029] The target task is used to instruct the search for a target element in the human-machine interface. For example, taking the human-machine interface in the control room of a digital nuclear power plant as an example, the target task can be to find whether the value of parameter 2LBA10CP801C of 2 LAB DW001 under the nuclear island system is 13.86MPa21.

[0030] S102: Construct a knowledge graph corresponding to the human-machine interface, obtain the operation behavior records when the operator performs the target task, map the operation behavior records to the knowledge graph, and determine multiple target operation paths corresponding to the target task from multiple operation paths; the operation path is a connecting line between two knowledge graph nodes, the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface, and the operation path is used to indicate two consecutive operation behaviors.

[0031] In this step, a knowledge graph corresponding to the human-computer interface can be constructed. Specifically, all elements in the human-computer interface are extracted, the hierarchical relationship between the elements is determined, and the elements are connected according to the hierarchical relationship to generate the knowledge graph corresponding to the human-computer interface. Here, the knowledge graph includes multiple knowledge graph nodes, and each knowledge graph node corresponds one-to-one with an element in the human-computer interface. The connecting line between two knowledge graph nodes constitutes an operation path. It can be understood that each element in the human-computer interface can be triggered by the user, and correspondingly, each knowledge graph node corresponds to an operation behavior. The operation path is used to indicate two consecutive operation behaviors. Optionally, to facilitate path management, each operation path can be labeled with a path number to facilitate path differentiation.

[0032] When acquiring the operator's operation behavior record when performing the target task, optionally, it can be collected through a control system that matches the human-machine interface. The control system can be a simulated control system or a real control system, and there is no limitation here.

[0033] The operation behavior record may include the operation behavior itself, which is represented by the coordinates of the triggered element or the clicked interface. It can be understood that the triggered element can be determined by the clicked interface coordinates. The operation behavior record may also include the operation time and the result of the operation behavior.

[0034] The operation behavior records are mapped to the knowledge graph. The knowledge graph nodes corresponding to the triggered elements are determined sequentially from the knowledge graph. The determined knowledge graph nodes are connected in pairs according to the triggering order to obtain multiple target operation paths.

[0035] For example, see [link / reference] Figure 2This is a schematic diagram illustrating multiple target operation paths corresponding to a target task, as shown in an exemplary embodiment of this application. This example uses a human-machine interface in a digital nuclear power plant control room, where the target task is to determine whether the value of parameter 2LBA10CP801C of 2 LAB DW001 in the nuclear island system is 13.86 MPa. Figure 2 As shown, the triggered elements are, in order, “Initial Interface”, “Flowchart”, “Nuclear Island System”, “2 LAB DW001” and “2LBA10CP801C”. By connecting the knowledge graph nodes corresponding to the triggered elements in pairs according to the triggering order, four target operation paths can be obtained.

[0036] S103: For each target operation path, based on the knowledge graph and the human-computer interface, determine the interface index corresponding to the target operation path. The interface index is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path.

[0037] In this step, for each target operation path, based on the knowledge graph and the human-computer interface, the interface indicators corresponding to the target operation path are determined to quantify the influencing factors of the interface.

[0038] In some possible implementations, the interface metrics include visual density (VD), semantic interference density (SID), and interaction span (IS). The step of determining the interface metrics corresponding to the target operation path based on the knowledge graph and the human-computer interface includes: Based on the knowledge graph and the human-computer interface, determine the pointer movement distance when executing the target operation path, the maximum movement distance in the human-computer interface, and the number of elements, the number of first element pairs, and the number of second element pairs included in the human-computer interface; the first element pair includes any two elements in the human-computer interface, and the second element pair includes the target element and other elements whose semantic similarity to the target element is greater than a preset threshold. The visual density is determined based on the number of target elements and the number of elements. The semantic interference density is determined based on the number of the first element pairs and the number of the second element pairs; The interaction span is determined based on the pointer movement distance and the maximum movement distance.

[0039] In the above steps, based on the knowledge graph and the human-computer interface, the pointer movement distance when executing the target operation path, the maximum movement distance in the human-computer interface, and the number of elements, the number of first element pairs, and the number of second element pairs included in the human-computer interface can be determined. Here, the distance the pointer needs to move on the human-computer interface when triggering the elements corresponding to the two knowledge graph nodes connected by the target operation path can be determined as the pointer movement distance when executing the target operation path. The maximum movement distance in the human-computer interface is the maximum distance the pointer can move in the human-computer interface, which is generally the distance between two diagonal vertices of the human-computer interface.

[0040] The specific value of the preset threshold depends on the actual needs of the interface quantification and is not limited here. For example, the preset threshold can be 0.8.

[0041] The visual density is defined as the ratio between the number of target elements and the total number of elements, where the number of target elements is 1. Visual density can be used to measure the visual occupancy ratio and the degree of interface crowding.

[0042] For example, the visual density can be determined by the following formula (1): (1) in, This represents the visual density. This indicates the number of the target elements. This indicates the number of elements.

[0043] The ratio between the number of the second element pairs and the number of the first element pairs is determined as the semantic interference density. This semantic interference density can be used to measure the risk of semantic confusion.

[0044] For example, the semantic interference density can be determined by the following formula (2): (2) in, This represents the semantic interference density. Indicates the quantity of the second element pair. This indicates the quantity of the first element pair.

[0045] The ratio between the pointer movement distance and the maximum movement distance is determined as the interaction span. The interaction span can be used to characterize the operational range and workload of the target operation path.

[0046] For example, the interaction span can be determined by the following formula (3): (3) in, Indicates the interaction span, This indicates the distance the pointer has moved. This indicates the maximum travel distance.

[0047] By defining three interface metrics—visual density, semantic interference density, and interaction span—the complexity of the interface and the sources of potential cognitive load corresponding to the target operation path can be quantitatively represented and accurately defined. This provides clear quantitative basis and objective standards for the determination of interface metrics, effectively improving their scientific rigor, accuracy, and interpretability. It also provides reliable and quantitative support for the indicator data of the subsequent input probability prediction model, ensuring the accuracy and rationality of human-computer interface evaluation.

[0048] S104: Input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels.

[0049] In this step, the interface indicators and the operator's operator type can be input into a pre-trained probability prediction model. The probability prediction model can analyze the interface indicators and the operator's operator type, establish the relationship between interface structural features and the probability under each preset operation state label, and output the probability distribution of the target operation path under multiple preset operation state labels.

[0050] The operator type can be, for example, a student, an operator, etc.

[0051] For example, the input to the probability prediction model can be represented by the following formula (4): x={z,c}={VD,SID,IS,c}(4) Where x represents the input of the probability prediction model, z represents the interface index, VD represents the visual density, SID represents the semantic interference density, IS represents the interaction span, and c represents the operator type of the operator.

[0052] In some possible implementations, the plurality of preset operation status labels include an error-free execution status label, an omission status label, an execution deviation status label, a result deviation status label, and a timing deviation status label.

[0053] The error-free execution status label is labeled S0, indicating that the operator completed the operation correctly as expected, without any deviation. The omission status label, also known as the missing or non-coverage status label, is labeled S1, indicating that the operation steps that should have been performed were missing or not entered, resulting in an incomplete target task. The commission error (EOC–execution) status label is labeled S2, indicating that an incorrect control or instruction was executed, indicating an error in the operation itself. The commission outcome error (EOC–outcome) status label is labeled S3, indicating that the operation appeared correct but produced an incorrect or unexpected result. The time-deviated operation status label is labeled S4, indicating that the operation time exceeded a set threshold or the operation sequence was inconsistent, manifesting as delays or abnormal operation timing.

[0054] For example, see [link / reference] Figure 3 This is a schematic diagram illustrating the structure of a probability prediction model, as shown in an exemplary embodiment of this application. Figure 3 As shown, the probability prediction model has a hierarchical discriminative structure, which includes an anomaly discrimination unit, a time series deviation discrimination unit, an error type classification unit, and an anomaly probability combination unit.

[0055] In some possible implementations, the step of inputting the interface metrics and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels includes: The anomaly detection unit performs anomaly detection based on the interface indicators and the operator's operator type to obtain the probability of the target operation path under the error-free execution status label. The timing deviation discrimination unit, when the probability of the target operation path under the error-free execution state label is less than 1, performs timing deviation discrimination based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the timing deviation state. By using the error type classification unit, when the probability of the target operation path belonging to the timing deviation state is less than 1, the error type is classified based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the omission state, execution deviation state, and result deviation state, respectively. The anomaly probability combination unit determines the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively.

[0056] In the above steps, the anomaly detection unit can first perform anomaly detection based on the interface indicators and the operator's operator type to obtain the probability of the target operation path under the error-free execution status label.

[0057] For example, the probability of the target operation path under the error-free execution state label can be determined by the following formula (5): (5) in, This indicates the probability of the target operation path under the error-free execution status label; Indicates a no-error execution status; This refers to the interface metrics; , indicating the activation (Sigmoid) function; This represents the base log-odds probability that the target operation path is determined to be executed normally when all features are zero (in a standardized sense). This represents the weight of the impact of visual complexity on the probability of normal execution. This represents the visual density; The weight representing the impact of semantic interference on the normal execution determination; This represents the semantic interference density; This represents the weight of the impact of the discreteness of the operation path on execution stability. Indicates the interaction span; This represents the influence coefficient of the context weight vector in the anomaly detection unit; Indicates the operator type of the operator; This constitutes an additional linear compensation term.

[0058] The timing deviation discrimination unit determines the probability that the target operation path belongs to the timing deviation state when the probability of the target operation path being under the error-free execution state label is less than 1. Based on the interface indicators and the operator type of the operator, the timing deviation discrimination is performed to obtain the probability that the target operation path belongs to the timing deviation state. It can be understood that when the probability of the target operation path being under the error-free execution state label is less than 1, it can be determined that an anomaly has occurred, but the anomaly type has not yet been distinguished. The timing deviation discrimination unit can determine whether a timing deviation exists.

[0059] For example, the probability that the target operation path belongs to a timing deviation state can be determined by the following formula (6): (6) in, This indicates the probability that the target operation path belongs to a timing deviation state; Indicates the timing deviation status; Indicates an abnormal state; This refers to the interface metrics; , indicating the activation (Sigmoid) function; This represents the base log odds that the anomalous behavior belongs to the time bias type when all features are zero (in a standardized sense); The weight representing the impact of visual complexity on the probability of time deviation occurring; This represents the visual density; Indicates the weight of the impact of semantic interference on the determination of time deviation; This represents the semantic interference density; This represents the weight of the impact of operational path discreteness on time deviation. Indicates the interaction span; This represents the influence coefficient of the context weight vector in the timing deviation discrimination unit; Indicates the operator type of the operator; This constitutes an additional linear compensation term.

[0060] By using the error type classification unit, when the probability that the target operation path belongs to the timing deviation state is less than 1, the error type is classified based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the omission state, execution deviation state, and result deviation state, respectively. It can be understood that when the probability that the target operation path belongs to the timing deviation state is less than 1, it can be determined that an anomaly has occurred. Without considering the time deviation, the error type classification unit can classify and distinguish omission, execution deviation, and result deviation.

[0061] For example, the probabilities that the target operation path belongs to the omission state, the execution deviation state, and the result deviation state can be determined by the following formula (7): (7) in, This represents the probability that the target operation path belongs to the omission state, the execution deviation state, and the result deviation state, respectively. Represents the j-th type of state. Indicates an omission status. Indicates the execution deviation status. Indicates the deviation status of the result. Represents the linear prediction term for the j-th type of anomaly. Represents the linear prediction term for the k-th type of anomaly. This represents the sum of unnormalized scores for all candidate anomaly types (omission status, execution deviation status, and result deviation status).

[0062] Here, the linear prediction term for the j-th type of anomaly It can be determined by the following formula (8): (8) in, Represents the linear prediction term for the j-th type of anomaly; This represents the basic log-odds of the j-th anomaly type when all features are zero (in a standardized sense); This represents the weight of the influence of visual complexity on the probability of occurrence of the j-th anomaly type. This represents the visual density; This represents the weight of the influence of semantic interference on the determination of the j-th type of anomaly. This represents the semantic interference density; This represents the weight of the impact of the discreteness of the operation path on the j-th type of anomaly. Indicates the interaction span; This represents the influence coefficient of the context weight vector in the error type classification unit; Indicates the operator type of the operator; This constitutes an additional linear compensation term.

[0063] The anomaly probability combination unit determines the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively.

[0064] In this way, for five preset operation status labels—error-free execution, omission, execution deviation, result deviation, and timing deviation—a hierarchical discriminative probabilistic prediction model with four units—anomaly detection, timing deviation detection, error type classification, and anomaly probability combination—is adopted. Combining interface indicators and operator type, the model performs a progressive hierarchical process: first overall anomaly detection, then specific timing deviation detection, and finally omission, execution deviation, and result deviation classification. The anomaly probability combination unit integrates the various probability results to accurately determine the probability distribution of the target operation path under each preset operation status label. This achieves refined, hierarchical, and targeted discrimination of different human-machine operation states, especially various error states, effectively avoiding confusion in the discrimination of different error states. It improves the logic, accuracy, and hierarchy of probability prediction, providing accurate and reliable data support for subsequent human-machine interface evaluation based on probability distribution.

[0065] In some possible implementations, determining the probability of the target operation path being under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, based on the probability of the target operation path being under the error-free execution state label and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, includes: Based on the probability that the target operation path is under the error-free execution status label, the probability that the target operation path belongs to an abnormal state is determined; Based on the probability that the target operation path belongs to a timing deviation state, the probability that the target operation path does not belong to a timing deviation state is determined. Based on the probability that the target operation path belongs to an abnormal state and the probability that it belongs to a timing deviation state, the probability that the target operation path is under the timing deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an omission state, the probability that the target operation path is under the omission state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an execution deviation state, the probability that the target operation path is under the execution deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to a result deviation state, the probability that the target operation path is under the result deviation state label is determined.

[0066] In the above steps, the difference between 1 and the probability of the target operation path under the error-free execution state label is determined as the probability that the target operation path belongs to an abnormal state. For example, the probability of the target operation path under the error-free execution state label is... The probability that the target operation path belongs to an abnormal state is .

[0067] The difference between 1 and the probability that the target operation path belongs to a timing deviation state is determined as the probability that the target operation path does not belong to a timing deviation state. For example, the probability that the target operation path belongs to a timing deviation state is... The probability that the target operation path does not belong to the timing deviation state is .

[0068] The product of the probability that the target operation path belongs to an abnormal state and the probability that it belongs to a timing deviation state is determined as the probability that the target operation path is under the timing deviation state label.

[0069] For example, the probability of the target operation path under the timing deviation state label can be determined by the following formula (9): (9) in, This indicates the probability that the target operation path falls under the timing deviation state label. This indicates the probability that the target operation path belongs to an abnormal state. This indicates the probability that the target operation path belongs to a timing deviation state.

[0070] The product of the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an omission state is determined as the probability that the target operation path is under the omission state label.

[0071] For example, the probability that the target operation path is under the missing state label can be determined by the following formula (10): (10) in, This indicates the probability that the target operation path is under the missing status label. This indicates the probability that the target operation path belongs to an abnormal state. This indicates the probability that the target operation path does not belong to a timing deviation state. This indicates the probability that the target operation path belongs to the omitted state.

[0072] The product of the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an execution deviation state is determined as the probability that the target operation path is under the execution deviation state label.

[0073] For example, the probability of the target operation path under the execution deviation state label can be determined by the following formula (11): (11) in, This indicates the probability that the target operation path is under the execution deviation state label. This indicates the probability that the target operation path belongs to an abnormal state. This indicates the probability that the target operation path does not belong to a timing deviation state. This indicates the probability that the target operation path belongs to an execution deviation state.

[0074] The product of the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to a result deviation state is determined as the probability that the target operation path is under the result deviation state label.

[0075] For example, the probability of the target operation path under the result deviation status label can be determined by the following formula (12): (12) in, This indicates the probability of the target operation path under the result deviation status label. This indicates the probability that the target operation path belongs to an abnormal state. This indicates the probability that the target operation path does not belong to a timing deviation state. This indicates the probability that the target operation path belongs to the result deviation state.

[0076] In this way, through the above method, the probability of each preset operation state label is accurately converted by scientific probability correlation deduction, which effectively avoids the result deviation caused by independently calculating the probability of each state. This makes the probability distribution of the target operation path under each error state label more consistent with the actual human-computer operation state occurrence logic, and significantly improves the accuracy, logic and reliability of the probability calculation results.

[0077] In some possible implementations, the probabilistic prediction model is trained through the following steps: Obtain test operation behavior records when test operators execute test tasks, map the test operation behavior records to the knowledge graph, and determine multiple test operation paths corresponding to the test task from multiple operation paths; For each of the test operation paths, based on the knowledge graph and the human-computer interface, the test interface indicators corresponding to the test operation path are determined; Based on the test operation behavior record and the specified behavior requirements of the test task, a target operation status label that matches the test operation path is determined from multiple preset operation status labels. The test interface indicators and the operator type of the test operator are input into the neural network to obtain the test probability distribution of the test operation path under multiple preset operation state labels. Based on the target operation state label corresponding to the test operation path and the test probability distribution, the parameters of the neural network are adjusted until the training cutoff condition is met, and a trained probability prediction model is obtained.

[0078] The parameters of the neural network can be obtained through... Regular logistic regression and multinomial logistic regression can be used for estimation. Hierarchical Bayesian priors and random effects of participants / tasks can also be introduced in the case of small samples to achieve partial pooling and generalization for different operators and task scenarios.

[0079] In this way, the training of the probabilistic prediction model relies entirely on real data that fits actual human-computer interaction scenarios, and uses precisely labeled operation state tags as supervision. This allows the model to fully learn and fit the inherent correlation between interface indicators, operator types, and operation state probability distributions, effectively ensuring the scientific nature of model training, the authenticity of data, and the accuracy of labels. The resulting probabilistic prediction model has good scenario adaptability, prediction accuracy, and generalization ability, and can accurately match the prediction requirements for the operation state probability distribution of target operation paths in human-computer interface evaluation.

[0080] In some possible implementations, the probability prediction model is a hierarchical discriminative structure, which includes an anomaly detection unit, a time series deviation detection unit, an error type classification unit, and an anomaly probability combination unit. The following steps are used to determine if a neural network meets the cutoff condition: The neural network meets the cutoff condition when the negative log-likelihood function of the corresponding layer converges, the prediction probability of the neural network for the target operation state label is stable, the output probabilities of the anomaly discrimination unit and the timing deviation discrimination unit meet the consistency condition, and the value of the negative log-likelihood function of the layer does not decrease within a preset time window; the negative log-likelihood function of the layer is determined based on the test probability of the test operation path under the target operation state label.

[0081] In this step, to ensure the stable convergence of the probabilistic prediction model under small sample conditions and to avoid overfitting, a composite training cutoff condition based on the consistency between the labeled data and the model's probability output is adopted during training. This strategy simultaneously considers model likelihood convergence, probabilistic prediction stability, and the consistency of the hierarchical gating structure.

[0082] Although the probability prediction model ultimately outputs the probabilities of the target operation path under multiple preset operation state labels, its training objective is not based on a single normalization (softmax) function, but rather on a hierarchical negative log-likelihood function constructed through a combination of hierarchical conditional probabilities. Let the true label state of the nth sample be... The neural network predicts the probability of this state as follows: The overall training loss (i.e., the negative log-likelihood function of the hierarchy) is expressed by the following formula (13): (13) in, This represents the overall training loss, where N represents the total number of samples. This represents the true label annotation status of the nth sample. This represents the probability that the neural network predicts the nth sample to be in that state.

[0083] When the rate of change of the overall training loss between two adjacent iterations meets the corresponding preset threshold, and this condition is maintained for several consecutive iterations, the negative log-likelihood function of the corresponding layer of the neural network is determined to be converged.

[0084] In the validation set, the model's prediction confidence for the real labeled data is further monitored. Specifically, the predicted probability corresponding to the real state of each validation sample is calculated, and the average of these probabilities is used as the overall confidence index. When this overall confidence index no longer significantly improves over several consecutive training epochs, the model is considered to have stabilized in terms of discriminative ability, and further training will hardly improve generalization performance. Therefore, it can be determined that the neural network's prediction probability for the target operation state label is stable.

[0085] Since the probability prediction model adopts a layer-by-layer gating inference structure, this embodiment incorporates stability constraints for the key decision layers. For samples with error-free execution states directly determined by the anomaly detection unit, the corresponding gating probability output exhibits low variance characteristics in the later stages of training. For samples with time-deviation states determined by the time-deviation detection unit, the conditional probability output of these samples remains stable. This confirms that the output probabilities of the anomaly detection unit and the time-deviation detection unit meet the consistency condition, thus ensuring that the decision mechanisms at each layer do not oscillate or degenerate during training.

[0086] As a robustness safeguard, an early stopping strategy is introduced on the validation set. When the value of the negative log-likelihood function of the hierarchy (i.e., the overall training loss of the validation set) does not decrease further within a preset patience window, the training process is considered to be terminated to prevent the neural network from overfitting the training samples.

[0087] In this way, by adopting multi-dimensional neural network training cutoff conditions, the limitations of a single judgment criterion are overcome. This not only conforms to the structural characteristics of hierarchical discriminative models and ensures the coordination and rationality of the outputs of each level of discriminative units, but also comprehensively controls the training degree from dimensions such as model loss convergence, prediction result stability, and loss trend stagnation. This effectively avoids the problems of insufficient model training or overfitting, and ensures that the model fully learns and fits the inherent correlation between relevant features and the probability distribution of operational states. This significantly improves the prediction accuracy, structural adaptability, and reliability of the output results of the probabilistic prediction model after training.

[0088] S105: Based on the probability distribution and interface indicators corresponding to each of the target operation paths, determine the evaluation result of the human-machine interface.

[0089] In this step, based on the probability distribution and interface indicators corresponding to the target operation path, high-risk paths and corresponding interface triggering factors can be identified, and the evaluation results of the human-machine interface can be obtained, providing a basis for interface optimization design and operator risk warning.

[0090] In some possible implementations, the plurality of preset operation status labels include an error-free execution status label, an omission status label, an execution deviation status label, a result deviation status label, and a timing deviation status label; The step of determining the evaluation result of the human-computer interface based on the probability distribution and interface indicators corresponding to each of the target operation paths includes: For each target operation path, if the probability of the target operation path under the error-free execution state label is less than 1, the combined probability of the execution error corresponding to the target operation path is determined based on the probability of the target operation path under the omission state label, the execution deviation state label, and the result deviation state label, respectively. Based on the probability of the target operation path under the time deviation status label, the corresponding execution error synthesis probability, and the interface index, the human factor reliability analysis result corresponding to the target operation path is determined. Based on the human factor reliability analysis results corresponding to each of the target operation paths, the evaluation results of the human-machine interface are determined.

[0091] In the above steps, for each target operation path, if the probability of the target operation path under the error-free execution status label is less than 1, the sum of the probabilities of the target operation path under the omission status label, the execution deviation status label, and the result deviation status label can be determined as the combined execution error probability corresponding to the target operation path.

[0092] For example, the probability of execution error synthesis corresponding to the target operation path can be determined by the following formula (14): (14) in, This represents the combined probability of execution errors corresponding to the target operation path. This indicates the probability that the target operation path is under the missing status label. This indicates the probability that the target operation path is under the execution deviation state label. This indicates the probability of the target operation path being under the result deviation status label.

[0093] Based on the probability of the target operation path under the timing deviation state label, the corresponding execution error synthesis probability, and the interface indicators, the human factor reliability analysis result corresponding to the target operation path is determined. Here, a higher probability indicates a greater risk. Based on the human factor reliability analysis results corresponding to each target operation path, the evaluation result of the human-machine interface is determined.

[0094] In this way, we can not only achieve quantitative integration and accurate identification of various human error risks at the operational path level, but also deeply integrate interface indicators that characterize interface complexity and cognitive load into the analysis process. This allows the evaluation results of single paths and the overall interface to be highly correlated with the characteristics of the interface itself, effectively avoiding the limitations of single-dimensional evaluation. It ensures the comprehensiveness, scientificity and accuracy of human-computer interface evaluation results, and can reflect the human reliability level of human-computer interfaces in a comprehensive and multi-dimensional way. It provides accurate and specific quantitative basis for the scientific evaluation and targeted optimization design of human-computer interfaces.

[0095] For a clearer illustration of the human-computer interface evaluation process, please refer to [link / reference]. Figure 4 This is a schematic diagram illustrating a human-computer interface evaluation process, as shown in an exemplary embodiment of this application. Figure 4As shown, the process involves obtaining human factor reliability analysis requirements for the human-machine interface, modeling interface elements, constructing a knowledge graph corresponding to the human-machine interface, acquiring operator behavior records when performing the target task, collecting and mapping operation behavior data, determining multiple target operation paths corresponding to the target task, and then performing quantitative analysis of interface elements. For each target operation path, based on the knowledge graph and the human-machine interface, interface indicators corresponding to the target operation path are determined. The interface indicators and the operator's operator type are then input into a pre-trained probability prediction model for discrimination, obtaining the probability distribution of the target operation path under multiple preset operation state labels. Finally, based on the probability distribution corresponding to each target operation path and the interface indicators, the evaluation result of the human-machine interface is determined to achieve risk identification. Specific steps are described in the aforementioned embodiments and will not be repeated here.

[0096] To better understand the process of human-machine interface (HMI) evaluation, the following description uses an example. In a digital nuclear power plant control room inspection scenario, to determine the operator's operational risks during parameter lookup and confirmation, the objective task is to find and confirm the main pump outlet pressure parameter. The operator can start from the current interface, switch to the target sub-interface, locate the main pump outlet pressure parameter in the target sub-interface, and ultimately complete the value reading and confirmation. A knowledge graph corresponding to the HMI can be constructed to obtain the operator's operational behavior records when performing the objective task. These operational behavior records are mapped to the knowledge graph to determine multiple target operational paths corresponding to the objective task from multiple operational paths.

[0097] For one of the target operation paths, based on the knowledge graph and the human-computer interface, the corresponding interface metrics are determined. Specifically, the visual density (VD) is 0.78, indicating a large number of visible interactive elements and relatively dense information presentation; the semantic interference density (SID) is 0.65, indicating the presence of multiple semantically similar parameters or controls on the interface, such as main pump pressure and auxiliary pump pressure, which can easily lead to semantic confusion; and the interaction span (IS) is 0.72, indicating that completing this operation path requires a large mouse movement distance or interface switching span. In this example, the operator is a student, and the model input can be x={z,c}={VD,SID,IS,c}={0.78,0.65,0.72,student}.

[0098] Input x into the pre-trained probability prediction model, and through the anomaly detection unit, obtain the probability of the target operation path under the error-free execution state label. Therefore, it can be determined that the probability of the target operation path belonging to an abnormal state is... The results indicate that the target operation path has a 45% probability of exhibiting non-normal operational behavior, suggesting that the path carries a certain potential risk of anomalies.

[0099] If the probability of the target operation path under the error-free execution state label is less than 1, an anomaly is determined. The timing deviation discrimination unit then obtains the probability that the target operation path belongs to the timing deviation state. The results indicate that, once an anomaly occurs, approximately 40% of the cases will manifest as a significant slowdown in the operation pace or a marked increase in search time, i.e., time deviation behavior.

[0100] The probability that the target operation path belongs to a timing deviation state is less than 1. The probability that the target operation path belongs to an omission state is obtained through the error type classification unit. The probability that the target operation path belongs to the execution deviation state. The probability that the target operation path belongs to the result deviation state. This indicates that, on this target operation path, if it is not a time deviation, the more likely cause is an execution-related error, i.e., the operator clicking or selecting the wrong target control.

[0101] The probability of the target operation path under the timing deviation state label is determined by the anomaly probability combination unit. The results indicate that this interface path has an 18% probability of significantly slowing down the operator's pace. The probability of the target operation path being under a missing status label can also be determined. The probability of the target operation path under the execution deviation state label. And the probability of the target operation path under the result deviation status label. .

[0102] Based on this, the probability of execution error synthesis corresponding to the target operation path can be determined. The result indicates that this interface path has a 27% risk in terms of operational correctness.

[0103] The combined risk of time deviation and execution error yields the human factor reliability analysis results for this target operation path. The time deviation risk is 18%, and the execution error risk is 27%, indicating that this target operation path is a high-risk path that is prone to both slowing down operations and inducing errors. From an interface design perspective, this risk characteristic is highly correlated with the structural characteristics of the human-computer interface. High semantic interference density leads to similar parameter semantics, increasing the possibility of execution-related errors; a large interaction span increases search and location costs, easily causing time deviations; and high visual density overall raises the probability of anomalies.

[0104] This example demonstrates that the embodiments of this disclosure can quantitatively analyze each target operation path corresponding to the target task based solely on interface indicators and operator type, without relying on subjective expert evaluation. It outputs whether the operation is prone to slowing down, whether it is prone to errors, and what types of errors are more likely to occur, thereby providing direct quantitative basis for high-risk interface path identification and interface reconstruction design.

[0105] The human-machine interface (HMI) evaluation method provided in this application constructs a knowledge graph corresponding to the HMI and maps the operator's operational behavior records when performing a target task to the knowledge graph. This effectively determines multiple target operation paths corresponding to the target task. By combining the knowledge graph and the HMI, the method determines the interface indicators corresponding to each target operation path, thereby quantifying the influencing factors of the interface. The interface indicators and the operator's operator type are input into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. This effectively determines the impact of interface factors on the probability of human-caused failure. Based on human-caused reliability analysis, the method completes the evaluation of the HMI, realizing real-time risk identification and trend prediction of the operation process. It can effectively identify high-risk operation paths and their corresponding interface-induced factors, achieving dynamic diagnosis and quantitative evaluation of operational behavior and key interaction nodes. This helps improve the interpretability and predictive ability of HMI-induced human-caused failures, and provides a scientific and reliable basis for the optimized design of the HMI and risk warning for the operator.

[0106] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0107] Corresponding to the embodiments of the aforementioned human-machine interface evaluation method, this application also provides embodiments of a human-machine interface evaluation device.

[0108] Please see Figure 5 This is a schematic diagram illustrating a human-machine interface evaluation device according to an exemplary embodiment of this application. Figure 5 As shown in the figure, the human-machine interface evaluation device 500 provided in this application embodiment includes: The requirement acquisition module 501 is used to acquire human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, and the target tasks indicate how to find target elements from the human-machine interface; The path mapping module 502 is used to construct a knowledge graph corresponding to the human-machine interface, obtain the operation behavior records when the operator performs the target task, map the operation behavior records to the knowledge graph, and determine multiple target operation paths corresponding to the target task from multiple operation paths; the operation path is a connecting line between two knowledge graph nodes, the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface, and the operation path is used to indicate two consecutive operation behaviors. The indicator determination module 503 is used to determine the interface indicator corresponding to each target operation path based on the knowledge graph and the human-computer interface. The interface indicator is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path. The probability prediction module 504 is used to input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. The result determination module 505 is used to determine the evaluation result of the human-machine interface based on the probability distribution and interface indicators corresponding to each of the target operation paths.

[0109] In some possible implementations, the interface metrics include visual density, semantic interference density, and interaction span; When the indicator determination module 503 determines the interface indicator corresponding to the target operation path based on the knowledge graph and the human-computer interface, it is specifically used for: Based on the knowledge graph and the human-computer interface, determine the pointer movement distance when executing the target operation path, the maximum movement distance in the human-computer interface, and the number of elements, the number of first element pairs, and the number of second element pairs included in the human-computer interface; the first element pair includes any two elements in the human-computer interface, and the second element pair includes the target element and other elements whose semantic similarity to the target element is greater than a preset threshold. The visual density is determined based on the number of target elements and the number of elements. The semantic interference density is determined based on the number of the first element pairs and the number of the second element pairs; The interaction span is determined based on the pointer movement distance and the maximum movement distance.

[0110] In some possible implementations, the plurality of preset operation status labels include error-free execution status labels, omission status labels, execution deviation status labels, result deviation status labels, and timing deviation status labels; the probability prediction model is a hierarchical discriminative structure, and the probability prediction model includes an anomaly discrimination unit, a timing deviation discrimination unit, an error type classification unit, and an anomaly probability combination unit; The probability prediction module 504 is specifically used for: The anomaly detection unit performs anomaly detection based on the interface indicators and the operator's operator type to obtain the probability of the target operation path under the error-free execution status label. The timing deviation discrimination unit, when the probability of the target operation path under the error-free execution state label is less than 1, performs timing deviation discrimination based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the timing deviation state. By using the error type classification unit, when the probability of the target operation path belonging to the timing deviation state is less than 1, the error type is classified based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the omission state, execution deviation state, and result deviation state, respectively. The anomaly probability combination unit determines the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively.

[0111] In some possible implementations, the probability prediction module 504, when determining the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label based on the probability of the target operation path under the error-free execution state label and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state respectively, is specifically used for: Based on the probability that the target operation path is under the error-free execution status label, the probability that the target operation path belongs to an abnormal state is determined; Based on the probability that the target operation path belongs to a timing deviation state, the probability that the target operation path does not belong to a timing deviation state is determined. Based on the probability that the target operation path belongs to an abnormal state and the probability that it belongs to a timing deviation state, the probability that the target operation path is under the timing deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an omission state, the probability that the target operation path is under the omission state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an execution deviation state, the probability that the target operation path is under the execution deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to a result deviation state, the probability that the target operation path is under the result deviation state label is determined.

[0112] In some possible implementations, the human-computer interface evaluation device 500 further includes a model training module 506, which is used to train the probability prediction model through the following steps: Obtain test operation behavior records when test operators execute test tasks, map the test operation behavior records to the knowledge graph, and determine multiple test operation paths corresponding to the test task from multiple operation paths; For each of the test operation paths, based on the knowledge graph and the human-computer interface, the test interface indicators corresponding to the test operation path are determined; Based on the test operation behavior record and the specified behavior requirements of the test task, a target operation status label that matches the test operation path is determined from multiple preset operation status labels. The test interface indicators and the operator type of the test operator are input into the neural network to obtain the test probability distribution of the test operation path under multiple preset operation state labels. Based on the target operation state label corresponding to the test operation path and the test probability distribution, the parameters of the neural network are adjusted until the training cutoff condition is met, and a trained probability prediction model is obtained.

[0113] In some possible implementations, the probability prediction model is a hierarchical discriminative structure, which includes an anomaly detection unit, a time series deviation detection unit, an error type classification unit, and an anomaly probability combination unit. The model training module 506 is used to determine whether the neural network meets the cutoff condition through the following steps: The neural network meets the cutoff condition when the negative log-likelihood function of the corresponding layer converges, the prediction probability of the neural network for the target operation state label is stable, the output probabilities of the anomaly discrimination unit and the timing deviation discrimination unit meet the consistency condition, and the value of the negative log-likelihood function of the layer does not decrease within a preset time window; the negative log-likelihood function of the layer is determined based on the test probability of the test operation path under the target operation state label.

[0114] In some possible implementations, the plurality of preset operation status labels include an error-free execution status label, an omission status label, an execution deviation status label, a result deviation status label, and a timing deviation status label; The result determination module 505 is specifically used for: For each target operation path, if the probability of the target operation path under the error-free execution state label is less than 1, the combined probability of the execution error corresponding to the target operation path is determined based on the probability of the target operation path under the omission state label, the execution deviation state label, and the result deviation state label, respectively. Based on the probability of the target operation path under the time deviation status label, the corresponding execution error synthesis probability, and the interface index, the human factor reliability analysis result corresponding to the target operation path is determined. Based on the human factor reliability analysis results corresponding to each of the target operation paths, the evaluation results of the human-machine interface are determined.

[0115] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0116] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0117] Based on the same technical concept, this application also provides a computer device 600, referring to... Figure 6 The diagram shown is a schematic representation of the structure of a computer device according to an exemplary embodiment of this application, comprising: The processor 610, memory 620, and bus 630 are included. The memory 620 is used to store execution instructions and includes main memory 621 and external memory 622. The main memory 621, also known as internal memory, is used to temporarily store the operation data in the processor 610 and the data exchanged with external memory 622 such as hard disk. The processor 610 exchanges data with external memory 622 through main memory 621.

[0118] In this embodiment, the memory 620 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 610. That is, when the electronic device 600 is running, the processor 610 communicates with the memory 620 through the bus 630, or the processor 610 communicates with the memory 620 through other means, so that the processor 610 executes the application code stored in the memory 620, and then executes the steps of the human-machine interface evaluation method described in any of the foregoing embodiments.

[0119] The memory 620 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0120] Processor 610 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0121] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 600. In other embodiments of this application, the electronic device 600 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0122] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the human-machine interface evaluation method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0123] This disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the human-computer interface evaluation method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0124] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0125] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0126] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0127] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0128] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0129] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0130] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0131] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0132] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A human-computer interface evaluation method, characterized in that, The method includes: Obtain human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, the target tasks indicating the search for target elements from the human-machine interface; A knowledge graph corresponding to the human-machine interface is constructed, and operation behavior records of the operator when performing the target task are obtained. The operation behavior records are mapped to the knowledge graph, and multiple target operation paths corresponding to the target task are determined from multiple operation paths. The operation path is a connecting line between two knowledge graph nodes, and the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface. The operation path is used to indicate two consecutive operation behaviors. For each target operation path, based on the knowledge graph and the human-computer interface, an interface indicator corresponding to the target operation path is determined. The interface indicator is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path. Input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. The evaluation result of the human-machine interface is determined based on the probability distribution and interface indicators corresponding to each of the target operation paths.

2. The method according to claim 1, characterized in that, The interface metrics include visual density, semantic interference density, and interaction span. The step of determining the interface metrics corresponding to the target operation path based on the knowledge graph and the human-computer interface includes: Based on the knowledge graph and the human-computer interface, determine the pointer movement distance when executing the target operation path, the maximum movement distance in the human-computer interface, and the number of elements, the number of first element pairs, and the number of second element pairs included in the human-computer interface; the first element pair includes any two elements in the human-computer interface, and the second element pair includes the target element and other elements whose semantic similarity to the target element is greater than a preset threshold. The visual density is determined based on the number of target elements and the number of elements. The semantic interference density is determined based on the number of the first element pairs and the number of the second element pairs; The interaction span is determined based on the pointer movement distance and the maximum movement distance.

3. The method according to claim 1, characterized in that, The multiple preset operation status labels include error-free execution status labels, omission status labels, execution deviation status labels, result deviation status labels, and timing deviation status labels; the probability prediction model is a hierarchical discriminative structure, and the probability prediction model includes an anomaly discrimination unit, a timing deviation discrimination unit, an error type classification unit, and an anomaly probability combination unit; The step of inputting the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels includes: The anomaly detection unit performs anomaly detection based on the interface indicators and the operator's operator type to obtain the probability of the target operation path under the error-free execution status label. The timing deviation discrimination unit, when the probability of the target operation path under the error-free execution state label is less than 1, performs timing deviation discrimination based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the timing deviation state. By using the error type classification unit, when the probability of the target operation path belonging to the timing deviation state is less than 1, the error type is classified based on the interface indicators and the operator's operator type to obtain the probability that the target operation path belongs to the omission state, execution deviation state, and result deviation state, respectively. The anomaly probability combination unit determines the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, respectively.

4. The method according to claim 3, characterized in that, The step of determining the probability of the target operation path under the error-free execution state label, and the probability of the target operation path belonging to the timing deviation state, omission state, execution deviation state, and result deviation state, respectively, based on the probability of the target operation path under the omission state label, execution deviation state label, result deviation state label, and timing deviation state label, includes: Based on the probability that the target operation path is under the error-free execution status label, the probability that the target operation path belongs to an abnormal state is determined; Based on the probability that the target operation path belongs to a timing deviation state, the probability that the target operation path does not belong to a timing deviation state is determined. Based on the probability that the target operation path belongs to an abnormal state and the probability that it belongs to a timing deviation state, the probability that the target operation path is under the timing deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an omission state, the probability that the target operation path is under the omission state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to an execution deviation state, the probability that the target operation path is under the execution deviation state label is determined. Based on the probability that the target operation path belongs to an abnormal state, the probability that it does not belong to a timing deviation state, and the probability that it belongs to a result deviation state, the probability that the target operation path is under the result deviation state label is determined.

5. The method according to claim 1, characterized in that, The probability prediction model is trained through the following steps: Obtain test operation behavior records when test operators execute test tasks, map the test operation behavior records to the knowledge graph, and determine multiple test operation paths corresponding to the test task from multiple operation paths; For each of the test operation paths, based on the knowledge graph and the human-computer interface, the test interface indicators corresponding to the test operation path are determined; Based on the test operation behavior record and the specified behavior requirements of the test task, a target operation status label that matches the test operation path is determined from multiple preset operation status labels. The test interface indicators and the operator type of the test operator are input into the neural network to obtain the test probability distribution of the test operation path under multiple preset operation state labels. Based on the target operation state label corresponding to the test operation path and the test probability distribution, the parameters of the neural network are adjusted until the training cutoff condition is met, and a trained probability prediction model is obtained.

6. The method according to claim 5, characterized in that, The probability prediction model has a hierarchical discriminative structure, which includes an anomaly detection unit, a time series deviation detection unit, an error type classification unit, and an anomaly probability combination unit. The following steps are used to determine if a neural network meets the cutoff condition: The neural network meets the cutoff condition when the negative log-likelihood function of the corresponding layer converges, the prediction probability of the neural network for the target operation state label is stable, the output probabilities of the anomaly discrimination unit and the timing deviation discrimination unit meet the consistency condition, and the value of the negative log-likelihood function of the layer does not decrease within the preset time window. The hierarchical negative log-likelihood function is determined based on the test probability of the test operation path under the target operation state label.

7. The method according to claim 1, characterized in that, The multiple preset operation status labels include error-free execution status label, omission status label, execution deviation status label, result deviation status label and timing deviation status label; The step of determining the evaluation result of the human-computer interface based on the probability distribution and interface indicators corresponding to each of the target operation paths includes: For each target operation path, if the probability of the target operation path under the error-free execution state label is less than 1, the combined probability of the execution error corresponding to the target operation path is determined based on the probability of the target operation path under the omission state label, the execution deviation state label, and the result deviation state label, respectively. Based on the probability of the target operation path under the time deviation status label, the corresponding execution error synthesis probability, and the interface index, the human factor reliability analysis result corresponding to the target operation path is determined. Based on the human factor reliability analysis results corresponding to each of the target operation paths, the evaluation results of the human-machine interface are determined.

8. A human-machine interface evaluation device, characterized in that, The device includes: The requirement acquisition module is used to acquire human factors reliability analysis requirements for the human-machine interface; the human factors reliability analysis requirements include target tasks for evaluating the human-machine interface, and the target tasks indicate how to find target elements from the human-machine interface; The path mapping module is used to construct a knowledge graph corresponding to the human-machine interface, obtain the operation behavior records of the operator when performing the target task, map the operation behavior records to the knowledge graph, and determine multiple target operation paths corresponding to the target task from multiple operation paths; the operation path is a connecting line between two knowledge graph nodes, the knowledge graph nodes correspond one-to-one with the elements in the human-machine interface, and the operation path is used to indicate two consecutive operation behaviors. The indicator determination module is used to determine the interface indicator corresponding to each target operation path based on the knowledge graph and the human-computer interface. The interface indicator is used to characterize the interface complexity and potential cognitive load sources when executing the target operation path. The probability prediction module is used to input the interface indicators and the operator's operator type into a pre-trained probability prediction model to obtain the probability distribution of the target operation path under multiple preset operation state labels. The result determination module is used to determine the evaluation result of the human-machine interface based on the probability distribution and interface indicators corresponding to each of the target operation paths.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the human-machine interface evaluation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the human-machine interface evaluation method according to any one of claims 1 to 7.

Images