Human-machine interface evaluation method combining hand action capture and motive analysis

Abstract

The application discloses a man-machine interaction interface evaluation method combining hand action capture and motive analysis, relates to the technical field of man-machine interaction, can acquire usable behavior information, and combines the motive analysis method to classify interaction behavior and realize evaluation on the man-machine interaction interface. First, the finger joint bending degree data of each moment during man-machine interaction is collected, and the user interaction operation video is recorded. The finger joint bending degree data of the user is normalized. The evaluation task time is recorded, and the finger joint bending degree data of each moment is converted into the cumulative finger joint activity amount within the task time. The finger joint actually used by the subject in the task is selected. The activity amount of the selected finger joint is summed to obtain the finger joint activity amount at each moment within the task time. The finger joint activity amount is classified by using the motive analysis method, the operation time and the activity amount of each type of motive behavior are counted, and the man-machine interaction interface is evaluated.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and more specifically to a human-computer interaction interface evaluation method that combines hand motion capture and motion element analysis. Background Technology

[0002] With the rapid popularization of handheld mobile devices (including smartphones, tablets, etc.) and the massive growth of users, mobile applications (Web Applications, APPs) developed based on web browsers are emerging in large numbers, making the design of the application's interactive interface particularly important.

[0003] The interface design of web applications should focus on both usability (efficiency) and sensory interaction (user experience) to improve the comfort of human-computer interaction. Complex hierarchical relationships, multi-dimensional data, and numerous operation paths in web applications can lead to user distraction, anxiety, and low satisfaction after prolonged use, resulting in a poor user experience and reduced user engagement. With increasing attention to cognitive psychology and engineering psychology in the field of human-computer interaction, application interfaces, while meeting functional and efficiency requirements, need to simplify user operations, reduce cognitive load and learning costs, and improve users' physiological and psychological satisfaction when using the information interface. Therefore, effectively combining various measurement methods to develop scientific and effective interaction evaluation methods and form comprehensive interaction evaluation indicators has become a key research focus. Existing interaction design evaluation methods mainly include self-report methods and physiological measurement methods.

[0004] Self-report methods, typically using interviews, questionnaires, and scales, assume users report their cognitive processes through introspection after interacting with the application, thus obtaining subjective evaluation information. Research on subjective functional evaluation of application interfaces using self-report methods has led to the development of many mature evaluation indicators and scales: For overall usability assessment, there are Nielsen's Ten Usability Principles, the SUS System Usability Scale, the SUMI Software Usability Questionnaire, and the CSUQ System Usability Scale questionnaire; for measuring user emotional data, there are standardized scales such as the PAD scale, the PrEmo scale, and utility and hedonism scales; for cognitive load (CL), it measures the individual psychological resources users use to solve problems or complete tasks within a specific timeframe. Self-report methods can conveniently collect large amounts of data in a short time, making them the most efficient existing methods for measuring application interface experience. However, they are highly subjective, and the results are not always reliable; therefore, they need to be combined with objective measurement methods.

[0005] Physiological measurement methods compensate for the shortcomings of self-report evaluation methods by directly measuring physiological and behavioral indicators reflecting user experience during application interface interactions, thereby acquiring implicit emotional data from users. Relevant physiological measurement research includes using physiological indicators such as heart rate, electromyography (EMG), and electrodermal signaling (EDS) that fluctuate with emotional arousal for evaluation. Mandryk et al. selected EDS, ECG, EMG, respiratory rate, and respiratory amplitude as indicators for evaluating user experience, demonstrating through experiments that changes in the mean of these indicators can reflect the level of user experience. Physiological and behavioral information can be continuously recorded using relevant instruments, offering advantages such as high data accuracy, objectivity, and immediacy. However, physiological measurement methods are based on the assumption that human physiological changes reflect an individual's psychological state; changes in physiological data are related to the level of stimulation experienced by the individual and represent different levels of psychological processing. The challenge lies in selecting appropriate methods for user data collection, processing and analyzing the data, and identifying the correlation between data indicators and application interface evaluations.

[0006] User behavior and cognition during human-computer interaction are complex and dynamic, making a single evaluation method insufficient to accurately reflect the user's true experience. To avoid the influence of individual participant characteristics on experimental results, it is necessary to combine the advantages of various measurement methods, select appropriate evaluation indicators from numerous assessment methods, evaluate the application interface, and propose suggestions for improving the application interface interaction. Reducing user cognitive load and optimizing human-computer interaction have become key research issues.

[0007] Chinese patent specification CN202211263019.6 discloses a method and system for evaluating multi-channel HMI interfaces. This method collects image data when a user performs test tasks on different functional modules of the Human Machine Interface (HMI). Behavioral and emotional data are extracted from the image data. Evaluation results for different functional modules on the HMI are calculated according to a preset HMI interface evaluation system based on the gaze data / behavioral data / emotional data and their scores. However, because this method only focuses on gaze and facial expression data within the user's head area and lacks analysis of hand movements in key areas during human-computer interaction, the final evaluation results may contain errors.

[0008] Chinese patent specification CN202210410184.3 discloses an evaluation method for intelligent cockpit HMI based on eye-tracking control. This method analyzes user gaze behavior by collecting head position data and user eye movement data. However, this method only analyzes eye-tracking control interface behavior and lacks more possible gesture operation scenarios.

[0009] Users primarily interact with application interfaces through hand gestures. Existing research focuses solely on hand motion capture technology for gesture and animation development and design, with limited use of hand motion capture technology for interface evaluation. Summary of the Invention

[0010] In view of this, the present invention provides a human-computer interaction interface evaluation method that combines hand motion capture and motion element analysis. It can collect hand interaction motion information, effectively process the data, obtain usable behavioral information, and classify the interaction behavior by combining motion element analysis, analyze the motion elements that can be improved, thereby improving the interaction process and optimizing the layout of the interaction interface.

[0011] To achieve the above objectives, the technical solution of the present invention includes the following steps:

[0012] Collect data on the bending degree of the user's finger joints at every moment during human-computer interaction, and record video of the user's interaction.

[0013] Normalize the user's finger joint flexion data.

[0014] Record the evaluation task time and convert the individual finger joint flexion data at each moment into the cumulative finger joint activity over the task time.

[0015] Select the finger joints that the participants actually used during the task.

[0016] The summation of the selected finger joint movements yields the amount of finger joint movement at each moment within the task time.

[0017] Using the motion element analysis method, the amount of finger joint activity is classified, the operation time and activity amount of each type of motion element behavior are statistically analyzed, and the human-computer interaction interface is evaluated.

[0018] Furthermore, data on the bending degree of the user's finger joints is collected during human-computer interaction. Specifically, during human-computer interaction, the user wears a 5DT Data Glove 14Ultra data glove to collect data on the bending degree of the user's finger joints.

[0019] Furthermore, the user's finger joint curvature data is normalized, specifically as follows:

[0020] A user's hand contains 14 finger joints, which are numbered as i, where i = {1, 2, 3, ..., 14}. The user's finger joint flexion data includes: the minimum flexion value of each finger joint is raw. min The maximum bending value is raw max Real-time bending value is raw val ;

[0021] Real-time bending values ​​rawval A linear mapping is used to obtain values ​​in the interval [0, 1] to achieve normalization;

[0022] The real-time normalized curvature value of the finger joints is x. i ,but:

[0023]

[0024] Furthermore, the evaluation task time is recorded, and the individual finger joint flexion data at each moment is converted into the cumulative finger joint activity over the task time. This includes the following steps:

[0025] t k Represents a time record node, t k The value of finger joint flexion at time t is x(t) k ), t k The finger joint flexion value at time +Δt is x(t) k+ Δt), t k Time to t k The amount of finger joint movement at time +Δt is Y(t) k By recording the time range of each finger joint operation via video, the activity level of a single finger joint during the task time is:

[0026]

[0027] Where m represents the total number of time points within the time range of each finger joint operation.

[0028] Furthermore, selecting the finger joints actually used by the participants in the task also includes: filtering out the finger joints actually used in the task to form dataset S1, i.e.:

[0029] S1={ x j | j=1, 2, 3, …, n} (3)

[0030] Where n is the number of finger joints actually used in the task, n≤14; in dataset S1: x j Joint curvature data that changes over time characterizes the real-time activity of the fingers.

[0031] Furthermore, the range of motion of the selected finger joints is summed to obtain the total range of motion of the user's finger joints in the task, specifically:

[0032] Based on the selected finger joints, if the total range of motion of the user's finger joints in the hand task is Y, then:

[0033]

[0034] The sum of equation (4) represents the change in finger activity, indicating the degree of activity of a specified finger during the task. Further, using kinematic analysis, the finger joint activity is categorized and encoded as kinematic elements of user interaction behavior, specifically:

[0035] Based on finger movements, three types of motion elements are identified as follows:

[0036] ① Effective motivator: Participants clicked on options according to the task sheet;

[0037] ② Assistive actions: Assistive actions performed by the participants before searching for options, including clicking the menu bar, swiping the screen, and clicking the next guide button;

[0038] ③ Invalid actions: These include the actions of the participants after clicking to complete the task configuration in each subtask, and the actions of the user before clicking the incomplete option in each subtask.

[0039] Based on the above classification, task time is divided into: effective task time, auxiliary task time, and ineffective task time. The total task time is then the set of effective time, ineffective time, and auxiliary time.

[0040] Based on the time of the finger joint movement, the finger joint movement is divided into: task-effective movement, task-assisted movement, and task-ineffective movement; thus, the finger joint movement is classified.

[0041] Furthermore, the operation time and activity level of various action elements are statistically analyzed to evaluate the human-computer interaction interface. Specifically, the operation time and activity level of the task-assisted activity level corresponding to the auxiliary action element are used as the measurement indicators for evaluating the human-computer interaction interface design.

[0042] Beneficial effects:

[0043] This invention provides a human-computer interaction interface evaluation method combining hand motion capture and motion element analysis. It optimizes human-computer interaction gestures by incorporating hand motion capture information: data gloves can provide posture tracking information for user gestures, enabling high-precision gesture acquisition. Simultaneously, it improves the interaction process through auxiliary motion elements: motion element analysis can provide guidance for reducing operator cognitive load and fatigue. This invention combines motion element analysis to classify human-computer interaction operation time and finger activity in detail. Experimental verification shows that the differences in auxiliary motion elements at different task stages exhibit typical characteristics, with auxiliary time and auxiliary activity levels serving as important data variables characterizing motion element status. Therefore, reducing cognitive load requires focusing on the specific interaction details of auxiliary motion elements, while optimizing operation gestures, improving auxiliary activity levels, and reducing unnecessary hand operations that consume cognitive load. Attached Figure Description

[0044] Figure 1 A flowchart illustrating the evaluation process for a human-computer interaction interface evaluation method that combines hand motion capture and motion element analysis;

[0045] Figure 2 A diagram showing the correspondence between finger joints and motion capture data record numbers;

[0046] Figure 3 This is a classification diagram of gesture operations for motion element analysis. Detailed Implementation

[0047] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0048] This invention provides a human-computer interaction interface evaluation method combining hand motion capture and motion element analysis, recording real-time data of the user's human-computer interaction process through hand motion capture technology. Hand motion capture technology is a technique used to accurately measure the movement of finger joints in three-dimensional space. Based on computer graphics, it records the movement of the capture device (mainly sensors) through images or other forms. It records the spatial coordinates and bending angles of the finger joints at different times to obtain the object's motion behavior. The motion capture system mainly consists of a transmission unit, sensors, and a data processing unit. Typically, sensors acquire relevant motion information at a target location by being mounted on a target surface, thereby tracking the object. The data transmission unit is responsible for transmitting the information collected by the sensors to the data processing unit via wired, Bluetooth, or WiFi methods, where it is further analyzed and processed. Finger movement has a certain range of motion; the extreme positions of the fingers form the maximum and minimum bending points of the sensor, as shown in Table 1. By capturing the bending angles of the finger joints and processing them into activity data, hand operation behavior can be effectively recorded.

[0049] The 5DT Data Glove Ultra program generates real-time images to display the bending angles of each finger joint (14 capture points in total) to reflect the bending of the selected nodes over the entire time period.

[0050] Table 1 Degrees of freedom and angular range of local bone rotation

[0051]

[0052]

[0053] The overall evaluation process is as follows: Figure 1 As shown, the specific implementation method is as follows:

[0054] Step 1: Collect data on the bending degree of the user's finger joints at every moment during human-computer interaction, and record the user's interaction operation video.

[0055] The user wears a 5DT Data Glove 14 Ultra data glove on their right hand, with their right elbow resting on the perforated area of ​​the foam pad on the lab bench. They adjust their right arm posture and tablet position to ensure comfortable operation. Once the task begins, the elbow and iPad screen positions remain fixed. The program records the user's finger joint flexion data.

[0056] Step 2: Normalize the finger joint flexion data.

[0057] The 14 finger joints of the right hand are numbered i, where i = {1, 2, 3, ..., 14}. The correspondence between finger joints and their numbers is as follows: Figure 2 As shown. The minimum bending value of each finger joint is raw. min The maximum bending value is raw max Real-time bending value is raw val The real-time bending value will be raw. val A linear mapping is used to obtain values ​​within the interval [0, 1] to achieve normalization. Let x be the real-time normalized bending value of the right finger joint. i ,but:

[0058]

[0059] Step 3: Record the evaluation task time and convert the individual finger joint flexion data at each moment into the cumulative finger joint activity over the task time.

[0060] t k Represents a time record node, t k The value of finger joint flexion at time t is x(t) k ), t k The finger joint flexion value at time +Δt is x(t) k+ Δt), t k Time to t k The amount of finger joint movement at time +Δt is Y(t) k By recording the time range of each finger joint operation via video, the activity level of a single finger joint during the task time is:

[0061]

[0062] Step 4: Select the finger joints that the subject actually used in the task.

[0063] Based on the experimental video recordings, the finger joints actually used by each participant in the task were selected to form dataset S1, namely:

[0064] S1={ x j | j=1, 2, 3, …, n} (n≤14) (3)

[0065] In dataset S1: x j The joint curvature data that changes over time characterizes the real-time activity of the fingers, providing an intuitive visual reference for subsequent analysis.

[0066] Step 5: Sum the range of motion of the selected finger joints to obtain the range of motion of the finger joints at each moment during the task time.

[0067] Based on the finger joints selected in equation (3), the total amount of finger joint movement of the subject during the task is Y.

[0068]

[0069] The sum of equation (4) represents the change in finger activity, indicating the degree of activity of a specified finger in a task.

[0070] Step 6: Use the motion element analysis method to classify the amount of finger joint activity, count the operation time and activity amount of each type of motion element behavior, and evaluate the human-computer interaction interface.

[0071] Motion element analysis refers to a detailed analysis of the sequence of actions and the correlation of activity in different parts of the body to identify and improve problems in the actions. In human-computer interaction, there are necessary clicking actions, thinking processes, and rest and delays. This method can effectively reflect the objective weight of different operations, analyze the cognitive load corresponding to different hand operations, and identify three types of motion elements in the participants' operation process in this experiment as follows: Figure 3 The diagram shows: ① Effective motifs: Actions performed by participants clicking on options according to the task description. ② Auxiliary motifs: Actions performed by participants before searching for options, such as clicking the menu bar, swiping the screen, and clicking the "next" guide button. ③ Ineffective motifs: Actions performed by participants after completing the task configuration in each subtask, as well as actions such as incorrect selections before clicking on incomplete options in each subtask. This facilitates the coding of user behaviors in the task and allows for the provision of improvement suggestions.

[0072] Based on the above classification, task time was divided into: effective task time, auxiliary task time, and ineffective task time; finger joint activity was divided into: effective task activity, auxiliary task activity, and ineffective task activity. The number of errors was counted and the three types of kinematic elements were screened based on the video recordings of the subjects' operation process, and the activity level was calculated using data recorded by the data gloves.

[0073] definition:

[0074] t u For the effective time intervals to be selected, u = 1, 2, 3, ..., n1, Yt u This refers to the effective activity level at the corresponding time.

[0075] tv For the invalid time intervals to be filtered, v = 1, 2, 3, ..., n2, Yt v This refers to the effective activity level at the corresponding time.

[0076] t w For auxiliary time intervals used in the selection, w = 1, 2, 3, ..., n3, Yt w This refers to the effective activity level at the corresponding time.

[0077] The total task time is then the set of effective time, ineffective time, and auxiliary time, i.e.:

[0078] t = t u1 + t v1 + t w1 + t u2 + t v2 + t w2+ t u3 + t v3 + t w3 +…+ t un + t vn + t wn (5)

[0079] The total amount of finger joint movement during the task is the set of effective time activity, ineffective time activity, and auxiliary time activity, that is:

[0080] Y = Yt u1 +Yt v1 +Yt w1 +Yt u2 +Yt v2 +Yt w2+ Yt u3 +Yt v3 +Yt w3 +…+Yt un +Yt vn +

[0081] Yt wn (6)

[0082] in accordance with Figure 3 The classification method encodes user interaction behaviors into action elements and statistically analyzes the operation time and activity level of each type of action element:

[0083] Effective time is t u :

[0084] t u =t u1 + t u2 +t u3 +…+ t un (7)

[0085] Invalid time is t v :

[0086] t v =t v1 + t v2 +t v3 +…+ t vn (8)

[0087] The auxiliary time is t w :

[0088] t w =t w1 +t w2 +t w3 +…+t wn (9)

[0089] Effective activity level is Yt u :

[0090] Yt u =Yt u1 +Yt u2 +Yt u3 +…+Yt un (10)

[0091] Ineffective activity level is Yt v :

[0092] Yt v =Yt v1 +Yt v2 +Yt v3 +…+Yt vn (11)

[0093] The amount of auxiliary activity is Yt w :

[0094] Yt w =Yt w1 +Yt w2 +Yt w3 +…+Yt wn (12)

[0095] Motion element analysis provides guidance for efficient work, reduced cognitive load, and fatigue reduction in the field of industrial engineering. This invention introduces motion element analysis to classify APP operation time and finger activity in detail. Experiments show that the differences in assistive motion elements at different task stages have typical characteristics, among which assistive time and assistive activity can serve as important data variables characterizing motion element status. Therefore, in this embodiment of the invention, the operation time and activity amount of the task-related assistive activity corresponding to the assistive motion element are used as measurement indicators for evaluating human-computer interaction interface design. Reducing cognitive load requires focusing on the specific interaction details of assistive motion elements, while optimizing operation gestures, improving assistive activity, and reducing unnecessary hand operations that consume cognitive load.

[0096] In the touchscreen era, gestures have gradually become the primary human-computer interaction channel. Data gloves can provide posture tracking information of user gestures, enabling high-precision gesture acquisition. In this invention, interactive gesture analysis was constructed under different task scenarios. Experimental results show that when users browse 3D products on an app, dragging to view them has a lower cognitive load and takes less time compared to finger swiping. This is because when users browse products using the application interface, the continuous dynamic product image helps them build a clear overall understanding of the product, and continuous dragging interaction satisfies the above cognitive needs. Therefore, dragging gestures should be considered as an interactive form to help users build a complete and continuous product image. During the completion of sub-tasks, participants used different fingers to complete click and swipe operations. Experimental results show that the thumb operation had the least activity. This is because the thumb, as the finger closest to the body, is easier to mobilize, and its smaller joint bending allows control of a larger range of interface interactions. Therefore, in app interaction design, the thumb swiping hotspot should be selected as much as possible for high-frequency, simple task selection layouts.

[0097] The essence of human-computer interaction (HCI) digital interfaces is to transform abstract and complex information in a system into visual elements that are easy for users to search, recognize, and understand. Currently, graphical user interfaces (GUIs) composed of elements such as icons, symbols, text, and colors are the most widely used. As the types and quantities of information continue to increase, it is necessary to reorganize and optimize the information architecture and navigation layout of the interface to improve the efficiency of user interaction with the system. Experimental results from embodiments of this invention show that overlapping function button interaction areas and excessively large interaction button areas in mobile application interfaces both increase the rate of operational errors, and long text descriptions increase thinking time and cognitive load.

[0098] In the experiments of this invention, by constructing interactive processes under different tasks and analyzing hand motion data, it was found that dragging to view and browse 3D products has a lower cognitive load and takes less time compared to finger swiping. Compared to other finger swiping operations, thumb operation involves the least amount of activity. When designing human-computer interaction gesture operations, the thumb swiping hotspot should be selected as much as possible for high-frequency and simple task selection layouts. Using hand motion capture technology can minimize the impact of physiological measurement evaluation methods on user interaction behavior, effectively reduce the burden of evaluation instruments on the user interaction process, and allow users to operate in a relatively realistic and natural scenario. At the same time, based on embodied cognition theory, hand movements and brain thinking are synchronous in the cognitive process of human-computer interaction interfaces, and they also influence and relate to each other. Introducing hand motion capture as evaluation data can explore the coupling mechanism between hand operation load and brain cognitive load, helping users better understand the user's thinking and interactive behavior decision-making process, and thus evaluate the human-computer interaction interface.

[0099] Based on this invention, the design principles for human-computer interaction interfaces are as follows:

[0100] ① When users browse and identify the product as a whole during human-computer interaction, drag gestures should be considered to construct a complete and continuous product image.

[0101] ② When setting up swipe gestures, try to select the thumb swipe hotspot for the layout of operation tasks.

[0102] ③ Assistive factors (mainly including assistive time and assistive activity amount) show significant differences in different tasks, and can be used as important data variables to characterize the status of assistive factors, and as a measurement index for evaluating human-computer interaction interface design.

[0103] ④ In order to reduce the error rate in the human-computer interaction process, the design should consider expanding the operation range of the interactive buttons to improve convenience. It is also necessary to distinguish the styles of the operable areas of the control information interface, and to segment the text according to semantics to reduce the difficulty of recognizing the text description options.

[0104] ⑤ Using a single navigation path to build an information architecture will result in shorter operation time, less finger movement, and lower cognitive load.

[0105] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A human-computer interaction interface evaluation method combining hand motion capture and motion element analysis, characterized in that, Includes the following steps: Collect data on the bending degree of the user's finger joints at every moment during human-computer interaction, and record video of the user's interaction operation; The user's finger joint curvature data is normalized, specifically as follows: A user's hand contains 14 finger joints, which are numbered as follows: i , i ={1, 2, 3, …, 14}; The user's finger joint curvature data includes: the minimum curvature value of each finger joint. raw min The maximum bending value is raw max Real-time bending value raw val ; The real-time bending value raw val A linear mapping is used to obtain values ​​in the interval [0, 1] to achieve normalization; The real-time normalized curvature value of the finger joints is: x i ,but: （1）； Record the evaluation task time, and convert the individual finger joint flexion data at each moment into the cumulative finger joint activity over the task time. The specific steps include the following: t k Indicates the time record node, t k The finger joint flexion value at time ____ x ( t k ), t k + Δ t The finger joint flexion value at time ____ x ( t k+ Δt ), t k Time's up t k + Δ t The range of motion of the finger joints at any given time is Y ( t k By recording the time range of each finger joint operation via video, the activity level of a single finger joint during the task time is: Y j = Y( t k ) = | x ( t k +Δ t ) - x ( t k ) |,k=1, 2, 3, …, m.(2) Where m represents the total number of time nodes within the time range of each finger joint operation; Select the finger joints that the participants actually used during the task; The summation of the selected finger joint movements yields the finger joint movement at each moment during the task time. Using kinematic analysis, finger joint activity was categorized, and the operation time and activity level of each kinematic behavior were statistically analyzed to evaluate the human-computer interaction interface. Specifically: Based on finger movements, three types of motion elements are identified as follows: ① Effective motivator: Participants clicked on options according to the task sheet; ② Assistive actions: Assistive actions performed by the participants before searching for options, including clicking the menu bar, swiping the screen, and clicking the next guide button; ③ Invalid actions: These include the actions of the participants after clicking to complete the task configuration in each subtask, and the actions of the user before clicking the incomplete option in each subtask. Based on the above classification, task time is divided into: effective task time, auxiliary task time, and ineffective task time. The total task time is then the set of effective time, ineffective time, and auxiliary time. Based on the time of the finger joint movement, the finger joint movement is divided into: task-effective movement, task-assisted movement, and task-ineffective movement; thus, the finger joint movement is classified.

2. The human-computer interaction interface evaluation method combining hand motion capture and motion element analysis as described in claim 1, characterized in that, The collection of user finger joint flexion data during human-computer interaction specifically includes: During human-computer interaction, users wear 5DT Data Glove 14 Ultra data gloves to collect data on the bending degree of their finger joints.

3. The human-computer interaction interface evaluation method combining hand motion capture and motion element analysis as described in claim 1 or 2, characterized in that, The selection of finger joints actually used by the subjects in the task also includes: filtering out the finger joints actually used in the task to form a dataset. S 1, that is: S 1={ x j | j =1, 2, 3, …, n}（3） Where n is the number of finger joints actually used in the task, n≤14; Dataset S 1 in: x j Joint curvature data that changes over time characterizes the real-time activity of the fingers.

4. The human-computer interaction interface evaluation method combining hand motion capture and motion element analysis as described in claim 3, characterized in that, The summation of the selected finger joint movements to obtain the total user finger joint movements during the task is as follows: Based on the selected finger joints, the total amount of finger joint movement during the hand task is: Y ,but: Y= Y j （4） The sum of equation (4) represents the change in finger activity, indicating the degree of activity of a specified finger in a task.

5. The human-computer interaction interface evaluation method combining hand motion capture and motion element analysis as described in claim 4, characterized in that, The method of statistically analyzing the operation time and activity level of various dynamic element behaviors to evaluate the human-computer interaction interface specifically involves using the operation time and activity level of the task-assisted activity level corresponding to the auxiliary dynamic element as a measurement index for evaluating the human-computer interaction interface design.

Images