Eye movement behavior classification method based on Gaussian mixture model and hidden Markov model
Through the combination of Gaussian hybrid model and hidden Markov model, the existing ocular movement behavior classification method is solved inadequate robustness and adaptability in uncontrolled environments, and high accuracy and stability ocular movement behavior are achieved Classified, suitable for commercial-grade desktop eye trackers.
Patent Information
- Application Number
- CN202411023991.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-29
AI Technical Summary
The existing classification methods for eye movement behavior lack robustness and adaptability in uncontrolled environments, and there are classification inconsistencies and errors. In particular, the classification performance of ternary eye movement behavior is low, making it difficult to effectively process diversified eye movement data.
Using the combination of Gaussian hybrid model and hidden Markov model, the eye movement data is efficiently and accurately classified through hierarchical training and clustering algorithms, and the multimodal features of the data are modeled using Gaussian hybrid model. The model captures the timing relationship of eye movement patterns and realizes the accurate classification of gaze, tracking and eye saccade behavior.
It significantly improves the accuracy and stability of eye movement behavior classification, reduces the complexity of early data processing, adapts to the real-time needs of commercial-grade desktop eye movement instruments, and has a classification accuracy of 94.39%±2.08%, with excellent accuracy and recall. in traditional algorithms.
Smart Images

Figure CN119007275A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of human-computer interaction, and in particular relates to a high-accuracy eye movement behavior classification method. Background Art
[0002] Eye tracking technology is a technology that monitors and records individual eye movements by using dedicated equipment, such as an eye tracker. Eye tracking aims to accurately capture changes in the eye in space and time in order to analyze and understand the visual behavior of individuals in specific tasks or situations. As a cutting-edge human factors analysis technology, this technology not only reveals the complexity of the visual system, but is also widely used in multiple fields. In the experience research of the test subjects, it helps designers deeply understand the focus of attention and visual preferences of the test subjects during the interaction process, and provides substantial data support for the optimization of product interfaces. In medical research, eye tracking technology has become a powerful tool for studying cognitive functions and disease diagnosis, especially in the field of ophthalmology, where the visual state of individuals can be effectively evaluated by analyzing eye movements. In the field of traffic safety, eye tracking technology is widely used in driving behavior research. By monitoring the driver's gaze point and evaluating his attention to road conditions, it provides support for the improvement of intelligent driving systems.
[0003] Despite the broad application prospects of eye tracking technology in various fields, effective classification of eye movement data has always been an important research problem. The ternary eye movement classification task aims to classify eye movement behaviors into three main types: fixation, saccade, and smooth pursuit. These three eye movement types reflect the different movement states of the human eye when observing visual scenes, each with its own unique characteristics and functions. Existing eye movement classification methods have some significant shortcomings, including but not limited to the following aspects: 1) Lack of robustness in uncontrolled test subject environments. In test subject environments that cannot be controlled in advance, eye movement data is more diverse and difficult to predict. Existing algorithms lack robustness and adaptability in dealing with these uncertainties, which affects the classification performance. 2) Inconsistent classification of specific eye movement behaviors. Existing methods have confusion and errors in classifying three typical eye movement behaviors, resulting in inconsistent performance of the classifier in distinguishing these different behaviors, reducing the accuracy of interpreting eye movement data.
[0004] The methods of eye movement behavior classification are mainly divided into two categories: threshold method and probability method. Among the threshold methods, the common ones include velocity threshold recognition (I-VT) and discreteness threshold recognition (I-DT). The velocity threshold method distinguishes between fixation and rapid saccade by setting a threshold based on the speed of the sample point. If the sample speed is lower than the threshold, it is marked as fixation, otherwise it is a saccade. The discreteness threshold method calculates the discreteness of the coordinate information of the sample point in a certain interval and separates the fixation and saccade points through the discreteness threshold. Based on these binary classification algorithms, the velocity-velocity threshold recognition (I-VVT) method emerged, which uses two velocity thresholds to identify ternary eye movement behaviors. In addition, there is also an I-VDT algorithm that combines the two thresholds. It uses the velocity threshold to detect saccade behavior, and adjusts the discreteness to distinguish between fixation and smooth pursuit, and classifies each sub-segment according to the unique characteristics of each movement type. However, the limitations of these methods are their reliance on statistical data, and the determination of the threshold is limited by the scale and quality of the data, so they are less adaptable in practical applications. In addition to the threshold method, there is also a class of probabilistic algorithms, among which the hidden Markov model (HMM) is a common representative. HMM recognition uses probability analysis to infer the most likely eye movement pattern and characterizes the probability distribution of gaze shifts through bimodal HMM. The I-BDT algorithm based on Bayesian decision theory calculates the posterior classification probability through prior and likelihood information, and has the advantage of online recognition. However, the performance of I-BDT depends on the accuracy of prior and likelihood information. Inaccurate prior information or data distribution deviation from the likelihood assumption may lead to errors. The I-VDT-HMM fusion algorithm, which is built on the concept of I-VDT and uses speed and discreteness as feature parameters to hierarchically train the HMM model, does not fully achieve feature fusion for data classification from the perspective of model training. Summary of the invention
[0005] In order to achieve high-accuracy and high-robustness ternary eye movement classification, the present invention relates to an eye movement behavior recognition and classification method and system, in particular, a method and system suitable for commercial-grade desktop eye trackers.
[0006] The method of the present invention combines a Gaussian mixture model with a hidden Markov model and adopts a hierarchical training method to achieve efficient and accurate classification of eye movement data.
[0007] An eye movement behavior classification method based on a Gaussian mixture model and a hidden Markov model, wherein a hardware system for implementing the eye movement behavior classification method comprises a desktop eye tracker, a host computer and a display screen; the host computer is a host computer, and the host computer has a Gaussian mixture model program and a hidden Markov model program;
[0008] The steps for eye movement behavior classification are as follows:
[0009] (1) Collecting the original eye movement data of the tested person (1.1) When the hardware system is working, a red dot with a diameter of 80 pixels is displayed on the display screen. The red dot is located at any position on the display screen, and the duration of each appearance is 500 to 1000 milliseconds;
[0010] (1.2) The subject sits in front of a display screen and clicks a start button on the display screen with a mouse, and a visual stimulus of red dots appears on the display screen more than three times in a regular manner; the visual stimulus includes three types of eye movement behaviors to be identified, namely fixation, tracking, and saccade;
[0011] (1.3) During the operation of the hardware system, the desktop eye tracker continuously collects the position coordinates and timestamp of the gaze point of the subject on the display screen, and transmits them to the host computer, which processes them to obtain raw eye movement data, wherein each piece of data in the raw eye movement data consists of the pixel coordinates of the gaze point and the speed of the gaze point;
[0012] (2) Raw eye movement data preprocessing
[0013] Performing spatial feature preprocessing on the raw eye movement data, using a clustering method to group the pixel coordinates of the gaze point and the speed of the gaze point as a whole into at least 3 groups, each group of eye movement data having no less than 50 pieces, to obtain more than 3 groups of preprocessed eye movement data;
[0014] (3) Establishing Gaussian mixture model and hidden Markov model
[0015] (3.1) Establishing Gaussian mixture model
[0016] (3.1.1) performing statistical analysis on the three or more pre-processed eye movement data, and based on the similarity in speed of the three or more pre-processed eye movement data, when the speed difference between any two data points is less than a preset threshold of 0.002, the two data points are considered to be similar and are classified into the same cluster;
[0017] The three or more groups of eye movement data for statistical analysis are divided into three clusters according to the speed difference. Each cluster corresponds to a predefined eye movement behavior, namely, the fixation Gaussian distribution model, the pursuit Gaussian distribution model, and the saccade Gaussian distribution model.
[0018] Initialize three predefined eye movement behaviors. For the gaze Gaussian distribution model, take the center position of the cluster with the lowest average speed as the center position of the gaze Gaussian distribution model, set the initial weight of the gaze Gaussian distribution to 1 / 3, and set the initial covariance matrix of the gaze Gaussian distribution to the identity matrix.
[0019] For the saccade Gaussian distribution model, the center position of the cluster with the largest average speed is used as the center position of the saccade Gaussian distribution model. Similarly, the initial weight of the saccade Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the saccade Gaussian distribution is set to the unit matrix.
[0020] For the tracking Gaussian distribution model, the center position of the remaining third cluster is used as the center position of the tracking Gaussian distribution model, the initial weight of the tracking Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the tracking Gaussian distribution is set to the unit matrix;
[0021] The initialization Gaussian mixture model is composed of the fixation Gaussian distribution model, the pursuit Gaussian distribution model and the saccade Gaussian distribution model;
[0022] (3.1.2) adjusting the parameters of the initialized Gaussian mixture model by an iterative optimization method so that it better fits the actual probability distribution of each eye movement behavior, and setting the number of iterations to at least 50 times; and determining the degree of optimization of the Gaussian mixture model by introducing a Q function to obtain an optimized Gaussian mixture model;
[0023] (3.2) Establish the hidden Markov model (3.2.1) Initialize the three parameters of the hidden Markov model:
[0024] Initial probability matrix π=[π1,π2,…,π N ], represents the initial probability of each eye movement behavior at the initial moment, and the initial probability of each eye movement behavior is set to 1 / 3;
[0025] The transition probability matrix A = [a ij ], represents the transition probability from one eye movement behavior to another, ensuring that the sum of each row element of the transition probability matrix A is 1;
[0026] Emission probability matrix B = [b j (o t )], which represents the probability that each data in the eye movement data belongs to one of the three eye movement behaviors. Each row element of the emission probability matrix B is determined by the parameters of the Gaussian mixture model in (3.1.2);
[0027] By initializing the above three parameters of the hidden Markov model, an initialized hidden Markov model is obtained;
[0028] (3.2.2) optimizing the initialized hidden Markov model by using the preprocessed three or more sets of eye movement data through an optimization algorithm to obtain an optimized hidden Markov model;
[0029] (4) Classification of eye movement behavior
[0030] Based on the Gaussian mixture model optimized in step (3.1.2) and the hidden Markov model optimized in step (3.2.2), the eye movement behavior of the three or more preprocessed eye movement data is classified; based on the eye movement data at each time point, it is inferred whether the current eye movement behavior is fixation, saccade, or tracking eye movement behavior.
[0031] The further technical solution is as follows:
[0032] The desktop eye tracker is connected to a host computer via a universal serial bus; the display screen has a size greater than 13.3 inches and is located at least 75 centimeters away from the person being measured; the display screen is connected to the host computer via a universal serial bus.
[0033] The host computer connected to the desktop eye tracker and the display screen is equipped with a central processing unit with a basic frequency of more than 2.0 GHz, and the operating system is Windows 10 64-bit operating system.
[0034] In step (1.2), the number of fixations of the visual stimulus of the red dot is 3 times, 5 times, or more than 10 times. The subject completes the fixation task within the set number of fixations, and the interval between two consecutive fixation stimuli is 0.5 seconds or 1 second. Within the range of the display screen, the host computer randomly generates the pixel coordinates of the fixation position.
[0035] In step (1.2), the fixation refers to the eye movement behavior of staying in any area for at least 500 milliseconds, indicating that the person being tested is interested in the content of the area or is processing information;
[0036] Tracking refers to the eye movement behavior of smoothly following a moving target, including tracking moving objects or text on a display screen;
[0037] The eye movement refers to the eye movement behavior in which the line of sight quickly jumps from one fixation point to another. It is a fast and short eye movement that usually occurs between fixations and is used to shift visual attention from one area to another.
[0038] In step (2), the original eye movement data is grouped using a clustering method. The specific operations are as follows:
[0039] (2.1) Clustering algorithm:
[0040] a) Initialize cluster centers:
[0041] Randomly select K data from the original eye movement data as the initial clustering centers, and divide the original eye movement data into K eye movement data groups, where the value of K is at least 4;
[0042] b) Assign data to the nearest cluster center:
[0043] Calculate the distance between each original eye movement data in the K eye movement data groups and the K cluster centers, assign the original eye movement data closest to a cluster center to the nearest eye movement data group, and obtain K new eye movement data groups;
[0044] c) Update cluster centers:
[0045] Calculate the data average of each of the K new eye movement data groups, and use the data average as a new cluster center;
[0046] d) Iterative processing:
[0047] Repeat step b) and step c) until a predetermined number of iterations of 50 is reached; K final cluster centers and K final eye movement data groups are obtained;
[0048] (2.2) Elbow rule determines the optimal number of groups:
[0049] Apply the elbow rule to determine the optimal number of groups. The specific steps are as follows:
[0050] Calculate the sum of squared errors (Sum of Squared Errors) of the clustering results of the K final cluster centers. The sum of squared errors formula (1) is as follows:
[0051]
[0052] In formula (1), k is the number of cluster centers, n i is the number of data points in the ith group, is the jth data point in the i-th group, c i is the cluster center of the i-th group;
[0053] When the number of groups k gradually increases, the error sum of squares SSE(k) usually shows a decreasing trend, because more groups can better fit the distribution of data and reduce the distance between the data point and the cluster center of the group to which it belongs. After a certain number of groups k, the decrease of the error sum of squares SSE(k) will slow down significantly and form an inflection point. The inflection point represents the optimal balance between clustering performance and the number of groups. Based on the trend that the error sum of squares of clustering results decreases as the number of groups k increases, the number of cluster centers k corresponding to the point where the error sum of squares SSE(k) decreases is selected to determine the optimal number of groups. For the k final cluster centers, the preprocessed k-1 groups of eye movement data are obtained.
[0054] In step (3.1.2), the operation of adjusting the parameters of the Gaussian mixture model by the iterative optimization method is as follows:
[0055] a). Initial parameter definition of Gaussian mixture model
[0056] The formula (2) of the Gaussian mixture model is as follows:
[0057]
[0058] In formula (2), P(x) represents the probability of observing eye movement data x, k represents the kth eye movement behavior, which is fixation, tracking, and saccade, respectively, and c k represents the weight of the k-th eye movement behavior, N(x|μ k ,∑ k ) represents the probability density function of the kth eye movement Gaussian distribution, μ k and∑ k Respectively represent the mean and covariance matrix of the kth Gaussian distribution;
[0059] Taking the three eye movement behaviors of fixation, tracking and saccade as examples, the probability density function (3) of the Gaussian mixture model is as follows:
[0060] P(x)=c1·N(x|μ1,∑1)+c2·N(x|μ2,∑2)+c3·N(x|μ3,∑3)(3)
[0061] In formula (3), the weights c1, c2, c3, the mean values μ1, μ2, μ3 and the covariance matrices ∑1, ∑2, ∑3 are all parameters to be optimized in the Gaussian mixture model;
[0062] b). Calculation of responsibility
[0063] To adjust the Gaussian mixture model parameters, it is necessary to calculate each eye movement data x n The probability of the three eye movement behaviors of fixation, tracking, and saccade is called the responsibility γ(z nk ), responsibility γ(z nk ) is calculated as follows:
[0064]
[0065] In formula (4), x n represents the nth observed eye movement data, z nk is a symbolic variable, indicating that the nth eye movement data belongs to the kth Gaussian distribution;
[0066] c). Update Gaussian mixture model parameters
[0067] Responsibility for useγ(z nk ) to update the parameters of the Gaussian mixture model, which include the weight c k , average value μ k and the covariance matrix ∑ k ;
[0068] Weight c k The update formula (5) is as follows:
[0069]
[0070] Average value μ k The update formula (6) is as follows:
[0071]
[0072] Covariance matrix∑ k The update formula (7) is as follows:
[0073]
[0074] Repeat the calculation of formula (5)-(7) at least 50 times to update the parameters of the Gaussian mixture model;
[0075] In order to determine the degree of optimization of the Gaussian mixture model, the Q function is introduced. The Q function formula (8) is as follows:
[0076]
[0077] In formula (8), θ represents the Gaussian mixture model parameters in the current iteration, including the mixing coefficients c of each Gaussian distribution k , mean μ k and the covariance matrix ∑ k ,θ (t) represents the Gaussian mixture model parameters in the previous iteration, P(x i ,z ik |θ) Under the current model parameters θ, the eye movement data x i The probability of belonging to the kth Gaussian distribution, P(z ik |x i ,θ (t) ) indicates that the model parameter θ in the previous iteration (t) Next, eye movement data x i The probability of belonging to the kth Gaussian distribution;
[0078] The Q function measures the degree of optimization of the model by comparing the parameters of the current Gaussian mixture model with the parameters of the Gaussian mixture model of the previous iteration. The larger the value of the Q function, the greater the improvement of the current model parameters relative to the previous iteration. When the value of the Q function is less than the set threshold of 0.001, or the number of calculations reaches the preset number of repetitions, the parameter optimization of the Gaussian mixture model is completed, and the parameters of the Gaussian mixture model at this time are recorded, including the weight c k , average value μ k and the covariance matrix ∑ k , that is, the optimized Gaussian mixture model is obtained.
[0079] In step (3.2.2), the method for optimizing the parameters of the hidden Markov model is performed as follows:
[0080] a). Define and calculate forward probability
[0081] The observation state sequence of eye movement behavior is defined as O = (o1, o2, ..., o T ), the above sequence represents the eye movement behavior observed from the initial moment to the last moment. In the process of hidden Markov model optimization, the forward probability α is first calculated. t (i), the forward probability is calculated as follows:
[0082]
[0083] Formula (9) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o1, o2, …, o t ), in formula (9), a ji represents the transition probability from state j at time t-1 to state i at time t, b i represents the emission probability of state i, a ji and b i are all parameters of the hidden Markov model;
[0084] b) Define and calculate backward probability
[0085] Calculate the backward probability β t (i), the calculation formula (10) of the backward probability is as follows:
[0086]
[0087] Formula (10) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o t+1 ,o t+1 ,…,o t ), in formula (10), a ij represents the transition probability from state j at time t+1 to state i at time t, b j represents the emission probability of state j, a ij and b j are all parameters of the hidden Markov model;
[0088] c). Update the parameters of the hidden Markov model
[0089] Using the forward probability and the backward probability, calculate the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the parameters of the hidden Markov model, where the prior probability γ t(i) represents the probability of state i at time t, and the joint probability ξ t (i, j) represents the probability that the state is i at time t and the state is j at time t+1, and the prior probability γ t The calculation formula (11) of (i) is as follows:
[0090]
[0091] Joint probability ξ t The calculation formula (12) of (i, j) is as follows:
[0092]
[0093] In formulas (11) and (12), the transition probability a ij represents the probability of transitioning from state i to state j, and the emission probability b j (o t+1 ) means that the observed value in state j is o t+1 The probability of ; by using the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the three parameters of the initial probability matrix π, the transition probability matrix A and the emission probability matrix B in the hidden Markov model. The specific update operation is as follows:
[0094] The update formula (13) of the initial state probability vector π is as follows:
[0095] π i =γ1(i) (13)
[0096] In formula (13), π i Represents the i-th element of the optimized initial state probability vector;
[0097] The update formula (14) of the state transition probability matrix A is as follows:
[0098]
[0099] In formula (14), a ij Represents the element of the i-th row and j-th column of the optimized state transition probability matrix;
[0100] The update formula (15) of the emission probability matrix B is as follows:
[0101]
[0102] In formula (15), b j (k) represents the jth row and kth column element of the optimized emission probability matrix, v k represents the kth possible observation value;
[0103] By repeating the calculation of formulas (13)-(15) at least 100 times, the three parameters of the above hidden Markov model are optimized so that the hidden Markov model can continuously fit the eye movement data. When the preset number of repetitions is reached, the hidden Markov model is considered to have been optimized, and finally the optimized hidden Markov model is obtained.
[0104] In step (4), the eye movement behavior classification is performed on the preprocessed three or more groups of eye movement data to obtain the eye movement behavior at each moment. The specific operation is as follows:
[0105] (4.1) Determine the optimal path probability at each moment
[0106] Assume that the hidden state sequence of eye movement behavior is S = (s1, s2, ..., s T ), the hidden state of the eye movement behavior represents the actual eye movement behavior, which is one of the eye movement behaviors of fixation, tracking and saccade, and the sequence S represents all the actual eye movement behaviors from the initial moment to the final moment;
[0107] Assume that the observed state sequence of eye movement behavior is O = (o1, o2, ..., o T ), the observed state of the eye movement behavior represents the eye movement behavior estimated based on the observed eye movement data, which is also a kind of eye movement behavior among fixation, tracking and saccade, and the sequence O represents all the eye movement behaviors estimated based on the observed eye movement data from the initial moment to the final moment;
[0108] In order to obtain the hidden state sequence with the highest probability, that is, to obtain the most likely eye movement behavior, the optimal sequence function is defined as δ t (i), the optimal sequence function δ t (i) indicates that when the hidden state is i at time t, the observed state is (o1, o2, ..., o t ), the maximum probability of the optimal sequence function δ t The specific value of (i) is obtained by recursive calculation, and the optimal sequence function δ t The recursive formula (16) of (i) is as follows:
[0109]
[0110] In formula (16), a ij represents the transition probability from state i to state j, Indicates that o is observed in state j t The emission probability, δ t-1 (i) represents the probability of state i at time t-1.
[0111] By recursively formula (16), the maximum probability δ corresponding to each hidden state i is calculated at each time t t(i); (4.2) Determine the optimal precursor state at each moment
[0112] Get the maximum probability δ of the hidden state at time t t (i) Based on this, we need to record the most likely previous state of hidden state i at time t, that is, the optimal predecessor state at time t; define the optimal predecessor state ψ t (i), record the most likely previous eye movement behavior of state i at time t, the optimal precursor state ψ t Formula (17) of (i) is as follows:
[0113]
[0114] Formula (17) records the optimal precursor state at each time t, and recursively finds the hidden state from the final time to the initial time, thereby classifying the eye movement behavior at each time point;
[0115] (4.3) Classification of eye movement behavior
[0116] Using the optimal path probability δ t (i) and the optimal precursor state ψ t (i), calculate the maximum probability P of the hidden state sequence * and the terminal state of the hidden state sequence Classify eye movement behavior;
[0117] The maximum probability of the hidden state sequence is P * Indicates that at the final moment T, the optimal path probability δ corresponding to the three hidden states of fixation, pursuit and saccade is calculated respectively t (i) and select the optimal path probability value with the largest value as the maximum probability P of the hidden state sequence * , the maximum probability P * The calculation formula (18) is as follows:
[0118]
[0119] In formula (18), N represents the number of hidden states, and the upper limit here is 3;
[0120] The maximum probability P of the hidden state sequence at the final moment T is obtained * On this basis, it is also necessary to determine the terminal state of the hidden state sequence Termination Status The calculation formula (19) is as follows:
[0121]
[0122] When the termination status is obtained , we start to recurse from the final moment T to the initial moment to obtain the hidden state at each moment; the recursive formula (20) is as follows:
[0123]
[0124] In formula (20), represents the optimal hidden state at time t, represents the optimal hidden state at time t+1 The most likely previous state corresponding to the hidden state is obtained; the hidden state of the entire hidden state sequence at each moment is determined by formula (20), that is, the eye movement behavior from the initial moment to the final moment is classified; it is inferred that the current eye movement behavior belongs to which type of fixation, saccade or tracking eye movement behavior, and the eye movement behavior classification is completed.
[0125] The three types of eye movement behaviors are specifically classified as follows:
[0126] Attention: When the probability δ corresponding to the hidden state t (i) When it exceeds 0.9 and the state lasts for more than 200 milliseconds, it indicates that the current eye movement behavior is fixation;
[0127] Eye saccade: When the hidden state changes within 50 milliseconds, and the corresponding transition probability a ij If it is greater than 0.8, it indicates that the current eye movement behavior is saccade;
[0128] Tracking: When the hidden state changes within 500 milliseconds, and the corresponding transition probability a ij If it is greater than 0.9, it indicates that the current eye movement behavior is tracking.
[0129] The beneficial technical effects of the present invention are embodied in the following aspects:
[0130] 1. The present invention adopts an algorithm combining a Gaussian Mixture Model (GMM) and a Hidden Markov Model (HMM). GMM is a mathematical model composed of multiple Gaussian distributions, each of which represents a subgroup or mode in the data. The characteristics of this multimodal modeling enable GMM to better adapt to the different distribution states of multiple eye movement behaviors in the data. Mathematically, GMM models the probability distribution of the data and determines the parameters of each Gaussian distribution by maximum likelihood estimation, thereby achieving reasonable modeling of the data. HMM is a statistical model for modeling sequence data, which performs probability analysis on the state transitions in the sequence. In eye movement classification, HMM can be used to capture the state changes between eye movement patterns, so as to more accurately infer the relationship between different eye movement behaviors. Mathematically, HMM consists of a state set, an observation set, a state transition probability matrix and an observation probability matrix. Through the adjustment and optimization of these elements, HMM can better reflect the dynamic changes of eye movement behavior. By combining the two, the algorithm of the present invention can more flexibly adapt to multimodal eye movement behaviors and more deeply model the relationship between different eye movement behaviors through probabilistic analysis. This combination of mathematical models provides a more accurate and comprehensive tool for eye movement classification, thus solving the limitations of existing methods.
[0131] 2. Use clustering algorithm to finely segment the eye movement path data, and divide the eye movement path into different sub-segments according to the position and speed characteristics of the original eye movement path. By calculating the sum of squared clustering errors (SSE) under different numbers of segments, the number of segments corresponding to the point where the SSE decreases slowly is selected as the optimal number of segments. This method can find a balance between clustering performance and computational complexity, avoiding overfitting or underfitting problems. This method can effectively capture local features in eye movement data and provide more fine-grained information for subsequent eye movement behavior classification.
[0132] 3. Adaptable to commercial-grade desktop eye trackers: The present invention takes full account of the characteristics of commercial-grade desktop eye trackers during design and has strong real-time performance. Traditional eye movement behavior classification methods may require the use of a large amount of eye movement data for model pre-training or rely on a large amount of statistical data to determine appropriate thresholds to ensure that the model can better understand the characteristics of eye movement data. Compared with traditional methods, the present invention does not require complex pre-training or model construction, thereby reducing the burden of early data processing.
[0133] 4. High accuracy and stability: The hidden Markov model is introduced to model the eye movement behavior in time series, and the hierarchical modeling method is used to significantly improve the classification accuracy of eye movement behavior. The present invention adopts a path segmentation strategy, combined with the elbow rule to segment the eye movement path, extract key eye movement data features, and perform two-layer classification of the Gaussian hidden Markov model through the segmented path. The first layer is characterized by coordinate information, and the second layer is mainly characterized by speed information, which realizes accurate classification of eye movement behaviors such as gaze, saccade and smooth pursuit. Experiments have proved that the present invention is superior to traditional algorithms in terms of accuracy, precision and recall rate, especially in terms of stability when processing different eye movement behaviors. Compared with the traditional algorithms I-BDT and I-VDT, the performance of the method of the present invention is more stable, with less variability in accuracy and recall rate. The GMM-HMM algorithm of the present invention has an accuracy of 94.39% ± 2.08%, an accuracy of 95.31% ± 3.71%, and a recall rate of 94.98% ± 3.93% in the classification of three types of eye movement behaviors. In terms of gaze behavior, the algorithm has an accuracy of up to 0.9743, a recall rate of 0.9665, and an F1 score of 0.9699, which shows that the algorithm minimizes the balance effect of false positives and false negatives while accurately identifying real positioning events. For the classification of tracking behavior, the algorithm has an accuracy of 0.8784, a recall rate of 0.9076, and an F1 score of 0.8893, indicating that the algorithm has achieved a balanced effect in effectively processing complex tracking behaviors. In the classification of eye movement behavior, the Gaussian-Hidden Markov algorithm of the present invention has a higher accuracy, with an accuracy of 0.9301, which is better than the I-BDT algorithm.
[0134] The specific data and performance of these beneficial technical effects indicate that the eye movement behavior classification algorithm of the present invention has achieved significant advantages in improving classification accuracy, reducing complexity, and adapting to practical applications, providing strong support for the progress in the field of eye movement data processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0135] Figure 1 is a flowchart of the visual stimulation program executed by the host computer in an embodiment of the present invention;
[0136] Figure 2 This is a segmentation result diagram of selecting three cluster centers for the acquired eye movement sequence in Example 1 of the present invention;
[0137] Figure 3 This is a segmentation result diagram of selecting 4 cluster centers for the acquired eye movement sequence in Example 1 of the present invention;
[0138] Figure 4 This is a segmentation result diagram of selecting 5 cluster centers for the acquired eye movement sequence in Example 1 of the present invention;
[0139] Figure 5 is the SSE value of the eye movement sequence under different clustering numbers in Example 1 of the present invention, which is used to determine the optimal segmentation method diagram;
[0140] Figure 6 is a visualization display diagram of the original eye movement trajectory in Example 1 of the present invention;
[0141] Figure 7 This is a visualization result diagram of ternary eye movement classification performed on the original eye movement trajectory using the present invention in Example 1 of the present invention;
[0142] Figure 8 is a classification result diagram of the x-axis of the eye movement trajectory in Example 1 of the present invention;
[0143] Fig. 9 is a classification result diagram of the y-axis of the eye movement trajectory in Example 1 of the present invention;
[0144] Fig.10 This is a segmentation result diagram of selecting 9 cluster centers for the acquired eye movement sequence in Example 2 of the present invention;
[0145] Fig.11 is the SSE value of the eye movement sequence under different clustering numbers in Example 2 of the present invention, which is used to determine the optimal segmentation method diagram;
[0146] Fig.12 is a visualization diagram of the original eye movement trajectory in Example 2 of the present invention;
[0147] Fig.13 This is a visualization result diagram of ternary eye movement classification performed on the original eye movement trajectory using the present invention in Example 2 of the present invention;
[0148] Fig.14 is a classification result diagram of the x-axis of the eye movement trajectory in Example 2 of the present invention;
[0149] Fig.15 It is a classification result diagram of the y-axis of the eye movement trajectory in Example 2 of the present invention.
[0150] Specific implementation method
[0151] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings. Example
[0152] An eye movement behavior classification method based on a Gaussian mixture model and a hidden Markov model, and a hardware system for implementing the eye movement behavior classification method comprises a desktop eye tracker, a host computer and a display screen.
[0153] The host computer is a host computer, and the desktop eye tracker is connected to the host computer via a universal serial bus. The host computer has a Gaussian mixture model program and a hidden Markov model program. At the same time, the host computer is equipped with a central processing unit with a basic frequency of more than 2.0 GHz, and the operating system is a Windows 10 64-bit operating system. The size of the display screen is greater than 13.3 inches and is located at a distance of at least 75 cm from the person being measured. The display screen is connected to the host computer via a universal serial bus.
[0154] The steps for eye movement behavior classification are as follows:
[0155] (1) Collecting the original eye movement data of the subjects
[0156] See also Figure 1 , showing the workflow of collecting the original eye movement data of the subjects, including four stages: start, user gaze behavior, rapid positioning (including eye saccade and tracking behavior), and end.
[0157] (1.1) When the system is working, a red dot with a diameter of 80 pixels is displayed on the screen. The red dot is located at any position on the screen and lasts for 500 to 1000 milliseconds each time.
[0158] (1.2) The subject sits in front of a display screen and clicks a start button on the display screen with a mouse, and a visual stimulus of red dots appears on the display screen more than three times in a regular pattern; the visual stimulus includes three types of eye movement behaviors to be identified, namely, fixation, tracking, and saccade.
[0159] In the above step (1.2), the number of fixations of the visual stimulus of the red dot is 4 times, and the subject completes the fixation task within the set number of fixations, and the interval between two consecutive fixation stimuli is 0.5 seconds; within the range of the display screen, the host computer randomly generates the pixel coordinates of the fixation position.
[0160] The term "gaze" refers to an eye movement behavior in which the gaze stays in any area for at least 500 milliseconds, indicating that the person being tested is interested in the content of the area or is processing information;
[0161] Tracking refers to the eye movement behavior of smoothly following a moving target, including tracking moving objects or text on a display screen;
[0162] The eye movement refers to the eye movement behavior in which the line of sight quickly jumps from one fixation point to another. It is a fast and short eye movement that usually occurs between fixations and is used to shift visual attention from one area to another.
[0163] (1.3) During the operation of the system, the desktop eye tracker continuously collects the position coordinates and timestamp of the subject's gaze point on the display screen, and transmits them to the host computer. After processing by the host computer, the raw eye movement data is obtained. Each piece of data in the raw eye movement data consists of the pixel coordinates of the gaze point and the speed of the gaze point.
[0164] In this embodiment 1, the following program is set in the host computer: at the initial moment, a target circular pattern appears in the center of the display screen to prompt the person being tested that the program has started and sampling has begun. After waiting for about 1000 milliseconds, the stimulus begins to be presented, and a circular target appears above the display screen. 500 milliseconds after it appears, the circular target will quickly move to the left side of the display screen for 1000 milliseconds. It will remain on the left side of the display screen for 500 milliseconds, and then quickly flash to the bottom of the display screen and the right side of the display screen in turn, both remaining for 1000 milliseconds. Finally, the circular target returns to the center of the display screen, stays for 500 milliseconds, and then disappears, indicating that the program has ended and sampling has stopped.
[0165] (2) Raw eye movement data preprocessing
[0166] The original eye movement data is preprocessed with spatial features, and the pixel coordinates of the gaze point and the speed of the gaze point are grouped as a whole using a clustering method. In this embodiment, they are divided into 3 groups, each with 50 eye movement data, to obtain 3 groups of preprocessed eye movement data.
[0167] The specific operations of using clustering method to group the original eye movement data are as follows:
[0168] (2.1) Clustering algorithm:
[0169] The specific operation process is as follows:
[0170] a) Initialize cluster centers:
[0171] Randomly select K data from the original eye movement data as initial cluster centers, and divide the original eye movement data into K eye movement data groups. In this embodiment, the maximum value of K is 4;
[0172] b) Assign data to the nearest cluster center:
[0173] The distance between each original eye movement data in the K eye movement data groups and the K cluster centers is calculated, and the original eye movement data closest to a cluster center is assigned to the nearest eye movement data group to obtain K new eye movement data groups.
[0174] c) Update cluster centers:
[0175] The data average value of each new eye movement data group in the K new eye movement data groups is calculated, and the data average value is used as a new cluster center.
[0176] d) Iterative processing:
[0177] Repeat step b) and step c) until a predetermined number of iterations of 50 is reached; and K final cluster centers and K final eye movement data groups are obtained.
[0178] (2.2) Elbow rule determines the optimal number of groups
[0179] See also Figure 2 , 3 ,4,The scattered points in the above three figures all represent the gaze point trajectory, and the coordinates of the gaze point trajectory are all expressed in normalized coordinates. Figure 2 It represents the result of clustering the original eye movement data into three groups. Figure 3 It represents the result of clustering the original eye movement data into four groups. Figure 4 It represents the result of clustering the original eye movement data into 5 groups. The stimulation program written by the host computer in step (1.3) contains four fixed stimulation targets to represent the gaze behavior. When the set number of clusters is close to the gaze behavior of the person being measured, the eye movement data can better describe the gaze behavior of the person being measured.
[0180] Apply the elbow rule to determine the optimal number of groups and calculate the sum of squared errors of the clustering results of the K final cluster centers. The sum of squared errors formula (1) is as follows:
[0181]
[0182] In formula (1), k is the number of cluster centers, n i is the number of data points in the ith group, is the jth data point in the i-th group, c i is the cluster center of the ith group.
[0183] When the number of groups k gradually increases, the error sum of squares SSE(k) usually shows a decreasing trend, because more groups can better fit the distribution of data and reduce the distance between the data point and the cluster center of the group to which it belongs. After a certain number of groups k, the decrease of the error sum of squares SSE(k) will slow down significantly and form an inflection point. The inflection point represents the optimal balance between clustering performance and the number of groups. Based on the trend that the error sum of squares of clustering results decreases as the number of groups k increases, the number of cluster centers k corresponding to the point where the error sum of squares SSE(k) decreases is selected to determine the optimal number of groups. For the k final cluster centers, the preprocessed k-1 groups of eye movement data are obtained.
[0184] See also Figure 5 , for this embodiment, the optimal number of cluster centers k can be determined to be 4, and 3 groups of eye movement data are obtained, corresponding to Figure 5 The inflection point is when k=4.
[0185] See also Figure 6 The black scattered points in the trajectory diagram of the original eye movement data represent the trajectory of the gaze point. Figure 6 The x-axis and y-axis represent the coordinates of the gaze point, using normalized coordinates, and the z-axis represents the speed of the gaze point. Figure 7 , using the determined number of groups as 3, the original eye movement data is clustered to form 3 groups, and each group is marked with scatter points in yellow, green and blue.
[0186] (3) Establishment of Gaussian mixture model and hidden Markov model
[0187] (3.1) Establishing Gaussian mixture model
[0188] (3.1.1) The three groups of preprocessed eye movement data are statistically analyzed. Based on the similarity in speed of the three groups of preprocessed eye movement data, that is, when the speed difference between any two data points is less than the preset threshold of 0.002, the two data points are considered to be similar and are classified into the same cluster.
[0189] The three groups of eye movement data for statistical analysis were divided into three clusters according to the speed differences. Each cluster corresponds to a predefined eye movement behavior, namely, the gaze Gaussian distribution model, the pursuit Gaussian distribution model and the saccade Gaussian distribution model.
[0190] Initialize three predefined eye movement behaviors. For the gaze Gaussian distribution model, take the center position of the cluster with the lowest average speed as the center position of the gaze Gaussian distribution model, set the initial weight of the gaze Gaussian distribution to 1 / 3, and set the initial covariance matrix of the gaze Gaussian distribution to the unit matrix.
[0191] For the saccade Gaussian distribution model, the center position of the cluster with the largest average speed is used as the center position of the saccade Gaussian distribution model. Similarly, the initial weight of the saccade Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the saccade Gaussian distribution is set to the unit matrix.
[0192] For the tracking Gaussian distribution model, the center position of the remaining third cluster is used as the center position of the tracking Gaussian distribution model, the initial weight of the tracking Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the tracking Gaussian distribution is set to the unit matrix.
[0193] The initialization Gaussian mixture model is composed of the fixation Gaussian distribution model, the pursuit Gaussian distribution model and the saccade Gaussian distribution model.
[0194] (3.1.2) The parameters of the initialized Gaussian mixture model are adjusted by an iterative optimization method to make it better fit the actual probability distribution of each eye movement behavior. The number of iterations is set to at least 50 times. The degree of optimization of the Gaussian mixture model is determined by introducing the Q function to obtain an optimized Gaussian mixture model.
[0195] The specific optimization operations are as follows:
[0196] a) Initial parameter definition of Gaussian mixture model
[0197] The formula (2) of the Gaussian mixture model is as follows:
[0198]
[0199] In formula (2), P(x) represents the probability of observing eye movement data x, k represents the kth eye movement behavior, which is fixation, tracking, and saccade, respectively, and c k represents the weight of the k-th eye movement behavior, N(x|μ k ,∑ k ) represents the probability density function of the kth eye movement Gaussian distribution, μ k and∑ k denote the mean and covariance matrix of the kth Gaussian distribution respectively.
[0200] Taking the three eye movement behaviors of fixation, tracking and saccade as examples, the probability density function (3) of the Gaussian mixture model is as follows:
[0201] P(x)=c1·N(x|μ1,∑1)+c2·N(x|μ2,∑2)+c3·N(x|μ3,∑3)(3)
[0202] In formula (3), the weights c1, c2, c3, the mean values μ1, μ2, μ3 and the covariance matrices ∑1, ∑2, ∑3 are all parameters to be optimized in the Gaussian mixture model.
[0203] b) Calculation of liability
[0204] To adjust the Gaussian mixture model parameters, it is necessary to calculate each eye movement data x n The probability of the three eye movement behaviors of fixation, tracking, and saccade is called the responsibility γ(z nk ), responsibility γ(z nk ) is calculated as follows:
[0205]
[0206] In formula (4), x n represents the nth observed eye movement data, z nkis a symbolic variable, indicating that the nth eye movement data belongs to the kth Gaussian distribution.
[0207] c) Update Gaussian mixture model parameters
[0208] Responsibility for useγ(z nk ) to update the parameters of the Gaussian mixture model, which include the weight c k , average value μ k and the covariance matrix ∑ k ;
[0209] Weight c k The update formula (5) is as follows:
[0210]
[0211] Average value μ k The update formula (6) is as follows:
[0212]
[0213] Covariance matrix∑ k The update formula (7) is as follows:
[0214]
[0215] Repeat the calculation of formulas (5)-(7) at least 50 times to update the parameters of the Gaussian mixture model.
[0216] In order to determine the degree of optimization of the Gaussian mixture model, the Q function is introduced. The Q function formula (8) is as follows:
[0217]
[0218] In formula (8), θ represents the Gaussian mixture model parameters in the current iteration, including the mixing coefficients c of each Gaussian distribution k , mean μ k and the covariance matrix ∑ k ,θ (t) represents the Gaussian mixture model parameters in the previous iteration, P(x i ,z ik |θ) Under the current model parameters θ, the eye movement data x i The probability of belonging to the kth Gaussian distribution, P(z ik |x i ,θ (t) ) indicates that the model parameter θ in the previous iteration (t) Next, eye movement data x i The probability of belonging to the kth Gaussian distribution.
[0219] The Q function measures the degree of optimization of the model by comparing the parameters of the current Gaussian mixture model with the parameters of the Gaussian mixture model of the previous iteration. The larger the value of the Q function, the greater the improvement of the current model parameters relative to the previous iteration. When the value of the Q function is less than the set threshold of 0.001, or the number of calculations reaches the preset number of repetitions, the parameter optimization of the Gaussian mixture model is completed, and the parameters of the Gaussian mixture model at this time are recorded, including the weight c k , average value μ k and the covariance matrix ∑ k , that is, the optimized Gaussian mixture model is obtained.
[0220] (3.2) Establishing Hidden Markov Model
[0221] (3.2.1) Initialize the three parameters of the hidden Markov model:
[0222] Initial probability matrix π=[π1,π2,…,π N ], represents the initial probability of each eye movement behavior at the initial moment, and the initial probability of each eye movement behavior is set to 1 / 3;
[0223] The transition probability matrix A = [a ij ], represents the transition probability from one eye movement behavior to another, ensuring that the sum of each row element of the transition probability matrix A is 1;
[0224] Emission probability matrix B = [b j (o t )], which represents the probability that each data in the eye movement data belongs to one of the three eye movement behaviors. Each row element of the emission probability matrix B is determined by the parameters of the Gaussian mixture model in (3.1.2).
[0225] By initializing the above three parameters of the hidden Markov model, an initialized hidden Markov model is obtained.
[0226] (3.2.2) The initialized hidden Markov model is optimized by an optimization algorithm using the three sets of preprocessed eye movement data to obtain an optimized hidden Markov model.
[0227] The specific optimization steps are as follows:
[0228] a) Define and calculate forward probability
[0229] The observation state sequence of eye movement behavior is defined as O = (o1, o2, ..., o T ), the above sequence represents the eye movement behavior observed from the initial moment to the last moment. In the process of hidden Markov model optimization, the forward probability α is first calculated. t (i), the forward probability is calculated as follows:
[0230]
[0231] Formula (9) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o1, o2, …, o t ), in formula (9), a ji represents the transition probability from state j at time t-1 to state i at time t, b i represents the emission probability of state i, a ji and b i are the parameters of the hidden Markov model.
[0232] b) Define and calculate backward probability
[0233] Calculate the backward probability β t (i), the calculation formula (10) of the backward probability is as follows:
[0234]
[0235] Formula (10) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o t+1 ,o t+1 ,…,o t ), in formula (10), a ij represents the transition probability from state j at time t+1 to state i at time t, b j represents the emission probability of state j, a ij and b j are the parameters of the hidden Markov model.
[0236] c) Update the parameters of the hidden Markov model
[0237] Using the forward probability and the backward probability, calculate the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the parameters of the hidden Markov model, where the prior probability γ t (i) represents the probability of state i at time t, and the joint probability ξ t (i, j) represents the probability that the state is i at time t and the state is j at time t+1, and the prior probability γ t The calculation formula (11) of (i) is as follows:
[0238]
[0239] Joint probability ξ t The calculation formula (12) of (i, j) is as follows:
[0240]
[0241] In formulas (11) and (12), the transition probability a ij represents the probability of transitioning from state i to state j, and the emission probability b j (o t+1 ) means that the observed value in state j is o t+1 The probability of ; by using the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the three parameters of the initial probability matrix π, the transition probability matrix A and the emission probability matrix B in the hidden Markov model. The specific update operation is as follows:
[0242] The update formula (13) of the initial state probability vector π is as follows:
[0243] π i =γ1(i) (13)
[0244] In formula (13), π i Represents the i-th element of the optimized initial state probability vector;
[0245] The update formula (14) of the state transition probability matrix A is as follows:
[0246]
[0247] In formula (14), a ij Represents the element of the i-th row and j-th column of the optimized state transition probability matrix;
[0248] The update formula (15) of the emission probability matrix B is as follows:
[0249]
[0250] In formula (15), b j (k) represents the jth row and kth column element of the optimized emission probability matrix, v k represents the kth possible observation value.
[0251] By repeating the calculation of formulas (13)-(15) at least 100 times, the three parameters of the above hidden Markov model are optimized so that the hidden Markov model can continuously fit the eye movement data. When the preset number of repetitions is reached, the hidden Markov model is considered to have been optimized, and finally the optimized hidden Markov model is obtained.
[0252] (4) Classification of eye movement behavior
[0253] Based on the Gaussian mixture model optimized in step (3.1.2) and the hidden Markov model optimized in step (3.2.2), the three sets of preprocessed eye movement data are classified into eye movement behaviors; based on the eye movement data at each time point, it is inferred which of the three eye movement behaviors the current eye movement behavior is, namely, fixation, saccade, or tracking.
[0254] (4.1) Determine the optimal path probability at each moment
[0255] Assume that the hidden state sequence of eye movement behavior is S = (s1, s2, ..., s T ), the hidden state of the eye movement behavior represents the actual eye movement behavior, which is a type of eye movement behavior among fixation, tracking and saccade, and the sequence S represents all the actual eye movement behaviors from the initial moment to the final moment.
[0256] Assume that the observed state sequence of eye movement behavior is O = (o1, o2, ..., o T ), the observed state of the eye movement behavior represents the eye movement behavior estimated based on the observed eye movement data, which is also a type of eye movement behavior among gaze, tracking and saccade. The sequence O represents all eye movement behaviors estimated based on the observed eye movement data from the initial moment to the final moment.
[0257] In order to obtain the hidden state sequence with the highest probability, that is, to obtain the most likely eye movement behavior, the optimal sequence function is defined as δ t (i), the optimal sequence function δ t (i) indicates that when the hidden state is i at time t, the observed state is (o1, o2, ..., o t ), the maximum probability of the optimal sequence function δ t The specific value of (i) is obtained by recursive calculation, and the optimal sequence function δ t The recursive formula (16) of (i) is as follows:
[0258]
[0259] In formula (16), a ij represents the transition probability from state i to state j, Indicates that o is observed in state j t The emission probability, δ t-1 (i) represents the probability of state i at time t-1.
[0260] By recursively formula (16), the maximum probability δ corresponding to each hidden state i is calculated at each time t t (i). (4.2) Determine the optimal precursor state at each moment
[0261] Get the maximum probability δ of the hidden state at time tt (i) Based on this, we need to record the most likely previous state of hidden state i at time t, that is, the optimal predecessor state at time t; define the optimal predecessor state ψ t (i), record the most likely previous eye movement behavior of state i at time t, the optimal precursor state ψ t Formula (17) of (i) is as follows:
[0262]
[0263] Formula (17) records the optimal precursor state at each time t, and recursively finds the hidden state from the final moment to the initial moment, thereby classifying the eye movement behavior at each time point.
[0264] (4.3) Classification of eye movement behavior
[0265] Using the optimal path probability δ t (i) and the optimal precursor state ψ t (i), calculate the maximum probability P of the hidden state sequence * and the terminal state of the hidden state sequence Classify eye movement behavior.
[0266] The maximum probability of the hidden state sequence is P * Indicates that at the final moment T, the optimal path probability δ corresponding to the three hidden states of fixation, pursuit and saccade is calculated respectively t (i) and select the optimal path probability value with the largest value as the maximum probability P of the hidden state sequence * , the maximum probability P * The calculation formula (18) is as follows:
[0267]
[0268] In formula (18), N represents the number of hidden states, and the upper limit here is 3;
[0269] The maximum probability P of the hidden state sequence at the final moment T is obtained * On this basis, it is also necessary to determine the terminal state of the hidden state sequence Termination Status The calculation formula (19) is as follows:
[0270]
[0271] When the termination status is obtained , we start to recurse from the final moment T to the initial moment to obtain the hidden state at each moment; the recursive formula (20) is as follows:
[0272]
[0273] In formula (20), represents the optimal hidden state at time t, represents the optimal hidden state at time t+1 The most likely previous state corresponding to the hidden state. The hidden state of each moment of the entire hidden state sequence is determined by formula (20), that is, the eye movement behavior from the initial moment to the end moment is classified. It is inferred that the current eye movement behavior belongs to which of the three eye movement behaviors, namely fixation, saccade or tracking, and the eye movement behavior classification is completed.
[0274] See also Figure 8 , Fig. 9 , the result output after state recognition of 3 sets of eye movement data. Figure 8 The a in the figure represents the trajectory of the gaze point x-axis changing with time. The horizontal axis represents time in seconds, the left vertical axis represents the gaze point position in normalized coordinates, the right vertical axis represents the gaze point speed change curve, the blue curve represents the gaze point x-axis position indicator, and the red curve represents the gaze point speed. Figure 8 b in the figure is the classification result. The horizontal axis represents time in seconds, and the vertical axis represents the classification result. The blue dots represent gaze behavior, which is concentrated in the time period with low speed and little change in position index. The yellow dots represent tracking behavior, which appears in the time period with stable change in position index but certain speed. The red dots represent saccade behavior, which is concentrated in the time period with rapid change in speed. Fig. 9 The a in the figure represents the trajectory of the gaze point y-axis changing with time. The horizontal axis represents time in seconds. The left vertical axis represents the gaze point position in normalized coordinates. The right vertical axis represents the gaze point speed change curve. The blue curve represents the gaze point y-axis position indicator, and the red curve represents the gaze point speed. Fig. 9 The b in the figure is the classification result. The horizontal axis represents time in seconds, and the vertical axis represents the classification result. Similarly, the blue dots represent gaze behavior, the yellow dots represent tracking behavior, and the red dots represent saccade behavior.
[0275] The three types of eye movement behaviors are specifically classified as follows:
[0276] Attention: When the probability δ corresponding to the hidden state t (i) When it exceeds 0.9 and the state lasts for more than 200 milliseconds, it indicates that the current eye movement behavior is fixation;
[0277] Eye saccade: When the hidden state changes within 50 milliseconds, and the corresponding transition probability a ij If it is greater than 0.8, it indicates that the current eye movement behavior is saccade;
[0278] Tracking: When the hidden state changes within 500 milliseconds, and the corresponding transition probability a ijIf it is greater than 0.9, it indicates that the current eye movement behavior is tracking.
[0279] Example 2
[0280] This embodiment 2 is an implementation process for classifying eye movement behaviors in complex eye movement trajectories. The hardware system used in this embodiment 2 is the same as that in embodiment 1, and the operation steps of this embodiment 2 are basically the same as those in embodiment 1.
[0281] The steps for eye movement behavior classification are as follows:
[0282] (1) Collecting the original eye movement data of the subjects
[0283] In this step (1), the host computer sets the visual stimulus to appear at different positions on the display screen in a random manner, see Fig.10 , Fig.10 The blue scattered dots in the figure represent the fixation points, and the red scattered dots represent that 9 visual stimuli are set to guide the subjects to perform multiple fixation eye movement behaviors.
[0284] Other operations are the same as in Example 1.
[0285] (2) Raw eye movement data preprocessing
[0286] See also Fig.11 , 12, Fig.12 The scattered points in the figure represent the trajectory of the original eye movement data, and the black scattered points represent the trajectory of the fixation point. Fig.12 The x-axis and y-axis represent the coordinates of the gaze point, which are expressed in normalized coordinates, and the z-axis represents the speed of the gaze point. The original eye movement data is preprocessed with spatial features, and the pixel coordinates of the gaze point and the speed of the gaze point are grouped as a whole using a clustering method. The optimal number of groups is determined by analyzing the elbow rule. For this embodiment, the optimal number of cluster centers k can be determined to be 10, and 9 groups of eye movement data are obtained, corresponding to Fig.11 The inflection point of k=10 is obtained, and 9 sets of eye movement data after preprocessing are obtained.
[0287] Other operations are the same as in Example 1.
[0288] (3) Establishment of Gaussian mixture model and hidden Markov model
[0289] In this embodiment 2, the operation process of step 3 is the same as that of embodiment 1, and the 9 groups of eye movement data after preprocessing are modeled to establish an optimized Gaussian mixture model and an optimized hidden Markov model.
[0290] (4) Classification of eye movement behavior
[0291] In this embodiment 2, based on the step Gaussian mixture model and the hidden Markov model, the 9 groups of eye movement data after preprocessing are classified into eye movement behaviors, and the current eye movement behaviors, such as fixation, saccade and tracking, are inferred according to the eye movement data at each time point.
[0292] See also Fig.13 , different eye movement behaviors are marked with different shapes on the original eye movement data, Fig.13 Medium blue dots represent saccadic behavior, green dots represent pursuit behavior, and orange dots represent fixation behavior.
[0293] refer to Fig.14 and Fig.15 , the result output after state recognition of 3 sets of eye movement data, the two figures represent the trajectory of x-axis and y-axis changing with time, Fig.14 and Fig.15 Different types of eye movement behaviors are marked with different colors.
[0294] Fig.14 The a in the figure represents the trajectory of the gaze point x-axis changing with time. The horizontal axis represents time in seconds, the left vertical axis represents the gaze point position in normalized coordinates, the right vertical axis represents the gaze point speed change curve, the blue curve represents the gaze point x-axis position indicator, and the red curve represents the gaze point speed. Fig.14 b in the figure is the classification result. The horizontal axis represents time in seconds, and the vertical axis represents the classification result. The blue dots represent gaze behavior, which is concentrated in the time period with low speed and little change in position index. The yellow dots represent tracking behavior, which appears in the time period with stable change in position index but certain speed. The red dots represent saccade behavior, which is concentrated in the time period with rapid change in speed. Fig.15 The a in the figure represents the trajectory of the gaze point y-axis changing with time. The horizontal axis represents time in seconds. The left vertical axis represents the gaze point position in normalized coordinates. The right vertical axis represents the gaze point speed change curve. The blue curve represents the gaze point y-axis position indicator, and the red curve represents the gaze point speed. Fig.15 The b in the figure is the classification result. The horizontal axis represents time in seconds, and the vertical axis represents the classification result. Similarly, the blue dots represent gaze behavior, the yellow dots represent tracking behavior, and the red dots represent saccade behavior.
[0295] Other operations are the same as in Example 1.
[0296] It is easy for those skilled in the art to understand that the above embodiments 1-2 are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An eye movement behavior classification method based on a Gaussian mixture model and a hidden Markov model, wherein the hardware system for implementing the eye movement behavior classification method comprises a desktop eye tracker, a host computer and a display screen; the host computer is a host computer, and the host computer has a Gaussian mixture model program and a hidden Markov model program; It is characterized in that the eye movement behavior classification operation steps are as follows: (1) Collecting the original eye movement data of the subjects (1.1) When the hardware system is working, a red dot with a diameter of 80 pixels is displayed on the display screen. The red dot is located at any position on the display screen, and the duration of each appearance is 500 to 1000 milliseconds; (1.2) The subject sits in front of a display screen and clicks a start button on the display screen with a mouse, and a visual stimulus of red dots appears on the display screen more than three times in a regular manner; the visual stimulus includes three types of eye movement behaviors to be identified, namely fixation, tracking, and saccade; (1.3) During the operation of the hardware system, the desktop eye tracker continuously collects the position coordinates and timestamp of the gaze point of the subject on the display screen, and transmits them to the host computer, which processes them to obtain raw eye movement data, wherein each piece of data in the raw eye movement data consists of the pixel coordinates of the gaze point and the speed of the gaze point; (2) Preprocessing of raw eye movement data Performing spatial feature preprocessing on the raw eye movement data, using a clustering method to group the pixel coordinates of the gaze point and the speed of the gaze point as a whole into at least 3 groups, each group of eye movement data having no less than 50 pieces, to obtain more than 3 groups of preprocessed eye movement data; (3) Establishing Gaussian mixture model and hidden Markov model (3.1) Establishing Gaussian mixture model (3.1.1) performing statistical analysis on the three or more pre-processed eye movement data, and based on the similarity in speed of the three or more pre-processed eye movement data, when the speed difference between any two data points is less than a preset threshold of 0.002, the two data points are considered to be similar and are classified into the same cluster; The three or more groups of eye movement data for statistical analysis are divided into three clusters according to the speed difference. Each cluster corresponds to a predefined eye movement behavior, namely, the fixation Gaussian distribution model, the pursuit Gaussian distribution model, and the saccade Gaussian distribution model. Initialize three predefined eye movement behaviors. For the gaze Gaussian distribution model, take the center position of the cluster with the lowest average speed as the center position of the gaze Gaussian distribution model, set the initial weight of the gaze Gaussian distribution to 1 / 3, and set the initial covariance matrix of the gaze Gaussian distribution to the identity matrix. For the saccade Gaussian distribution model, the center position of the cluster with the largest average speed is used as the center position of the saccade Gaussian distribution model. Similarly, the initial weight of the saccade Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the saccade Gaussian distribution is set to the unit matrix. For the tracking Gaussian distribution model, the center position of the remaining third cluster is used as the center position of the tracking Gaussian distribution model, the initial weight of the tracking Gaussian distribution is set to 1 / 3, and the initial covariance matrix of the tracking Gaussian distribution is set to the unit matrix; The initialization Gaussian mixture model is composed of the fixation Gaussian distribution model, the pursuit Gaussian distribution model and the saccade Gaussian distribution model; (3.1.2) adjusting the parameters of the initialized Gaussian mixture model by an iterative optimization method so that it better fits the actual probability distribution of each eye movement behavior, and setting the number of iterations to at least 50 times; and determining the degree of optimization of the Gaussian mixture model by introducing a Q function to obtain an optimized Gaussian mixture model; (3.2) Establishing Hidden Markov Model (3.2.1) Initialize the three parameters of the hidden Markov model: Initial probability matrix π=[π1,π2,…,π N ], represents the initial probability of each eye movement behavior at the initial moment, and the initial probability of each eye movement behavior is set to 1 / 3; The transition probability matrix A = [a ij ], represents the transition probability from one eye movement behavior to another, ensuring that the sum of each row element of the transition probability matrix A is 1; Emission probability matrix B = [b j (o t )], which represents the probability that each data in the eye movement data belongs to one of the three eye movement behaviors. Each row element of the emission probability matrix B is determined by the parameters of the Gaussian mixture model in (3.1.2); By initializing the above three parameters of the hidden Markov model, an initialized hidden Markov model is obtained; (3.2.2) optimizing the initialized hidden Markov model by using the preprocessed three or more sets of eye movement data through an optimization algorithm to obtain an optimized hidden Markov model; (4) Classification of eye movement behavior Based on the Gaussian mixture model optimized in step (3.1.2) and the hidden Markov model optimized in step (3.2.2), the eye movement behavior of the three or more preprocessed eye movement data is classified; based on the eye movement data at each time point, it is inferred whether the current eye movement behavior is fixation, saccade, or tracking eye movement behavior.
2. The eye movement behavior classification method according to claim 1, characterized in that: The desktop eye tracker is connected to the host computer via a universal serial bus; the display screen is larger than 13.3 inches and is located at least 75 centimeters away from the person being measured; The display screen is connected to the host computer via a universal serial bus.
3. The eye movement behavior classification method according to claim 1, characterized in that: The host computer connected to the desktop eye tracker and the display screen is equipped with a central processing unit with a basic frequency of more than 2.0 GHz, and the operating system is Windows 10 64-bit operating system.
4. The classification method according to claim 1, characterized in that: In step (1.2), the number of fixations of the visual stimulus of the red dot is 3 times, 5 times, or more than 10 times. The subject completes the fixation task within the set number of fixations, and the interval between two consecutive fixation stimuli is 0.5 seconds or 1 second. Within the range of the display screen, the host computer randomly generates the pixel coordinates of the fixation position.
5. The classification method according to claim 1, characterized in that: In step (1.2), the fixation refers to the eye movement behavior of staying in any area for at least 500 milliseconds, indicating that the person being tested is interested in the content of the area or is processing information; Tracking refers to the eye movement behavior of smoothly following a moving target, including tracking moving objects or text on a display screen; The eye movement refers to the eye movement behavior in which the line of sight quickly jumps from one fixation point to another. It is a fast and short eye movement that usually occurs between fixations and is used to shift visual attention from one area to another.
6. The classification method according to claim 1, characterized in that: In step (2), a clustering method is used to group the original eye movement data. The specific operation is as follows: (2.1) Clustering algorithm: a) Initialize cluster centers: Randomly select K data from the original eye movement data as the initial clustering centers, and divide the original eye movement data into K eye movement data groups, where the value of K is at least 4; b) Assign data to the nearest cluster center: Calculate the distance between each original eye movement data in the K eye movement data groups and the K cluster centers, assign the original eye movement data closest to a cluster center to the nearest eye movement data group, and obtain K new eye movement data groups; c) Update cluster centers: Calculate the data average of each new eye movement data group in the K new eye movement data groups, and average the data The mean is used as the new cluster center; d) Iterative processing: Repeat step b) and step c) until a predetermined number of iterations of 50 is reached; obtain K final cluster centers and K final eye movement data groups; (2.2) Elbow rule determines the optimal number of groups: Apply the elbow rule to determine the optimal number of groups. The specific steps are as follows: Calculate the sum of squared errors (Sum of Squared Errors) of the clustering results of the K final cluster centers. The sum of squared errors formula (1) is as follows: In formula (1), k is the number of cluster centers, n i is the number of data points in the ith group, is the jth data point in the i-th group, c i is the cluster center of the i-th group; When the number of groups k gradually increases, the error sum of squares SSE(k) usually shows a decreasing trend, because more groups can better fit the distribution of data and reduce the distance between the data point and the cluster center of the group to which it belongs. After a certain number of groups k, the decrease of the error sum of squares SSE(k) will slow down significantly and form an inflection point. The inflection point represents the optimal balance between clustering performance and the number of groups. Based on the trend that the error sum of squares of clustering results decreases as the number of groups k increases, the number of cluster centers k corresponding to the point where the error sum of squares SSE(k) decreases is selected to determine the optimal number of groups. For the k final cluster centers, the preprocessed k-1 groups of eye movement data are obtained.
7. The classification method according to claim 1, characterized in that: In step (3.1.2), the operation of adjusting the parameters of the Gaussian mixture model by the iterative optimization method is as follows: a). Initial parameter definition of Gaussian mixture model The formula (2) of the Gaussian mixture model is as follows: In formula (2), P(x) represents the probability of observing eye movement data x, k represents the kth eye movement behavior, which is fixation, tracking, and saccade, respectively, and c k represents the weight of the k-th eye movement behavior, N(x|μ k ,∑ k ) represents the probability density function of the kth eye movement Gaussian distribution, μ k and∑ k Respectively represent the mean and covariance matrix of the kth Gaussian distribution; Taking the three eye movement behaviors of fixation, tracking and saccade as examples, the probability density function (3) of the Gaussian mixture model is as follows: P(x)=c1·N(x|μ1,∑1)+c2·N(x|μ2,∑2)+c3·N(x|μ3,∑3)(3) In formula (3), the weights c1, c2, c3, the mean values μ1, μ2, μ3 and the covariance matrices ∑1, ∑2, ∑3 are all parameters to be optimized in the Gaussian mixture model; b). Calculation of responsibility To adjust the Gaussian mixture model parameters, it is necessary to calculate each eye movement data x n The probability of the three eye movement behaviors of fixation, tracking, and saccade is called the responsibility γ(z nk ), responsibility γ(z nk ) is calculated as follows: In formula (4), x n represents the nth observed eye movement data, z nk is a symbolic variable, indicating that the nth eye movement data belongs to the kth Gaussian distribution; c). Update Gaussian mixture model parameters Responsibility for useγ(z nk ) to update the parameters of the Gaussian mixture model, which include the weight c k , average value μ k and the covariance matrix ∑ k ; Weight c k The update formula (5) is as follows: Average value μ k The update formula (6) is as follows: Covariance matrix∑ k The update formula (7) is as follows: Repeat the calculation of formula (5)-(7) at least 50 times to update the parameters of the Gaussian mixture model; In order to determine the degree of optimization of the Gaussian mixture model, the Q function is introduced. The Q function formula (8) is as follows: In formula (8), θ represents the Gaussian mixture model parameters in the current iteration, including the mixing coefficients c of each Gaussian distribution k , mean μ k and the covariance matrix ∑ k ,θ (t) represents the Gaussian mixture model parameters in the previous iteration, P(x i ,z ik |θ) Under the current model parameters θ, the eye movement data x i The probability of belonging to the kth Gaussian distribution, P(z ik |x i ,θ (t) ) indicates that the model parameter θ in the previous iteration (t) Next, eye movement data x i The probability of belonging to the kth Gaussian distribution; The Q function measures the degree of optimization of the model by comparing the parameters of the current Gaussian mixture model with the parameters of the Gaussian mixture model of the previous iteration. The larger the value of the Q function, the greater the improvement of the current model parameters relative to the previous iteration. When the value of the Q function is less than the set threshold of 0.001, or the number of calculations reaches the preset number of repetitions, the parameter optimization of the Gaussian mixture model is completed, and the parameters of the Gaussian mixture model at this time are recorded, including the weight c k , average value μ k and the covariance matrix ∑ k , that is, the optimized Gaussian mixture model is obtained.
8. The classification method according to claim 1, characterized in that: In step (3.2.2), the method for optimizing the parameters of the hidden Markov model is performed as follows: a). Define and calculate forward probability The observation state sequence of eye movement behavior is defined as O = (o1, o2, ..., o T ), the above sequence represents the eye movement behavior observed from the initial moment to the last moment. In the process of hidden Markov model optimization, the forward probability α is first calculated. t (i), the forward probability is calculated as follows: Formula (9) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o1, o2, …, o t ), in formula (9), a ji represents the transition probability from state j at time t-1 to state i at time t, b i represents the emission probability of state i, a ji and b i are all parameters of the hidden Markov model; b) Define and calculate backward probability Calculate the backward probability β t (i), the calculation formula (10) of the backward probability is as follows: Formula (10) indicates that the state at time t is i, and the observation sequence of eye movement behavior is (o t+1 ,o t+1 ,…,o t ), in formula (10), a ij represents the transition probability from state j at time t+1 to state i at time t, b j represents the emission probability of state j, a ij and b j are all parameters of the hidden Markov model; c). Update the parameters of the hidden Markov model Using the forward probability and the backward probability, calculate the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the parameters of the hidden Markov model, where the prior probability γ t (i) represents the probability of state i at time t, and the joint probability ξ t (i, j) represents the probability that the state is i at time t and the state is j at time t+1, and the prior probability γ t The calculation formula (11) of (i) is as follows: Joint probability ξ t The calculation formula (12) of (i, j) is as follows: In formulas (11) and (12), the transition probability a ij represents the probability of transitioning from state i to state j, and the emission probability b j (o t+1 ) means that the observed value in state j is o t+1 The probability of ; by using the prior probability γ t (i) and the joint probability ξ t (i, j) is used to update the three parameters of the initial probability matrix π, the transition probability matrix A and the emission probability matrix B in the hidden Markov model. The specific update operation is as follows: The update formula (13) of the initial state probability vector π is as follows: p i =γ1(i) (13) In formula (13), π i Represents the i-th element of the optimized initial state probability vector; The update formula (14) of the state transition probability matrix A is as follows: In formula (14), a ij Represents the element of the i-th row and j-th column of the optimized state transition probability matrix; The update formula (15) of the emission probability matrix B is as follows: In formula (15), b j (k) represents the jth row and kth column element of the optimized emission probability matrix, v k represents the kth possible observation value; by repeating formula (13)-(15) for at least 100 times, the three parameters of the above hidden Markov model are optimized, so that the hidden Markov model continuously fits the eye movement data. When the preset number of repetitions is reached, the hidden Markov model is considered to have completed the optimization, and finally the optimized hidden Markov model is obtained.
9. The classification method according to claim 1, characterized in that: In step (4), the eye movement behavior classification is performed on the preprocessed three or more groups of eye movement data to obtain the eye movement behavior at each moment. The specific operation is as follows: (4.1) Determine the optimal path probability at each moment Assume that the hidden state sequence of eye movement behavior is S = (s1, s2, ..., s T ), the hidden state of the eye movement behavior represents the actual eye movement behavior, which is one of the eye movement behaviors of fixation, tracking and saccade, and the sequence S represents all the actual eye movement behaviors from the initial moment to the final moment; Assume that the observed state sequence of eye movement behavior is O = (o1, o2, ..., o T ), the observed state of the eye movement behavior represents the eye movement behavior estimated based on the observed eye movement data, which is also a kind of eye movement behavior among fixation, tracking and saccade, and the sequence O represents all the eye movement behaviors estimated based on the observed eye movement data from the initial moment to the final moment; In order to obtain the hidden state sequence with the highest probability, that is, to obtain the most likely eye movement behavior, the optimal sequence function is defined as δ t (i), the optimal sequence function δ t (i) indicates that when the hidden state is i at time t, the observed state is (o1, o2, ..., o t ), the maximum probability of the optimal sequence function δ t The specific value of (i) is obtained by recursive calculation, and the optimal sequence function δ t The recursive formula (16) of (i) is as follows: In formula (16), a ij represents the transition probability from state i to state j, b jot Indicates that o is observed in state j t The emission probability, δ t-1 (i) represents the probability of state i at time t-1; By recursively formula (16), the maximum probability δ corresponding to each hidden state i is calculated at each time t t (i); (4.2) Determine the optimal precursor state at each moment Get the maximum probability δ of the hidden state at time t t (i) Based on this, we need to record the most likely previous state of hidden state i at time t, that is, the optimal predecessor state at time t; define the optimal predecessor state ψ t (i), record the most likely previous eye movement behavior of state i at time t, the optimal precursor state ψ t Formula (17) of (i) is as follows: Formula (17) records the optimal precursor state at each time t, and recursively finds the hidden state from the final time to the initial time, thereby classifying the eye movement behavior at each time point; (4.3) Classification of eye movement behavior Using the optimal path probability δ t (i) and the optimal precursor state ψ t (i), calculate the maximum probability P of the hidden state sequence * and the terminal state of the hidden state sequence Classify eye movement behavior; The maximum probability of the hidden state sequence is P * Indicates that at the final moment T, the optimal path probability δ corresponding to the three hidden states of fixation, pursuit and saccade is calculated respectively t (i) and select the optimal path probability value with the largest value as the maximum probability P of the hidden state sequence * , the maximum probability P * The calculation formula (18) is as follows: In formula (18), N represents the number of hidden states, and the upper limit here is 3; The maximum probability P of the hidden state sequence at the final moment T is obtained * On this basis, it is also necessary to determine the terminal state of the hidden state sequence Termination Status The calculation formula (19) is as follows: When the termination status is obtained , we start to recurse from the final moment T to the initial moment to obtain the hidden state at each moment; the recursive formula (20) is as follows: In formula (20), represents the optimal hidden state at time t, represents the optimal hidden state at time t+1 The most likely previous state corresponding to the hidden state; determine the hidden state of the entire hidden state sequence at each moment through formula (20), that is, classify the eye movement behavior from the initial moment to the final moment; infer whether the current eye movement behavior belongs to fixation, saccade or tracking eye movement behavior, and complete the eye movement behavior classification; The three types of eye movement behaviors are specifically classified as follows: Attention: When the probability δ corresponding to the hidden state t (i) When it exceeds 0.9 and the state lasts for more than 200 milliseconds, it indicates that the current eye movement behavior is fixation; Eye saccade: When the hidden state changes within 50 milliseconds, and the corresponding transition probability a ij If it is greater than 0.8, it indicates that the current eye movement behavior is saccade; Tracking: When the hidden state changes within 500 milliseconds, and the corresponding transition probability a ij If it is greater than 0.9, it indicates that the current eye movement behavior is tracking.
Citation Information
Patent Citations
Method for recognizing human eye state based on built-in type hidden Markov model
CN101299234A
Eye movement analysis using copolymerized Hidden Markov Model (copolymerized EMMM) and Switching Hidden Markov Model (EMSHMM)
CN115175602A
Telemetering type eye movement behavior classification method and device for visual field defect crowd
CN116665287A
Eye movement analysis with co-clustering of hidden markov models (emhmm with co-clustering) and with switching hidden markov models (emshmm)
US20220361747A1
System and method for calibrating electrooculography signals based on head movement
US20240206799A1
Cited By
Motor insulation breakdown voltage prediction method based on continuous hidden Markov model
CN119622276A
Skeletal muscle spasm-to-contracture evolution rule quantitative analysis method and system and application
CN121237384A
