A fatigue measurement method and system based on eye movement data

Abstract

A fatigue measurement method and system based on eye movement data, the method comprising the following steps: using an eye tracker to respectively collect eye movement data of a subject performing a task in an awake state and a fatigued state, and calculating eye movement features based on the eye movement data; the eye movement features include the left eye pupil diameter change rate, the right eye pupil diameter change rate, the fixation duration, the average scanning speed, and the eyelid closure change rate; establishing a fatigue measurement machine learning model, training the fatigue measurement machine learning model based on the eye movement features, and obtaining a trained fatigue measurement machine learning model; obtaining eye movement data of a user to be measured, calculating eye movement features based on the eye movement data, and inputting the eye movement features into the trained fatigue measurement machine learning model to obtain a fatigue measurement result of the user to be identified.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue measurement, and in particular to a fatigue measurement method and system based on eye movement data. Background Art

[0002] At present, there are mainly the following ways to quantify subjective fatigue: questionnaire method, task completion behavior-based method, and physiological signal-based method. The questionnaire method mainly uses a self-subjective questionnaire or the Karolinska Sleepiness Scale to let users actively score. This type of method is not very reliable and is too affected by the user's subjective psychology. Task completion behavior-based method refers to the classification of fatigue based on the user's operating performance. This method is greatly affected by the environment and task difficulty. Fatigue detection methods based on physiological signals such as electroencephalogram, electrocardiogram, and electromyogram have complex data processing and high requirements for equipment.

[0003] Because the current method relies on questionnaires, users may be affected by personal psychological cues and user habitual biases when answering, and the data obtained may deviate from the user's true state. Methods that rely on task completion behavior may be affected by the difficulty of the task and the user's proficiency. Methods based on electroencephalograms and electromyograms require the user to be connected to many complex devices throughout the body, which affects the user's normal task execution state and is not universally applicable. Summary of the invention

[0004] In view of the above analysis, the embodiments of the present invention aim to provide a fatigue measurement method and system based on eye movement data, so as to solve the problems of inaccurate fatigue measurement and complicated measurement process in the prior art.

[0005] On the one hand, an embodiment of the present invention provides a fatigue measurement method based on eye movement data, comprising the following steps:

[0006] An eye tracker is used to collect eye movement data of the subjects performing tasks in an awake state and a fatigued state, and eye movement features are calculated based on the eye movement data; the eye movement features include a change rate of left eye pupil diameter, a change rate of right eye pupil diameter, fixation duration, average scanning speed, and a change rate of eyelid closure;

[0007] Establishing a fatigue measurement machine learning model, training the fatigue measurement machine learning model based on the eye movement features, and obtaining a trained fatigue measurement machine learning model;

[0008] Obtain eye movement data of the user to be measured, calculate eye movement features based on the eye movement data, and input the eye movement features into a trained fatigue measurement machine learning model to obtain a fatigue measurement result of the user to be identified.

[0009] Based on the further improvement of the above method, the eye movement data includes: pupil diameter, eye movement type and gaze coordinates; the eye movement type includes gaze, saccade and blink;

[0010] Calculating eye movement features based on the eye movement data includes:

[0011] According to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and has no task;

[0012] According to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration; according to the formula Calculate the rate of change of eyelid closure, where f represents the eyelid closure when the subject performs the task, and f0 represents the average eyelid closure when the subject is awake and has no task;

[0013] The average saccade velocity was calculated from the fixation coordinates.

[0014] Based on the further improvement of the above method, based on the mapping regression relationship between the eyelid closure degree and the fixation duration, the blink duration and the saccade duration, the eyelid closure degree is calculated, including calculating the eyelid closure degree according to the following formula:

[0015]

[0016] interval=blink+fixation+saccade+CLOS

[0017] Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

[0018] Based on the further improvement of the above method,

[0019] Calculating the average glance speed based on gaze coordinates includes:

[0020] The angle between the gaze coordinates of two adjacent sampling moments in unit time is calculated according to the following formula:

[0021]

[0022] (x i-1 ,y i-1 ,z i-1 ) represents the gaze coordinate at sampling time i-1, (xi ,y i ,z i ) represents the gaze coordinate at sampling time i;

[0023] The scanning speed at adjacent sampling moments is calculated according to the following formula:

[0024] V i =α i ×freq

[0025] Among them, freq represents the sampling frequency;

[0026] The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

[0027] Based on the further improvement of the above method, the blink coefficient is obtained by the following steps:

[0028] The image recognition method is used to obtain the eye aspect ratio when the eyes are closed and open, and the threshold of the eye aspect ratio for judging eyelid closure is calculated according to T=I1+(I2-I1)×(1-β); where I1 and I2 are the average values ​​of the eye aspect ratio when the eyes are closed and open, respectively, and β represents the proportion of eye closure;

[0029] The eyelid closure degree is obtained according to the time duration during which the eye aspect ratio is less than a threshold value in a unit time, and the eyelid closure degree is subjected to correlation analysis with the fixation duration, the blink duration and the scan duration to obtain the blink coefficient.

[0030] Based on the further improvement of the above method, a fatigue measurement machine learning model is constructed by using support vector machine; a radial basis kernel function is used as the kernel function, and a grid search method is used to determine the penalty coefficient of the radial basis kernel function and the optimal variable combination of the kernel variables.

[0031] On the other hand, an embodiment of the present invention provides a fatigue measurement system based on eye movement data, including the following modules:

[0032] A feature extraction module, for collecting eye movement data of the subject performing the task in the awake state and the fatigue state using an eye tracker, and calculating eye movement features based on the eye movement data; the eye movement features include the change rate of the left eye pupil diameter, the change rate of the right eye pupil diameter, the fixation duration, the average scanning speed and the change rate of the eyelid closure degree;

[0033] A model training module, used to establish a fatigue measurement machine learning model, train the fatigue measurement machine learning model based on the eye movement features, and obtain a trained fatigue measurement machine learning model;

[0034] The fatigue measurement module is used to obtain eye movement data of the user to be measured, calculate eye movement features based on the eye movement data, and input the eye movement features into a trained fatigue measurement machine learning model to obtain fatigue measurement results of the user to be identified.

[0035] Furthermore, the eye movement data includes: pupil diameter, eye movement type and gaze coordinates; the eye movement type includes gaze, saccade and blink;

[0036] The data acquisition module calculates the eye movement features based on the eye movement data, including:

[0037] According to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and has no task;

[0038] According to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration; according to the formula Calculate the rate of change of eyelid closure, where f represents the eyelid closure when the subject performs the task, and f0 represents the average eyelid closure when the subject is awake and has no task;

[0039] The average saccade velocity was calculated from the fixation coordinates.

[0040] Further, based on the mapping regression relationship between the eyelid closure degree and the fixation duration, the blink duration and the saccade duration, calculating the eyelid closure degree includes calculating the eyelid closure degree according to the following formula:

[0041]

[0042] interval=blink+fixation+saccade+CLOS

[0043] Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

[0044] Further,

[0045] Calculating the average glance speed based on gaze coordinates includes:

[0046] The angle between the gaze coordinates of two adjacent sampling moments in unit time is calculated according to the following formula:

[0047]

[0048] (x i-1 ,y i-1 ,z i-1 ) represents the gaze coordinate at sampling time i-1, (x i ,y i ,z i ) represents the gaze coordinate at sampling time i;

[0049] The scanning speed at adjacent sampling moments is calculated according to the following formula:

[0050] V i =α i ×freq

[0051] Among them, freq represents the sampling frequency;

[0052] The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

[0053] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0054] 1. By using an eye tracker to collect eye movement data and measuring the user's fatigue based on the eye movement data, the equipment is simple and easy to implement. It will not affect the user's normal task execution status due to the equipment, and the measurement results will not be affected by the user's subjective feelings, task difficulty and environmental factors, thereby making the measurement results more accurate.

[0055] 2. Using the left eye pupil diameter change rate, right eye pupil diameter change rate, fixation duration, scanning speed and eyelid closure change rate as features to train the fatigue measurement machine learning model can eliminate personal physiological factors and differences in eye usage habits, making the model more accurate.

[0056] 3. Based on the mapping regression relationship, the eyelid closure degree is directly calculated according to the eye movement data, thereby avoiding the problem of low calculation efficiency caused by complex data processing of the video image to obtain the value of the eyelid closure degree, thereby simplifying the processing process and improving the calculation efficiency.

[0057] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0059] Figure 1 This is a flow chart of a fatigue measurement method based on eye movement data according to an embodiment of the present invention;

[0060] Figure 2 A block diagram of a fatigue measurement system based on eye movement data according to an embodiment of the present invention;

[0061] Figure 3 Schematic diagram of eye feature points according to an embodiment of the present invention;

[0062] Figure 4 This is a statistical diagram of EAR data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0064] At present, there are mainly the following ways to quantify subjective fatigue: questionnaire method, task completion behavior-based method, and physiological signal-based method. The questionnaire method mainly uses a self-subjective questionnaire or the Karolinska Sleepiness Scale to let users actively score. This type of method is not very reliable and is too affected by the user's subjective psychology. Task completion behavior-based method refers to the classification of fatigue based on the user's operating performance. This method is greatly affected by the environment and task difficulty. Fatigue detection methods based on physiological signals such as electroencephalogram, electrocardiogram, and electromyogram have complex data processing and high requirements for equipment.

[0065] A specific embodiment of the present invention provides a fatigue measurement method based on eye movement data, such as Figure 1 As shown, the following steps are included:

[0066] S1. Using an eye tracker to collect eye movement data of the subjects performing tasks in an awake state and a fatigued state, respectively, and calculating eye movement features based on the eye movement data; the eye movement features include the change rate of the left eye pupil diameter, the change rate of the right eye pupil diameter, the fixation duration, the average scanning speed and the eyelid closure degree;

[0067] S2. Establish a fatigue measurement machine learning model, and train the fatigue measurement machine learning model based on the eye movement features to obtain a trained fatigue measurement machine learning model;

[0068] S3. Obtain eye movement data of the user to be measured, calculate eye movement features based on the eye movement data, and input the eye movement features into a trained fatigue measurement machine learning model to obtain a fatigue measurement result of the user to be identified.

[0069] During implementation, a wearable eye tracker is used, such as tobii's wearable eye tracker. The wearable eye tracker is easy to wear and does not affect the normal task execution state of the subject, thereby making the measurement results more accurate. The measurement process is simple and easy to implement.

[0070] By using an eye tracker to test eye movement data in different states, extracting eye movement features to train a fatigue measurement machine learning model, and inputting the eye movement features of the user to be measured into the trained fatigue measurement machine learning model, the measurement results can be obtained. Fatigue measurement is performed through objective eye movement data to avoid inaccurate results caused by the influence of user subjective psychology, environment and task difficulty.

[0071] Specifically, the eye movement data collected by the wearable eye tracker include: pupil diameter, eye movement type and gaze coordinates; among which, the eye movement types collected by the eye tracker include gaze, saccade, blink and unclassified.

[0072] The fixation duration is the fixation duration collected by the eye tracker per unit time. As the fatigue level increases, the fixation duration will significantly increase, so it is used as a measurement feature.

[0073] By performing variance homogeneity test and one-way ANOVA test on 11 subjects and 832,306 pupil diameter data, the test results are shown in Tables 1 and 2. Pupil diameter is affected by individual differences, and its significance P = 0.000 < 0.05, that is, the pupil diameters of different individuals are significantly different. There are 11 subjects in total, that is, they are divided into 11 groups, and their pupil diameter data are significantly different. If pupil diameter is directly used as an eye movement feature, fatigue recognition cannot be effectively performed.

[0074]

[0075]

[0076] Table 1 Variance homogeneity check of pupil diameter data

[0077]

[0078] Table 2 ANOVA verification of pupil diameter data

[0079] By analyzing the pupil diameter data, it was found that there was a significant difference in the rate of change of pupil diameter when the subjects were performing tasks in different states of fatigue and wakefulness compared with the pupil diameter when there was no task in the awake state.

[0080] Specifically, according to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and task-free.

[0081] Some data are shown in Table 3, which is a sample of 86,944 left and right pupil change rate data in the awake and fatigue states of a selected subject. The collected data are marked as awake or fatigued, with the awake state marked as 0 and the fatigue state marked as 1. After using SPSS software for descriptive statistics, it can be observed that the average value of the pupil diameter change rate in the fatigue state is greater than that in the awake state. Further analysis of variance and one-way ANOVA analysis were then performed. By observing the significance value of the SPSS analysis results, as shown in Tables 4 and 5, it can be concluded that its significance is P = 0.000 < 0.05, which is significant. That is, there is a significant difference in the pupil diameter change rate between awake and fatigue states, and this indicator can be used as a feature vector for fatigue measurement machine model training.

[0082]

[0083] Table 3 Pupil change rate data

[0084]

[0085] Table 4 Variance homogeneity test of pupil change rate data

[0086]

[0087] Table 5 ANOVA verification of pupil change rate data

[0088] There is a significant difference in the eye's scanning speed when in a fatigued state and an awake state, so it is used as an eye movement feature. Specifically, the average scanning speed is calculated based on the gaze coordinates, including:

[0089] Calculating the average glance speed based on gaze coordinates includes:

[0090] The angle between the gaze coordinates of two adjacent sampling moments in unit time is calculated according to the following formula:

[0091]

[0092] (x i-1 ,y i-1 ,z i-1 ) represents the gaze coordinate at sampling time i-1, (x i ,y i ,z i ) represents the gaze coordinate at sampling time i;

[0093] The scanning speed at adjacent sampling moments is calculated according to the following formula:

[0094] V i =α i ×freq

[0095] Among them, freq represents the sampling frequency;

[0096] The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

[0097] For example, if the unit time is 1s and the sampling frequency is 100Hz, there are 100 sampling moments in the unit time, and 100 sampling data are generated.

[0098] Therefore, according to the sampling data of two adjacent sampling moments, the scanning speed of two adjacent sampling moments is calculated according to the above formula, and the average of all scanning speeds is taken to obtain the average scanning speed per unit time.

[0099] Eyelid closure (perclos) is also a commonly used indicator for measuring fatigue. It uses the ratio of eye closure time to a specific time to determine the degree of fatigue, where closure refers to the state of the pupil being covered by the eyelid. The perclos method has three criteria: P70, P80, and EM, which represent the ratio of the time when the pupil is longitudinally covered by the eyelid for 70%, the ratio of the time when it is covered for 80%, and the mean square closure rate of the eyelid. Studies have shown that P80 has the best correlation with fatigue level.

[0100] Currently, the eyelid closure degree is usually calculated by image recognition. Since a series of complex data processing is required for the video image to calculate the eyelid closure degree, the processing process is complicated and the calculation efficiency is low. The embodiment of the present invention uses eye movement data to calculate the eyelid closure degree based on the mapping relationship between the eye movement data and the eyelid closure degree, thereby simplifying the processing process and improving the calculation efficiency.

[0101] Specifically, according to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration.

[0102] Specifically, the eyelid closure degree is calculated using the following formula:

[0103]

[0104] interval=blink+fixation+saccade+CLOS

[0105] Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

[0106] During implementation, the blink coefficient in the above formula is obtained in advance through the following steps:

[0107] S11. Using an image recognition method based on eye video data, the eye aspect ratio when the eyes are closed and open, that is, the EAR value, is obtained.

[0108] During implementation, an eye tracker was used to collect eye video data and eye movement data of 11 subjects at the same time.

[0109] like Figure 3 As shown, based on face feature recognition, eye feature data is obtained, and then 6 feature points of the eye are extracted. Calculate the eye aspect ratio. A total of 123,172 EAR data were collected. p1-p6 represent Figure 3 The coordinates of the 6 feature points P1-P6.

[0110] S12. Calculate the threshold of the eye aspect ratio for judging eyelid closure according to T=I1+(I2-I1)×(1-β); where I1 and I2 are the average values ​​of the eye aspect ratios when the eyes are closed and open, respectively, and β represents the proportion of eye closure. During implementation, the P80 judgment standard is adopted, so β ​​is taken as 0.8.

[0111] During implementation, the EAR data of the operator with eyes open and eyes closed are obtained through experiments. Figure 4 shown.

[0112] As can be seen from the figure, the peak value of the EAR distribution when the eyes are closed is at [0.13, 0.18], and the peak value of the EAR distribution when the eyes are open is at [0.28, 0.38]. The middle value of these two intervals is selected as the value of I1 and I2.

[0113] Right now

[0114]

[0115]

[0116] T=0.155+(0.33-0.155)*(1-0.8)=0.19

[0117] That is, the threshold value of EAR is T=0.19. When the EAR value is less than 0.19, it is considered that the eyelid is closed.

[0118] S13. Obtain the eyelid closure degree according to the duration during which the eye aspect ratio is less than a threshold value per unit time, and perform correlation analysis on the eyelid closure degree with the fixation duration, blink duration, and scan duration to obtain a blink coefficient.

[0119] The fixation duration, blink duration and saccade duration per unit time are calculated based on the collected user's eye movement data, and the relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration is analyzed to obtain the blink coefficient. The specific relationship analysis method is prior art and will not be described in detail here.

[0120] After obtaining the blink coefficient, the eyelid closure degree can be calculated based on the eye movement data measured by the eye tracker. No video image processing is required. The method is simple and can improve measurement efficiency.

[0121] PERCLOS is the percentage of eyelid closure time per unit time. However, due to personal physiological factors and differences in eye habits, the perclos value of each person performing the same task may be different. For example, some people habitually blink when they are not tired, and some people are used to squinting to observe. Therefore, the perclos value of the current subjects was subjected to a variance homogeneity test and a one-way ANOVA test, as shown in Tables 6 and 7.

[0122]

[0123] Table 6 Perclos test for homogeneity of variance

[0124]

[0125] Table 7 Perclos ANOVA test

[0126] After performing variance analysis on 758 perclos values ​​of 11 subjects, it was found that the significance P=0.003<0.05, and the perclos values ​​of different people were significantly different. Therefore, this application incorporates individual differences into the consideration range of indicator processing and decides to select the perclos individual change rate instead of the original perclos absolute value indicator.

[0127] The formula for this rate of change is as follows:

[0128]

[0129] Among them: f is the eyelid closure degree of the subject when performing the task, f0 is the average eyelid closure degree of the subject in the awake and task-free state, that is, the baseline value.

[0130] By analyzing the eye movement data, the left eye pupil diameter change rate, right eye pupil diameter change rate, fixation duration, average scanning speed and eyelid closure change rate were selected as eye movement feature data for training the fatigue measurement machine learning model.

[0131] Specifically, a support vector machine is used to construct a fatigue measurement machine learning model; a radial basis kernel function is used as a kernel function, and a grid search method is used to determine the penalty coefficient of the radial basis kernel function and the optimal variable combination of the kernel variables.

[0132] Support vector machine is a supervised machine learning algorithm, which is often used for binary classification problems. It uses a technique called kernel function to map data into a high-dimensional space. For example, radial basis kernel function is used as the kernel function, and grid search method is used to determine the penalty coefficient of the radial basis kernel function and the optimal variable combination with the kernel variable.

[0133] The data sample number is represented by i, and the characteristic data of the i-th sample, namely the change rate of the left pupil diameter, the change rate of the right pupil diameter, the fixation duration, the average scanning speed and the change rate of the eyelid closure degree are represented by xi. The fatigue state is represented by y, where y i =0 means in the awake state, y i =1 represents fatigue state. i ,y i ) constitutes a training set, and a support vector machine model is trained based on the training set to obtain a fatigue measurement model. The specific process of training the support vector machine is the existing technology and will not be repeated here.

[0134] For the user to be measured, after obtaining his / her eye movement data, the eye movement features are obtained according to the method in step S1, and the eye movement features are input into the trained fatigue measurement model to obtain the measurement results.

[0135] A specific embodiment of the present invention provides a fatigue measurement system based on eye movement data, such as Figure 2 As shown, it includes the following modules:

[0136] A feature extraction module, for collecting eye movement data of the subject performing the task in the awake state and the fatigue state using an eye tracker, and calculating eye movement features based on the eye movement data; the eye movement features include the change rate of the left eye pupil diameter, the change rate of the right eye pupil diameter, the fixation duration, the average scanning speed and the change rate of the eyelid closure degree;

[0137] A model training module, used to establish a fatigue measurement machine learning model, train the fatigue measurement machine learning model based on the eye movement features, and obtain a trained fatigue measurement machine learning model;

[0138] The fatigue measurement module is used to obtain eye movement data of the user to be measured, calculate eye movement features based on the eye movement data, and input the eye movement features into a trained fatigue measurement machine learning model to obtain fatigue measurement results of the user to be identified.

[0139] Preferably, the eye movement data includes: pupil diameter, eye movement type and gaze coordinates; the eye movement type includes gaze, saccade and blink;

[0140] The data acquisition module calculates the eye movement features based on the eye movement data, including:

[0141] According to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and has no task;

[0142] According to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration; according to the formula Calculate the rate of change of eyelid closure, where f represents the eyelid closure when the subject performs the task, and f0 represents the average eyelid closure when the subject is awake and has no task;

[0143] The average saccade velocity was calculated from the fixation coordinates.

[0144] Preferably, based on the mapping regression relationship between the eyelid closure degree and the fixation duration, the blink duration and the glance duration, calculating the eyelid closure degree includes calculating the eyelid closure degree according to the following formula:

[0145]

[0146] interval=blink+fixation+saccade+CLOS

[0147] Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

[0148] Preferably, calculating the scanning speed according to the gaze coordinates comprises:

[0149] The angle between the gaze coordinates of two adjacent moments is calculated according to the following formula:

[0150]

[0151] (x i-1 ,yi-1 ,z i-1 ) represents the gaze coordinate at time i-1, (x i ,y i ,z i ) represents the gaze coordinate at time i;

[0152] The scanning speed at adjacent sampling moments is calculated according to the following formula:

[0153] V i =α i ×freq

[0154] Among them, freq represents the sampling frequency;

[0155] The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

[0156] The above method embodiments and system embodiments are based on the same principle, and their related parts can be used for reference, and can achieve the same technical effect. The specific implementation process refers to the above embodiment, which will not be repeated here.

[0157] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0158] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A fatigue measurement method based on eye movement data, characterized in that: The following steps are involved: An eye tracker is used to collect eye movement data of the subjects performing tasks in an awake state and a fatigued state, and eye movement features are calculated based on the eye movement data; the eye movement features include a change rate of left eye pupil diameter, a change rate of right eye pupil diameter, fixation duration, average scanning speed, and a change rate of eyelid closure; Establishing a fatigue measurement machine learning model, training the fatigue measurement machine learning model based on the eye movement features, and obtaining a trained fatigue measurement machine learning model; Obtaining eye movement data of the user to be measured, calculating eye movement features based on the eye movement data, and inputting the eye movement features into a trained fatigue measurement machine learning model to obtain a fatigue measurement result of the user to be identified; The eye movement data includes: pupil diameter, eye movement type and gaze coordinates; the eye movement type includes gaze, saccade and blink; Calculating eye movement features based on the eye movement data includes: According to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and has no task; According to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration; according to the formula Calculate the rate of change of eyelid closure, where f represents the eyelid closure when the subject performs the task, and f0 represents the average eyelid closure when the subject is awake and has no task; The average saccade velocity was calculated from the fixation coordinates; Based on the mapping regression relationship between eyelid closure and fixation duration, blink duration and saccade duration, calculating eyelid closure includes: calculating eyelid closure according to the following formula: interval=blink+fixation+saccade+CLOS Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

2. The fatigue measurement method based on eye movement data according to claim 1, characterized in that: Calculating the average glance speed based on gaze coordinates includes: The angle between the gaze coordinates of two adjacent sampling moments in unit time is calculated according to the following formula: (x i-1 ,y i-1 ,z i-1 ) represents the gaze coordinate at sampling time i-1, (x i ,y i ,z i ) represents the gaze coordinate at sampling time i; The scanning speed at adjacent sampling moments is calculated according to the following formula: V i =a i ×freq Among them, freq represents the sampling frequency; The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

3. The fatigue measurement method based on eye movement data according to claim 1, characterized in that: The blink coefficient is obtained by the following steps: The image recognition method is used to obtain the eye aspect ratio when the eyes are closed and open, and the threshold of the eye aspect ratio for judging eyelid closure is calculated according to T=I1+(I2-I1)×(1-β); where I1 and I2 are the average values ​​of the eye aspect ratio when the eyes are closed and open, respectively, and β represents the closed eye area index; The eyelid closure degree is obtained according to the duration during which the eye aspect ratio is less than a threshold value within a unit time, and the eyelid closure degree is correlated with the fixation duration, the blink duration and the saccade duration to obtain the blink coefficient.

4. The fatigue measurement method based on eye movement data according to claim 1, characterized in that: A support vector machine is used to construct a fatigue measurement machine learning model; a radial basis kernel function is used as a kernel function, and a grid search method is used to determine the penalty coefficient of the radial basis kernel function and the optimal variable combination of the kernel variables.

5. A fatigue measurement system based on eye movement data, characterized in that: Includes the following modules: A feature extraction module, for collecting eye movement data of the subject performing the task in the awake state and the fatigue state using an eye tracker, and calculating eye movement features based on the eye movement data; the eye movement features include the change rate of the left eye pupil diameter, the change rate of the right eye pupil diameter, the fixation duration, the average scanning speed and the change rate of the eyelid closure degree; A model training module, used to establish a fatigue measurement machine learning model, train the fatigue measurement machine learning model based on the eye movement features, and obtain a trained fatigue measurement machine learning model; A fatigue measurement module is used to obtain eye movement data of the user to be measured, calculate eye movement features based on the eye movement data, and input the eye movement features into a trained fatigue measurement machine learning model to obtain a fatigue measurement result of the user to be identified; The eye movement data includes: pupil diameter, eye movement type and gaze coordinates; the eye movement type includes gaze, saccade and blink; The feature extraction module calculates the eye movement features based on the eye movement data, including: According to the pupil diameter data, based on the formula The pupil diameter change rate was calculated, where P represents the average pupil diameter per unit time when the subject performs the task, and P0 represents the average pupil diameter per unit time when the subject is awake and has no task; According to the eye movement type data, the fixation duration, blink duration and saccade duration per unit time are calculated, and the eyelid closure degree is calculated based on the mapping regression relationship between the eyelid closure degree and the fixation duration, blink duration and saccade duration; according to the formula Calculate the rate of change of eyelid closure, where f represents the eyelid closure when the subject performs the task, and f0 represents the average eyelid closure when the subject is awake and has no task; The saccade velocity was calculated based on the fixation coordinates; Based on the mapping regression relationship between eyelid closure and fixation duration, blink duration and saccade duration, calculating the eyelid closure includes calculating the eyelid closure according to the following formula: interval=blink+fixation+saccade+CLOS Among them, γ represents the blink coefficient, blink represents the blink duration per unit time, fixation represents the fixation duration per unit time, saccade represents the saccade duration per unit time, and CLOS represents the time without data from the eye tracker per unit time.

6. The fatigue measurement system based on eye movement data according to claim 5, characterized in that: Calculating the scan velocity based on gaze coordinates includes: The angle between the gaze coordinates of two adjacent moments is calculated according to the following formula: (x i-1 ,y i-1 ,z i-1 ) represents the gaze coordinate at time i-1, (x i ,y i ,z i ) represents the gaze coordinate at time i; The scanning speed at adjacent sampling moments is calculated according to the following formula: V i =a i ×freq Among them, freq represents the sampling frequency; The average scanning speed is obtained by averaging the scanning speeds of all adjacent sampling moments within a unit time.

Images