Fatigue detection method and device

Abstract

The invention belongs to the technical field of fatigue detection, and discloses a fatigue detection method and device. Entering a training scene, and calculating to obtain a reference score corresponding to the individual; circularly training to a fatigue state, and resting; entering the test scene after rest, continuously executing the detection task until the current round score reaches the fatigue relieving standard, recording the first fatigue relieving time point, continuously executing the detection task until the current round score is lower than the fatigue relieving standard, and recording the last fatigue relieving time point. According to the invention, the whole process of a subject from a normal state, a fatigue state, a fatigue relieving state to a re-fatigue state can be evaluated, and the subject is helped to clearly understand fatigue change. Meanwhile, the system records and analyzes task scores and fatigue change data, and evaluates the fatigue recovery effect and the duration thereof. The scoring standard provided by the invention considers individual differences, and ensures the objectivity and accuracy of the evaluation result.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue detection, and in particular to a fatigue detection method and device. Background Art

[0002] Fatigue detection technology aims to monitor and evaluate the fatigue status of individuals through various methods and equipment. These technologies can be applied to driver monitoring, industrial safety, medical health and other fields. Common fatigue detection methods include: 1. Biosignal detection such as electroencephalogram, electrocardiogram, skin galvanic response, etc.; 2. Behavioral detection such as eye tracking, facial expression analysis, etc.; 3. Subjective evaluation such as questionnaires, self-reports, etc.; 4. Operational performance monitoring, detecting the performance of individuals in specific roles such as reaction time, error rate, etc.

[0003] An eye tracker is a device used to monitor and record eye movements. It is widely used in healthcare, psychology research, marketing, user experience design and other fields. Its basic principles include video eye tracking and photoelectric reflection method. Video eye tracking uses an infrared light source and a camera to capture eye images, and calculates the position and movement trajectory of the eye through image processing algorithms; the photoelectric reflection method uses infrared light to illuminate the eye and determines the direction of the eye by detecting the position of the reflected light. The main parameters of the eye tracker include fixation point, saccade and blink frequency, which refer to the time and position of the eyes staying at a certain point, rapid eye movements, and the number and interval of blinks.

[0004] The fatigue detection method described in the Chinese patent "CN106073805B A fatigue detection method and device based on eye movement data" is: collect the eye movement data of the subject when interacting with the UI through an eye tracker to calculate the feature data, and then judge whether the subject is in a fatigue state by combining the sensitivity of the reaction and the eye movement fatigue feature data with a machine learning algorithm, wherein the UI used consists of a character search test and a reading comprehension test. However, this method does not allow the subject to interact directly using an eye tracker, but requires interaction through other devices such as a mouse and keyboard. The eye tracker is only used to collect the subject's eye movement data when taking the test, which leads to differences in the subject's proficiency in operating the device, which may lead to deviations in the final results.

[0005] The Chinese patent "CN118236066A A system and method for assessing and training attention based on eye movement capture and tracking" provides a method for assessing attention by combining an eye tracker with a game task, which allows users to directly train through an eye tracker. By providing targeted training tasks, personalized assessment and feedback can be achieved to improve the user's concentration and endurance. However, the patent does not assess the user's fatigue, and is mainly used for attention assessment and attention training.

[0006] Existing patents basically do not have an evaluation method for the fatigue and fatigue recovery of the subject. Most of them only conduct attention assessment or single fatigue detection, and cannot perform detection in multiple states such as normal state-fatigue state-fatigue recovery state-fatigue state again, and cannot meet some needs that require checking the complete fatigue process. In addition, the current fatigue detection method usually does not adjust the score calculation algorithm according to the individual physiological condition of the subject, which may cause individual differences to be ignored, resulting in inaccurate evaluation results and failure to truly reflect the fatigue of the subject. Summary of the invention

[0007] The purpose of the present invention is to provide a fatigue detection method that can evaluate the whole process of a subject from a normal state, a fatigue state, a fatigue relief state to a fatigue state again. Through this method, the subject can clearly understand his own fatigue situation and changes.

[0008] The present invention designs a detection task and system based on the proposed fatigue detection method. The system enables the subject to complete all detection tasks only through an eye tracker, and records and detects the task score data in the entire process. The implementation process of the fatigue detection method of the present invention is simple, and accurate fatigue detection results can also be obtained.

[0009] The technical solution of the present invention is as follows: A fatigue detection method comprises the following steps:

[0010] Step 1. Enter the training scene and calculate the benchmark score corresponding to the individual;

[0011] Step 2. Cycle training until fatigue, then rest;

[0012] Step 3. After a break, enter the test scene and continue to perform the detection task until the current round score reaches the fatigue relief standard, record the first fatigue relief time point, continue to perform the detection task until the current round score is lower than the fatigue relief standard, and record the last fatigue relief time point.

[0013] In the training scenario, the system presents the task screen according to the set parameters and task type. The subject selects the target in the scene through the eye tracker, and the system automatically scores according to a round of 30 seconds, with a full score of 100 points.

[0014] The benchmark score S0 is the average of the highest three scores in the first ten minutes of the training scene.

[0015] The cyclic training is a cycle of 30 seconds of normal training and 30 seconds of staring at the screen. If the score is lower than 70% of the baseline score, it is regarded as entering a fatigue state, and the score of this round is recorded as S1. By using this round score as the first score T1 of the test scene, the system automatically records the fatigue time time1 at this time.

[0016] The score of the test scenario is recorded as Tx, and Tx / T1 is calculated and recorded; the time point when the fatigue relief standard is first reached is recorded as the effective time time_start;

[0017] Continue the test, record the last fatigue relief standard time point reached as the single effect duration time_end, and exit the test scene.

[0018] The training scenario and the test scenario both include two detection tasks: a rapid visual task and a multi-target visual task; the rapid visual task is to randomly generate a fast-moving circular target from the four sides of the interface used for visual fatigue tasks at regular intervals; the multi-target visual task is to randomly generate multiple circular targets from the four sides of the interface at regular intervals; the subject uses an eye tracker to look at the screen, and when the eye position coincides with the moving circular target, it is determined that the target is eliminated, and a score is awarded after the target is eliminated, and the faster the elimination speed, the higher the score; when the moving target moves to the boundary of the interface, the subject's eye movement line of sight does not always look at the target, and the target will be destroyed.

[0019] The scoring score for a single target is calculated as follows:

[0020] Step 1) Record the creation time when the target is created: creationTime;

[0021] Step 2), when the target is gazed, record the gaze duration: clickTime = current time - creationTime;

[0022] Step 3) When the target leaves the implementation range, it will be destroyed, and the destruction time is recorded: destroyTime = current time - creationTime;

[0023] Step 4) Calculate the normalized time of the fixation time relative to the total time the target exists:

[0024]

[0025] Step 5), when the benchmark score S0, i.e., baseScore, has been generated, jump to step (7), otherwise, go to step (6);

[0026] Step 6) Calculate the score: score = max(0,100-normalizedTime×100), get the score, and the calculation ends;

[0027] Step 7) Calculate the benchmark coefficient middle:

[0028]

[0029] Step 8) Calculate the deviation: deviation = normalizedTime-middle

[0030] Step 9), when deviation < 0, that is, the fixation time is earlier than the reference value, jump to step 6), otherwise jump to step 10);

[0031] Step 10) Calculate the adjustment score. scoreSensitivity is the adjustable score sensitivity. The higher the sensitivity, the more sensitive the score changes, and the easier it is for changes in fatigue to be reflected in the score:

[0032] adjustedScore=baseScore-deviation×200×scoreSensitivity

[0033] Step 11), calculate the score: if adjustedScore>100, score=100; if adjustedScore<0, score=0; otherwise, score=adjustedScore, and the calculation ends;

[0034] The score of each target is recorded in the system, and the average of all targets in this round is calculated each time the score of this round is calculated to obtain the score of this round.

[0035] A fatigue detection device, comprising: an eye tracker, a display, a data processing module, and an input device;

[0036] The eye tracker is used to interact with the virtual scene and complete the task.

[0037] The display is used to display the system interface and the task virtual scene, and to simulate the training scene and the test scene;

[0038] The data processing module is used to calculate the score and obtain the current status of the individual;

[0039] The data processing module records a cycle of 30 seconds of normal training and 30 seconds of only staring at the screen. If the score is lower than 70% of the baseline score, it is regarded as entering a fatigue state, and the score of this round is recorded as S1. By using this round score as the first score T1 of the test scene, the system automatically records the fatigue time time1 at this time.

[0040] Based on the experimental data of multiple subjects, the data processing module designed a cyclic task mode that alternates between 30 seconds of normal training and 30 seconds of static screen staring. This mode can effectively simulate real situations and accurately capture the fatigue state of the subjects. At the same time, the scoring calculation method dynamically adjusts the individual baseline score to make the scoring more sensitive and able to reflect fatigue changes in real time.

[0041] The fatigue detection method of the present invention can be flexibly adjusted according to specific analysis requirements, such as fatigue standards, rest time and detection task parameters, so as to adapt to different application scenarios.

[0042] Beneficial effects of the present invention: The present invention can evaluate the whole process of the subject from the normal state, fatigue state, fatigue relief state to the re-fatigue state, and help the subject clearly understand the fatigue changes. At the same time, the system records and analyzes the task scores and fatigue change data to evaluate the fatigue recovery effect and its duration. The scoring criteria proposed by the present invention take individual differences into consideration to ensure the objectivity and accuracy of the evaluation results.

[0043] The system designed by the present invention is easy to operate, and the test subject can complete the test without complicated operations, thereby reducing labor costs. The eye tracker interacts with the virtual scene, making the detection task closer to actual work, thereby improving the authenticity and reliability of the test. The starting position, moving speed and target color of the circular moving target of the present invention can be adjusted according to actual conditions, and the invention has strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of fatigue detection method.

[0045] Figure 2 It is a line chart of the scores of the training scenes.

[0046] Figure 3 A line chart showing the scores of the test scenarios. DETAILED DESCRIPTION

[0047] The fatigue detection process proposed in the present invention covers the whole process evaluation of the subject from normal state, fatigue state, fatigue relief state to fatigue state again.

[0048] The scoring criteria of the present invention can intelligently adjust parameters, fully consider individual differences, and ensure the objectivity of the results. The system can not only accurately detect the time process from normal to fatigue of the subject, but also evaluate the fatigue recovery effect and its duration by comparing with the baseline data.

[0049] The fatigue detection system designed by the present invention is easy to operate and allows the subject to complete the entire test process independently, thereby effectively reducing labor costs. The system uses an eye tracker for interaction, making the operation in the virtual scene closer to the real working environment, reducing the subject's dependence on the proficiency of equipment operation.

[0050] The fatigue detection device includes: an eye tracker, a display, a host, and an input device. The eye tracker is used to interact with the virtual scene and complete the task, the display is used to display the system interface and the task virtual scene, the host is used to run the system, and the input device is used for UI interaction.

[0051] This method sets two scenarios for fatigue detection tasks: training scenario and test scenario. Considering that some errors may be caused by the subjects' unfamiliarity with the experimental tasks, the present invention formulates corresponding tasks in the training scenario to familiarize the subjects with the training scenario of the fatigue test. During this process, the subjects will gradually enter a fatigue state; at the same time, considering the physiological differences between different subjects, this method will also calculate the subject's baseline score in the training scenario, and the subsequent score calculation will make corresponding adjustments based on this baseline score to better adapt to the physiological differences between different subjects.

[0052] The training scenario is used to familiarize the subjects with the task, assess the subjects' baseline scores, and evaluate the first fatigue. The test scenario is used to evaluate the changes in fatigue after the subjects rest. Specifically, the subjects continuously complete the test tasks in the training scenario until they are tired, and then enter the test scenario after resting to continue completing the test tasks, and experience fatigue relief and fatigue again during this stage.

[0053] There are two types of detection tasks used in this method: rapid visual tasks and multi-target visual tasks. The rapid visual task is to randomly generate a fast-moving circular target from the four sides of the interface used for visual fatigue tasks at regular intervals; the multi-target visual task is to randomly generate multiple circular targets from the four sides of the interface at regular intervals. The subject uses an eye tracker to look at the screen. When the eye movement position coincides with the moving circular target, it is determined that the target is eliminated. After eliminating the target, points can be scored. The faster the elimination speed, the higher the score. If the moving target moves to the boundary of the interface, the subject's eye movement line of sight has never been focused on the target, the target will be destroyed.

[0054] The following is the whole process of fatigue detection performed by a volunteer according to the process of the present invention.

[0055] like Figure 2As shown, the score changes in the training scenario are manifested as the subject's score gradually increasing from a low value and reaching a high level at about the 10th round. During the experiment, most subjects also showed a similar trend, which means that the subject's proficiency is gradually improving. Therefore, the present invention selects the average of the three highest scores in the first 10 rounds as the benchmark score. Although the subsequent scores fluctuated, they never fell below the set fatigue standard and finally reached the fatigue standard in the 54th round.

[0056] like Figure 3 As shown in the figure, the score changes in the test scenario showed that the subjects basically returned to the baseline score in the 5th round and entered the fatigue relief state. Subsequently, there was a significant fluctuation around the 20th round, and the fatigue state was reached again in the 34th round.

[0057] In summary, the entire test process effectively captured the fatigue state of the subject. The subject became fatigued at the 54th minute after the start of the test, and after a 3-minute rest, entered the fatigue relief state at the 59th minute, but reached the fatigue state again at the 90th minute. Preliminary analysis showed that for this subject, the 3-minute rest was not enough to fully relieve fatigue, resulting in a faster entry into the fatigue state during the test phase.

Claims

1. A fatigue detection method, characterized in that: The steps include: Step 1. Enter the training scene and calculate the benchmark score corresponding to the individual; Step 2. Cycle training until fatigue, then rest; Step 3. After a break, enter the test scene and continue to perform the detection task until the current round score reaches the fatigue relief standard, record the first fatigue relief time point, continue to perform the detection task until the current round score is lower than the fatigue relief standard, and record the last fatigue relief time point.

2. The fatigue detection method according to claim 1, characterized in that: In the training scenario, the system presents the task screen according to the set parameters and task type. The subject selects the target in the scene through the eye tracker, and the system automatically scores according to a round of 30 seconds, with a full score of 100 points.

3. The fatigue detection method according to claim 1, characterized in that: The benchmark score S0 is the average of the highest three scores in the first ten minutes of the training scene.

4. The fatigue detection method according to claim 3, characterized in that: The cyclic training is a cycle of 30 seconds of normal training and 30 seconds of staring at the screen. If the score is lower than 70% of the baseline score, it is regarded as entering a fatigue state, and the score of this round is recorded as S1. By using this round score as the first score T1 of the test scene, the system automatically records the fatigue time time1 at this time.

5. The fatigue detection method according to claim 4, characterized in that: The score of the test scenario is recorded as Tx, and Tx / T1 is calculated and recorded; the time point when the fatigue relief standard is first reached is recorded as the effective time time_start; Continue the test, record the last fatigue relief standard time point reached as the single effect duration time_end, and exit the test scene.

6. The fatigue detection method according to any one of claims 1 to 5, characterized in that: The training scenario and the test scenario both include two detection tasks: a rapid visual task and a multi-target visual task; the rapid visual task is to randomly generate a fast-moving circular target from the four sides of the interface used for visual fatigue tasks at regular intervals; the multi-target visual task is to randomly generate multiple circular targets from the four sides of the interface at regular intervals; the subject uses an eye tracker to look at the screen, and when the eye position coincides with the moving circular target, it is determined that the target is eliminated, and a score is awarded after the target is eliminated, and the faster the elimination speed, the higher the score; when the moving target moves to the boundary of the interface, the subject's eye movement line of sight does not always look at the target, and the target will be destroyed.

7. The fatigue detection method according to claim 6, characterized in that: The scoring score for a single target is calculated as follows: Step 1) Record the creation time when the target is created: creationTime; Step 2), when the target is gazed, record the gaze duration: clickTime = current time - creationTime; Step 3) When the target leaves the implementation range, it will be destroyed, and the destruction time is recorded: destroyTime = current time - creationTime; Step 4) Calculate the normalized time of the fixation time relative to the total time the target exists: Step 5), when the benchmark score S0, i.e., baseScore, has been generated, jump to step 7), otherwise step 6); Step 6) Calculate the score: score = max(0,100-normalizedTime×100), get the score, and the calculation ends; Step 7) Calculate the benchmark coefficient middle: Step 8) Calculate the deviation: deviation = normalizedTime-middle Step 9), when deviation < 0, that is, the fixation time is earlier than the reference value, jump to step 6), otherwise jump to step 10); Step 10) Calculate the adjustment score. scoreSensitivity is the adjustable score sensitivity. The higher the sensitivity, the more sensitive the score changes, and the easier it is for fatigue changes to be reflected in the score: adjustedScore=baseScore-deviation×200×scoreSensitivity Step 11), calculate the score: if adjustedScore>100, score=100; if adjustedScore<0, score=0; Otherwise, score = adjustedScore, and the calculation ends; The score of each target is recorded in the system, and the average of all targets in this round is calculated each time the score of this round is calculated to obtain the score of this round.

8. A fatigue detection device, characterized in that: include: Eye tracker, display, data processing module, input device; The eye tracker is used to interact with the virtual scene and complete the task. The display is used to display the system interface and the task virtual scene, and to simulate the training scene and the test scene; The data processing module is used to calculate the score and obtain the current status of the individual; The data processing module records a cycle of 30 seconds of normal training and 30 seconds of only staring at the screen. If the score is lower than 70% of the baseline score, it is regarded as entering a fatigue state, and the score of this round is recorded as S1. By using this round score as the first score T1 of the test scene, the system automatically records the fatigue time time1 at this time.

Images