Intelligent visual fatigue detection method, system and device

Abstract

The invention discloses an intelligent asthenopia detection method, system and device, and relates to the field of asthenopia detection.The method comprises the steps that feature extraction is conducted on electroencephalogram information and eye movement information to obtain an electroencephalogram feature set and an eye movement feature set; respectively inputting the electroencephalogram feature set and the eye movement feature set into a plurality of preset asthenopia detection models to obtain a plurality of asthenopia detection results; determining a final asthenopia detection result according to the plurality of asthenopia detection results; wherein different preset asthenopia detection models correspond to different machine learning models, and each preset asthenopia detection model is obtained by training the machine learning model according to the asthenopia detection sample set. According to the invention, real-time, efficient, objective and accurate asthenopia detection can be realized.

Description

Technical Field

[0001] The present application relates to the field of visual fatigue detection, and in particular to an intelligent visual fatigue detection method, system and device. Background Art

[0002] Visual fatigue is a common health problem for modern people, especially when using electronic devices for extended periods. Existing methods for detecting visual fatigue include subjective assessment and physiological signal monitoring. The physiological signal monitoring method can monitor eye movements and electroencephalograms (EEGs), but it still has the following shortcomings:

[0003] (1) They usually rely on a single EEG signal or eye movement signal, ignoring the fusion analysis of multi-dimensional physiological data, resulting in insufficient accuracy and reliability of monitoring results.

[0004] (2) The implementation of fatigue detection requires subjective feedback, such as visual fatigue questionnaires, which are easily interfered with by factors such as individual emotions and psychological state, reducing the stability of the test results.

[0005] (3) The real-time performance is poor, and the data before and after the task need to be compared, which results in large errors. Summary of the Invention

[0006] The purpose of this application is to provide an intelligent visual fatigue detection method, system and device, which can realize real-time, efficient, objective and accurate visual fatigue detection.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides an intelligent method for detecting visual fatigue, comprising:

[0009] Obtain target user's EEG and eye movement information in real time;

[0010] performing feature extraction on the electroencephalogram information and the eye movement information respectively to obtain an electroencephalogram feature set and an eye movement feature set;

[0011] Inputting the EEG feature set and the eye movement feature set into a plurality of preset visual fatigue detection models respectively to obtain a plurality of visual fatigue detection results;

[0012] Determine a final visual fatigue detection result based on multiple visual fatigue detection results;

[0013] Among them, different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set.

[0014] In a second aspect, the present application provides an intelligent visual fatigue detection system, comprising:

[0015] Information acquisition module, used to: obtain the target user's EEG information and eye movement information in real time;

[0016] A feature extraction module is used to: extract features from the EEG information and the eye movement information respectively to obtain an EEG feature set and an eye movement feature set;

[0017] an asthenopia detection module, configured to input the electroencephalogram feature set and the eye movement feature set into a plurality of preset asthenopia detection models, respectively, to obtain a plurality of asthenopia detection results;

[0018] A detection result determination module is used to determine a final visual fatigue detection result based on multiple visual fatigue detection results;

[0019] Among them, different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set.

[0020] In a third aspect, the present application provides an intelligent visual fatigue detection device, including an eye movement information collection component, an electroencephalogram information collection component, a controller, and a display feedback component;

[0021] The eye movement information collection component, the electroencephalogram information collection component, and the display feedback component are all connected to the controller; the controller is used to: execute a computer program to implement an intelligent visual fatigue detection method based on the eye movement information collected by the eye movement information collection component and the electroencephalogram information collected by the electroencephalogram information collection component, and obtain a final visual fatigue detection result;

[0022] The display feedback component is used to display the final visual fatigue detection result to the target user.

[0023] According to the specific embodiments provided by the present application, the present application has the following technical effects: the present application provides an intelligent visual fatigue detection method, system and device, which uses an electroencephalogram feature set and an eye movement feature set to detect visual fatigue, takes into account multi-dimensional physiological data, and can improve the accuracy of visual fatigue detection. The present application sets up multiple preset visual fatigue detection models, and different preset visual fatigue detection models correspond to different machine learning models. Each preset visual fatigue detection model is obtained by training the machine learning model based on the visual fatigue detection sample set. Multiple visual fatigue detection results can be obtained through multiple preset visual fatigue detection models. The present application realizes visual fatigue detection through the above-mentioned model, and does not rely on complex pre- and post-task change values. Instead, it detects and outputs visual fatigue results through real-time physiological signal analysis, which has higher objectivity, stability and timeliness. In summary, the present application can realize real-time, efficient, objective and accurate visual fatigue detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a diagram of the application environment of an intelligent visual fatigue detection method in one embodiment of the present application.

[0026] Figure 2 A flowchart of an intelligent visual fatigue detection method provided in one embodiment of the present application.

[0027] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] By combining the spatiotemporal characteristics of the EEG and eye movement data, this application can comprehensively analyze the cognitive state and eye fatigue of the target user, simplify the visual fatigue detection method, and thus provide a more accurate and real-time visual fatigue assessment.

[0030] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0031] The intelligent visual fatigue detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the target user's EEG information and eye movement information to the server 104. After receiving the information, the server 104 performs feature extraction to obtain an EEG feature set and an eye movement feature set, and then inputs them into multiple preset visual fatigue detection models to obtain multiple visual fatigue detection results, and determines the final visual fatigue detection result based on the multiple visual fatigue detection results. The server 104 can feed back the final visual fatigue detection result to the terminal 102. In addition, in some embodiments, the intelligent visual fatigue detection method can also be implemented by the server 104 or the terminal 102 alone.

[0032] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. In one specific application, when the terminal 102 is a portable wearable device, the portable wearable device can be an integration of smart glasses and an electroencephalogram device. In another specific application, when the terminal 102 is a laptop, the smart glasses and the electroencephalogram device collect eye movement feature sets and electroencephalogram feature sets, and then transmit them to the laptop as the basis for subsequent data processing. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers, or it can be a cloud server.

[0033] In an exemplary embodiment, Figure 2 As shown, an intelligent visual fatigue detection method is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, including the following steps 201 to 204.

[0034] Step 201: Obtain the target user's EEG information and eye movement information in real time.

[0035] In visual fatigue detection, EEG signals can serve as an important indicator for assessing cognitive load and fatigue levels. EEG characteristics (such as changes in theta and alpha waves) are often used to detect individual fatigue. Research has shown that when a wearer is fatigued, theta wave power may increase. For example, after prolonged screen time or visual tasks, brain fatigue levels increase, and theta wave activity increases, reflecting a more tired and inefficient brain state. Just as a person's EEG has a higher number of theta waves when they are sleepy, changes in theta wave component are a key indicator for visual fatigue detection. Alpha waves are associated with the brain's states of relaxation and alertness. During visual fatigue, the power of alpha waves may decrease. Normally, alpha waves are more pronounced when a person is quiet, relaxed, and with their eyes closed. However, during visual tasks, as fatigue sets in, alpha waves weaken, indicating decreased brain alertness and visual processing ability, a sign of visual fatigue. Therefore, the EEG information includes Theta wave information and Alpha wave information. The frequency range of Theta wave is generally 4-8 Hz, and the frequency range of Alpha wave is generally 8-13 Hz.

[0036] The eye movement information includes blinking frequency (the number of blinks per minute), eye movement status (including horizontal and vertical movement of the eyeball, gaze point, gaze time, etc.), eyelid status, pupil diameter (changes in pupil size), eye saccade information (the number and speed of rapid eye movement) and gaze information (duration and distribution of gaze points). Specifically, an increase in blinking frequency usually indicates eye fatigue, because the eyes achieve lubrication and rest through frequent blinking; a decrease in blinking frequency may indicate a high degree of concentration or tension, but if it is reduced for a long time, it may also cause dry eyes and fatigue. The eye movement status can be characterized by the horizontal and vertical movement of the eyeball. When the horizontal and vertical movement of the eyeball is frequent and irregular, it indicates eye fatigue and difficulty in maintaining stable gaze. Frequent changes in gaze point indicate fatigue and difficulty in concentrating; shortened gaze time indicates eye fatigue and inability to stare at one point for a long time. Eyelid status such as incomplete closure, eyelid muscle fatigue, and orbicularis oculi muscle weakness may lead to insufficient blinking, indicating eye fatigue. The size of the pupil diameter reflects visual fatigue, which is mainly manifested in accommodation spasm. For example, continuous close-range use of the eyes causes ciliary muscle spasm, which makes the pupil shrink (accommodative miosis) and unstable pupil size, which are all manifestations of visual fatigue.

[0037] After completing step 201 and before executing step 202, the method further includes performing a data preprocessing operation, specifically including the following steps (1) to (2).

[0038] (1) The EEG information is subjected to a first filtering process and a denoising process in sequence. The acquisition process of EEG information is often interfered with by various noises, such as myoelectric interference, eye movement interference, etc. In order to ensure the accuracy of the signal, a series of filtering and denoising operations are required. The filtering used includes one of the following:

[0039] Bandpass filter: EEG information is mainly concentrated in the frequency band of 0.5Hz to 50Hz, so bandpass filters are often used to retain the signal in this range while removing high-frequency noise (such as power line noise) and low-frequency interference.

[0040] Notch filter: Used to remove 50Hz or 60Hz power line interference. In this frequency band, a notch filter with a narrow passband is used to suppress noise.

[0041] Lowpass filter: Used to remove high-frequency noise from EEG information, especially electrical noise above 50Hz. Lowpass filters are usually set with a low cutoff frequency (such as 50Hz).

[0042] (2) The eye movement information is subjected to a second filtering process and a denoising process in sequence. Eye movement information usually contains information such as pupil movement and blink frequency, but is also susceptible to interference from various noises (such as equipment noise, motion artifacts, etc.). Therefore, filtering and denoising of eye movement information are also very important. The filtering used includes one of the following:

[0043] Bandpass filtering: Similar to EEG information processing, eye movement information is usually filtered using a bandpass filter to filter out frequency bands below 1 Hz (such as baseline drift) and above 50 Hz (such as high-frequency noise).

[0044] Moving Average Filter: By locally averaging the eye movement information, it can smooth and remove noise, which is especially helpful for removing instantaneous interference.

[0045] The denoising process is a combination of wavelet denoising and independent component analysis. Wavelet denoising involves using wavelet transform to denoise the signal to be processed. By selecting an appropriate wavelet basis and denoising threshold, high-frequency noise can be effectively removed while preserving the main features of the signal.

[0046] Independent component analysis is to separate independent components from the signal to be processed, which can effectively remove noise interference.

[0047] In addition, in terms of denoising eye movement information, you can also use: mean denoising, which can reduce the impact of random noise on the signal by averaging and filtering the eye movement signal, retaining the main change trend of the signal; Kalman filter denoising (KalmanFilter), for continuous eye movement information, the Kalman filter can model the noise of the information and filter it to further improve the signal quality.

[0048] In step 202, feature extraction is performed on the EEG information and the eye movement information to obtain an EEG feature set and an eye movement feature set. In a specific application, step 202 includes the following steps (1) to (3) to extract more important features, thereby providing key feature data for subsequent intelligent detection.

[0049] (1) Performing initial feature extraction on the EEG information to obtain a plurality of initial EEG features. Specifically, the EEG information can reveal changes in the brain under different fatigue states by analyzing activities in different frequency bands. In practical applications, first, a bandpass filter is used to extract brain waves in a specific frequency band. For example, Alpha waves can be extracted by setting a bandpass filter of 813 Hz, and the calculation formula is as follows:

[0050]

[0051] Among them, f c is the center frequency, f is the signal frequency of the EEG, j is the imaginary unit, and H(f) is the extracted Alpha wave, which serves as an initial EEG feature.

[0052] Next, the extracted H(f) is converted from the time domain to the frequency domain using a Fast Fourier Transform (FFT) to analyze its power spectral density. The extracted Alpha wave is then subjected to a time-frequency analysis using a wavelet transform to extract signal characteristics at different time scales, which is particularly suitable for analyzing non-stationary signals.

[0053] Through the above processing, or in other words, including but not limited to the above processing methods, multiple EEG features including the power spectral density of the alpha wave, the peak-to-trough amplitude difference of the alpha wave, and the standard deviation of the alpha wave are obtained. Similarly, multiple EEG features including the power spectral density of the theta wave, the peak-to-trough amplitude difference of the theta wave, and the standard deviation of the theta wave can be obtained.

[0054] (2) Performing initial feature extraction on the eye movement information to obtain multiple initial eye movement features; specifically, changes in blink frequency and pupil diameter can reflect the individual's fatigue state. The blink frequency is extracted by analyzing blink events in the eye movement signal; changes in blink frequency can reflect the individual's fatigue level. Specifically, time domain or frequency domain methods can be used, such as analyzing the spectrum of the blink signal through FFT. Changes in pupil diameter reflect the individual's visual load and attention changes, and relevant features can be extracted by tracking changes in pupil diameter.

[0055] In practical applications, the initial eye movement feature extraction can generally be achieved by calculating the mean, maximum, minimum and change rate of the pupil diameter, and calculating the fluctuation amplitude of the signal.

[0056] (3) L1 regularization or tree-based feature importance analysis is used to perform feature screening on multiple initial EEG features and multiple initial eye movement features to obtain corresponding EEG feature sets and eye movement feature sets.

[0057] In practical applications, after completing steps (1) and (2), before executing step (3), further screening and post-processing are required to improve data quality, remove noise, and ensure that the extracted features can accurately reflect the fatigue state. Based on this, the initial EEG features and initial eye movement features obtained are sequentially subjected to denoising processing (independent component analysis can be used to remove eye movement and electromyographic noise, and wavelet transform can be used to denoise the signal and filter out high-frequency noise), and outlier detection and removal processing.

[0058] Among them, outlier detection and removal processing includes: using the Zscore method to detect and remove outliers. For each signal feature, its Zscore is calculated. If the Zscore exceeds the set threshold (such as 3), the value is considered an outlier and needs to be removed. The calculation formula of Zscore is:

[0059]

[0060] Among them, x i is the i-th signal feature, μ is the data mean, and σ is the standard deviation.

[0061] The EEG feature set includes the power spectral density of theta wave, the peak-to-trough amplitude difference of theta wave, the standard deviation of theta wave, the power spectral density of the alpha wave, the peak-to-trough amplitude difference of the alpha wave, and the standard deviation of the alpha wave. Among them, the power spectral density of theta wave and the alpha wave are key features for assessing the state of visual fatigue, and they can reflect the activity state of the cerebral cortex. For example, as the degree of fatigue increases, the power spectral density of theta wave may increase, while the power spectral density of the alpha wave may decrease. The difference between the peaks and troughs of the theta wave and the alpha wave in the time series is calculated, that is, the peak-to-trough amplitude difference, and this difference can reflect the intensity change of the brain wave signal. The standard deviation of the brain wave signal is calculated to measure the fluctuation of the waveform in the time series. The lower the stability, the higher the degree of fatigue.

[0062] The eye movement feature set includes eye movement trajectory features, eye movement speed, blink amplitude and blink frequency. Among them, the blink frequency is obtained by counting the number of blinks per unit time. The blink frequency may increase or decrease abnormally when fatigued. The degree of eyelid closure during each blink is measured to obtain the blink amplitude. The blink amplitude may change irregularly when fatigued. The ratio of the distance the eye moves in a certain time interval to the time is calculated to obtain the eye movement speed. The eye movement speed may slow down when fatigued. The average moving distance, maximum moving distance, and distribution of moving directions in the horizontal and vertical directions are determined to obtain the eye movement trajectory features. The eye movement trajectory features will become unstable when fatigued. During the fatigue assessment process, the above-mentioned multiple EEG features and multiple eye movement features can accurately reflect the individual's fatigue level.

[0063] In a practical application, after obtaining the above-mentioned EEG feature set and eye movement feature set, the extracted features are normalized to conform to the standard normal distribution to enhance the robustness of the model. After completing this process, the subsequent step 203 is executed.

[0064] Step 203, respectively input the EEG feature set and the eye movement feature set into a plurality of preset visual fatigue detection models to obtain a plurality of visual fatigue detection results; wherein, different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set, so that it can automatically learn the complex mapping relationship between the features and the visual fatigue state, thereby establishing a corresponding model to evaluate the wearer's visual fatigue state in real time. During the training process, the algorithm will automatically adjust the parameters to maximize the accuracy and generalization ability of the scoring. The machine learning models corresponding to the preset visual fatigue detection models include support vector machine models, linear regression models, etc.

[0065] In regression analysis, the relationship between a dependent variable and one or more independent variables is studied. In this invention, the dependent variable is the visual fatigue detection result, while the independent variables are the extracted EEG feature set and eye movement feature set. By establishing a regression model, the quantitative relationship between these independent variables and the dependent variable can be found. Using this model, the corresponding visual fatigue detection result can be predicted based on the input feature values.

[0066] Support vector machine is a supervised learning method used for classification and regression analysis. It performs classification by mapping data into a high-dimensional space and constructing an optimal hyperplane therein. For this application, the EEG feature set and the eye movement feature set can be used as input, and the visual fatigue detection results can be used as output categories (such as mild, moderate, severe fatigue, etc.). By training the support vector machine model, it can accurately classify different fatigue states according to the feature data. When processing nonlinear separable data, the support vector machine can use kernel functions (such as Gaussian kernel, polynomial kernel, etc.) to implicitly map the data into a high-dimensional space, thereby achieving effective classification.

[0067] In a specific practical application, model training includes the following steps:

[0068] (1) Data collection and preprocessing.

[0069] First, a large amount of EEG and eye movement information is collected, including data under different fatigue states. This data can be collected by recruiting volunteers through experiments and wearing EEG sensors and eye trackers during different tasks (such as long-term reading, watching videos, performing visual tasks, etc.). In addition, it is necessary to collect annotated information related to fatigue status, such as using questionnaires or objective physiological indicators to determine the degree of fatigue of volunteers at the time of data collection.

[0070] Secondly, the collected data is cleaned to remove invalid and abnormal data. For example, if some data points have abnormally high peaks or abnormally low values, it may be caused by equipment failure or external interference. It needs to be processed through methods such as median filtering and mean replacement or directly eliminated.

[0071] Then, step 202 described above is used to extract key features from the cleaned raw data, such as the power spectra of theta and alpha waves, blink frequency, eye movement amplitude, and other information mentioned above.

[0072] (2) Model training and optimization.

[0073] First, divide the data obtained in the previous step into training and test sets, with the training set typically accounting for 70%-80% and the test set for 20%-30%. During model training, the training set is used to construct model parameters, while the test set is used to evaluate model performance and optimize the trained model. Cross-validation can also be used during training to optimize the model. For example, K-fold cross-validation divides the dataset into K subsets, alternating between using K-1 subsets as the training set and the remaining subset as the validation set. This allows for more efficient data utilization while reducing the risk of overfitting the model.

[0074] Secondly, select an appropriate machine learning model based on the nature of the problem. In this technical solution, a linear regression model and a support vector machine model are used. These models are trained and their performance (such as mean squared error, accuracy, etc.) on the training set is optimized by adjusting the model parameters. For example, for a support vector machine, it is necessary to select an appropriate kernel function and adjust the penalty parameter and kernel function parameters.

[0075] Then, use the test set to evaluate the trained model and measure the performance of the model by calculating various indicators (such as accuracy, recall, F1 score, etc.). If the model performance is not ideal, you can optimize it in the following ways:

[0076] 1) Collect more data: Increasing the amount of data can improve the generalization ability of the model, especially when the amount of data is small.

[0077] 2) Re-examine the feature extraction process and try to add new features or optimize the extraction method of existing features. For example, in the case of EEG, more waveform features can be added; in the case of eye movement signals, new motion parameters can be added.

[0078] 3) Further adjust the model parameters through grid search, random search and other methods.

[0079] 4) If the current model cannot achieve the desired performance, consider using other types of machine learning models, such as deep learning models (such as convolutional neural networks, recurrent neural networks, etc.).

[0080] (3) Model deployment and real-time evaluation.

[0081] First, to improve the model's predictive performance and stability, an ensemble learning approach is used to combine the prediction results of multiple models (such as SVM models with different parameters, a combination of linear regression models and nonlinear models, etc.). The final prediction result is determined by voting or weighted averaging, which can reduce the overfitting problem of a single model and improve the robustness of the system.

[0082] Secondly, when the model is deployed in actual applications, the real-time collected EEG and eye movement signals need to be preprocessed and feature extracted. These features are then input into the trained machine learning model, and the model quickly outputs the evaluation results. For example, fatigue status is assessed on the collected data at regular intervals (such as every minute) and feedback is provided to the user in real time. In actual applications, the model can be continuously updated based on actual user feedback and new data. If a user reports that an evaluation result is inaccurate, the data in that scenario can be collected as new samples and added to the training set, and the model can be retrained to improve the accuracy and adaptability of the model.

[0083] In another practical application, the visual fatigue detection result is expressed as follows: for any moment, it is expressed as a visual fatigue score or visual fatigue degree classification; for any period of time, it is expressed as a visual fatigue score change curve. In short, the visual fatigue detection result is the wearer's visual fatigue assessment result, which can be a specific score (for example, 0-10 points, the higher the score, the deeper the fatigue), or a classification result (such as "normal", "mild fatigue", "moderate fatigue", "severe fatigue", etc.). These visual fatigue detection results are obtained by combining a comprehensive scoring method with the prediction of a machine learning model, and can more accurately reflect the wearer's visual fatigue status.

[0084] When multiple visual fatigue test results are obtained over a period of time, continuous scoring data can be obtained. These continuous scoring data can be updated in real time and displayed in digital form on the device's display. At the same time, a curve graph showing the score changes over time can be drawn, allowing users to intuitively see the changing trend of fatigue levels. In addition to the overall score, the specific values ​​of each key feature can also be displayed. For example, the power spectral density value of the Theta wave and the Alpha wave, the difference in peak and trough amplitude, the blink frequency, the eye movement speed, the blink amplitude, etc. are displayed. These data can be displayed in a list or in the form of bar charts, column charts, etc., so that users can understand the contribution of each feature to fatigue assessment.

[0085] For the classification results, the corresponding classification can be determined based on the threshold range of the score. During the evaluation process, the real-time calculated score is compared with these thresholds. When the score exceeds a certain threshold, the system automatically switches to the corresponding fatigue status classification and triggers the corresponding reminder or intervention measures. At the same time, it is displayed in the form of text on the interface, allowing users to quickly understand their current fatigue status. In addition to text descriptions, icons or symbols can also be used to represent different fatigue states. For example, a green icon represents a "normal" state (corresponding to a score less than 3 points), a yellow icon represents "mild fatigue" (corresponding to a score threshold range of 3-5 points), an orange icon represents "moderate fatigue" (corresponding to a score threshold range of 5-7 points), and a red icon represents "severe fatigue" (corresponding to a score greater than 7 points). This icon representation method is more intuitive. Even if the user does not pay attention to the text description, they can quickly judge the degree of fatigue by the color of the icon.

[0086] In another practical application, if a user's long-term historical data is stored, the current visual fatigue detection results can be compared with the user's baseline data or historical records. Baseline data refers to the characteristic parameters and scoring data of the user in a normal, non-fatigued state. By comparing the differences between the current data and the baseline data, changes in fatigue level can be more accurately determined. For example, if the user's current blink frequency increases by more than 50% compared to the baseline value, and the comprehensive score is also significantly higher than the baseline value, the system will determine that the user is in a state of fatigue and assign a higher level of fatigue classification.

[0087] Step 204 , determining a final visual fatigue detection result based on the multiple visual fatigue detection results; specifically, based on the multiple visual fatigue detection results, a voting method or a weighted average method is adopted to calculate the final visual fatigue detection result.

[0088] In summary, the present application realizes a simplified visual fatigue detection based on the combination of EEG signals and eye movement signals. This detection method directly outputs the visual fatigue detection results by collecting EEG signals and eye movement signals in real time, combining them with machine learning algorithm analysis, simplifying the monitoring requirements of changes before and after the task, and is more suitable for rapid feedback of visual fatigue status, avoiding the complexity and lack of real-time performance in the existing technology.

[0089] Based on the same inventive concept, the present application also provides an intelligent visual fatigue detection system for implementing the above method. The implementation solution provided by this system is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more system embodiments provided below can be referred to the limitations of the method above and will not be repeated here.

[0090] In an exemplary application, the intelligent visual fatigue detection system of the present application includes:

[0091] The information acquisition module is used to obtain the target user's EEG information and eye movement information in real time.

[0092] The feature extraction module is used to: perform feature extraction on the electroencephalogram information and the eye movement information respectively to obtain an electroencephalogram feature set and an eye movement feature set.

[0093] The visual fatigue detection module is used to: input the electroencephalogram feature set and the eye movement feature set into multiple preset visual fatigue detection models respectively to obtain multiple visual fatigue detection results.

[0094] The detection result determination module is used to determine the final visual fatigue detection result based on multiple visual fatigue detection results; wherein different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set.

[0095] Based on the same inventive concept, the present application also provides an intelligent visual fatigue detection device for implementing the above method. The solution provided by the device is similar to the solution described in the above method. Therefore, the specific limitations in one or more device embodiments provided below can be referred to the limitations of the method above and will not be repeated here.

[0096] In an exemplary application, the intelligent visual fatigue detection device of the present application includes an eye movement information collection component, an EEG information collection component, a controller, and a display feedback component. The EEG information collection component is an EEG sensor, which is worn by the target user to collect EEG information in real time, thereby focusing on changes in theta and alpha waves to reflect the wearer's fatigue status. The eye movement information collection component is an eye tracker or smart glasses, which are worn by the target user to collect the target user's eye movement information, focusing on monitoring blink frequency and eye movement status.

[0097] The eye movement information collection component, the electroencephalogram (EEG) information collection component, and the display feedback component are all connected to the controller. The controller is configured to execute a computer program based on the eye movement information and EEG information collected by the eye movement information collection component to implement the intelligent visual fatigue detection method and obtain a final visual fatigue detection result. The display feedback component is configured to display the final visual fatigue detection result to a target user.

[0098] In one specific application, when executing an intelligent visual fatigue detection method, a controller feeds an EEG feature set and an eye movement feature set into a trained machine learning model. The model then outputs a probability distribution of visual fatigue states, which is then used to determine a scoring and classification result. Specifically, the controller is further configured to: determine a visual fatigue level based on the final visual fatigue detection result; match a corresponding visual fatigue warning based on the visual fatigue level; and the display feedback component is further configured to: display the visual fatigue warning to the target user.

[0099] In another specific application, when obtaining the visual fatigue level, classification is performed according to a preset threshold range to obtain the corresponding visual fatigue degree. In this step, the threshold range determination process includes:

[0100] 1) Using standard testing equipment, a control group undergoes a viewing test using a standard task. Data on the wearer's eye characteristics (such as pupil diameter, blink frequency, and eye movement amplitude) and physiological changes before and after viewing are collected. This data reflects the wearer's physiological responses to different states of fatigue, such as the frequency and duration of eye characteristics. In this step, the standard task refers to a viewing test performed using a set of known fatigue tasks, such as prolonged viewing of a digital screen. The fatigue index set refers to the fatigue index set generated based on the wearer's physiological change data before and after the test.

[0101] 2) Based on the collected physiological change data, the threshold is set by calculating the comparative visual fatigue number and comparative visual fatigue speed. The comparative visual fatigue number refers to the specific physiological change value observed when the wearer reaches a certain fatigue state, which can be understood as the "critical fatigue value." The comparative visual fatigue speed refers to the time it takes for the wearer to go from a non-fatigued state to a fatigued state, reflecting the rate of fatigue accumulation. The comparative visual fatigue number and comparative visual fatigue speed calculated based on this data provide a reference for setting the threshold.

[0102] 3) By observing the changes in the wearer under different fatigue indicators during the test, a "target fatigue index" is determined. This index is used to indicate the critical point at which the fatigue state occurs. The threshold setting includes two aspects: the fatigue state frequency threshold and the fatigue state duration threshold. Fatigue state frequency threshold: When the frequency of the eye feature fatigue state (such as blinking frequency) exceeds the set value, it is judged as a fatigue state. Fatigue state duration threshold: When the fatigue feature (such as changes in pupil diameter) lasts for more than a certain period of time (for example, more than 10 seconds), it is judged as fatigue. These thresholds are calculated through experimental data and physiological change laws during standard tests to ensure that they can accurately reflect the wearer's fatigue state.

[0103] In another specific application, an eye tracker or edge processor of a smart glasses device is used to monitor the target user's eye features in real time and compare them with a previously set fatigue threshold. By logically analyzing and comparing the AI ​​synthetic eye model's time-segmented video set with the wearer's real-time eye monitoring video set, it is determined whether the fatigue threshold has been reached. Specifically, the AI ​​synthetic eye model generates a model of the wearer's eye features through historical data and standard task tests to predict possible fatigue changes; real-time video monitoring: The smart glasses device collects the wearer's eye video data in real time and compares and analyzes it with the model to determine whether fatigue has occurred.

[0104] In another specific application, when the display feedback component displays visual fatigue warnings to the target user, different levels of alarms and prompts are set according to the severity of visual fatigue. The specific process is as follows: if the user is in a "mild fatigue" state, a slight sound reminder or a flashing prompt symbol on the display screen is used to attract attention, and the user is advised to take a short break and adjust their eye habits; when the fatigue reaches "moderate fatigue", the alarm can be more obvious, such as the sound is increased and the screen color changes to a warning color; if the user is in a "severe fatigue" state, vibration feedback is activated, and the vision-related task interface (such as the computer screen, mobile phone screen, etc.) is forcibly closed, forcing the user to rest. The user is strongly advised to stop the current visual task and take a long break or take effective relief measures, such as eye massage, looking into the distance, etc., and the user will be prompted with some suggestions for improving the visual environment and eye habits to help the user reduce the occurrence of visual fatigue. This graded alarm mechanism can ensure that users can take timely measures when the fatigue level gradually deepens to avoid further deterioration of visual fatigue.

[0105] Compared with the existing technology, this application also has the following advantages:

[0106] (1) This application monitors EEG signals and eye movement signals in real time, eliminating the need for complex pre- and post-task change detection. This simplifies the traditional method's reliance on complex change values, provides more efficient and simple fatigue detection, and enhances the system's ease of use and practicality.

[0107] (2) This application combines EEG and eye movement signals, and through the fusion analysis of multi-dimensional data, avoids the uncertainty brought by traditional subjective evaluation methods and improves the reliability and real-time performance of detection.

[0108] (3) This application avoids the errors of single signal methods by integrating and processing multi-dimensional physiological signals in real time, thereby improving the accuracy and response speed of visual fatigue detection.

[0109] (4) This application is not only applicable to daily office, study, driving and other scenarios, but can also be applied to medical health, scientific research and other fields to provide more accurate visual fatigue assessment.

[0110] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0111] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0112] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0113] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0114] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0115] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0116] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An intelligent method for detecting visual fatigue, characterized in that: include: Obtain target user's EEG and eye movement information in real time; performing feature extraction on the electroencephalogram information and the eye movement information respectively to obtain an electroencephalogram feature set and an eye movement feature set; Inputting the EEG feature set and the eye movement feature set into a plurality of preset visual fatigue detection models respectively to obtain a plurality of visual fatigue detection results; Determine a final visual fatigue detection result based on multiple visual fatigue detection results; Among them, different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set.

2. The intelligent visual fatigue detection method according to claim 1, characterized in that: The EEG information includes Theta wave information and Alpha wave information; the eye movement information includes blink frequency, eye movement status, eyelid status, pupil diameter, eye saccade information and gaze information; The EEG feature set includes the power spectral density of theta wave, the peak-to-trough amplitude difference of theta wave, the standard deviation of theta wave, the power spectral density of the alpha wave, the peak-to-trough amplitude difference of the alpha wave, and the standard deviation of the alpha wave; The eye movement feature set includes eye movement trajectory features, eye movement speed, blink amplitude and blink frequency.

3. The intelligent visual fatigue detection method according to claim 1, characterized in that: Before the step of respectively extracting features from the electroencephalogram information and the eye movement information, the method further includes: The electroencephalogram information is sequentially subjected to a first filtering process and a denoising process; the eye movement information is sequentially subjected to a second filtering process and a denoising process; the denoising process is one of wavelet denoising and independent component analysis.

4. The intelligent visual fatigue detection method according to claim 1, characterized in that: Feature extraction is performed on the EEG information and the eye movement information respectively to obtain an EEG feature set and an eye movement feature set, including: Performing initial feature extraction on the electroencephalogram information to obtain a plurality of initial electroencephalogram features; performing initial feature extraction on the eye movement information to obtain a plurality of initial eye movement features; L1 regularization or tree model-based feature importance analysis is used to perform feature screening on multiple initial EEG features and multiple initial eye movement features to obtain corresponding EEG feature sets and eye movement feature sets.

5. The intelligent visual fatigue detection method according to claim 1, characterized in that: The machine learning model corresponding to the preset visual fatigue detection model includes a support vector machine model and a linear regression model.

6. The intelligent visual fatigue detection method according to claim 1, characterized in that: Determining a final visual fatigue detection result based on multiple visual fatigue detection results includes: calculating the final visual fatigue detection result based on the multiple visual fatigue detection results using a voting method or a weighted average method.

7. The intelligent visual fatigue detection method according to claim 1, characterized in that: The visual fatigue detection result is expressed as follows: At any moment, the visual fatigue score or visual fatigue degree classification is used to express; For any period of time, the visual fatigue score change curve is used to express it.

8. An intelligent visual fatigue detection system, characterized in that: The system includes: The information acquisition module is used to obtain the target user's EEG information and eye movement information in real time; A feature extraction module is used to: extract features from the EEG information and the eye movement information respectively to obtain an EEG feature set and an eye movement feature set; an asthenopia detection module, configured to input the electroencephalogram feature set and the eye movement feature set into a plurality of preset asthenopia detection models, respectively, to obtain a plurality of asthenopia detection results; A detection result determination module is used to determine a final visual fatigue detection result based on multiple visual fatigue detection results; Among them, different preset visual fatigue detection models correspond to different machine learning models, and each of the preset visual fatigue detection models is obtained by training the machine learning model based on the visual fatigue detection sample set.

9. An intelligent visual fatigue detection device, characterized in that: The device includes an eye movement information collection component, an electroencephalogram information collection component, a controller and a display feedback component; The eye movement information collection component, the electroencephalogram information collection component, and the display feedback component are all connected to the controller; the controller is configured to: execute a computer program based on the eye movement information collected by the eye movement information collection component and the electroencephalogram information collected by the electroencephalogram information collection component to implement the intelligent visual fatigue detection method according to any one of claims 1 to 7, and obtain a final visual fatigue detection result; The display feedback component is used to display the final visual fatigue detection result to the target user.

10. The intelligent visual fatigue detection device according to claim 9, characterized in that: The controller is further configured to: determine an eye fatigue level according to the final eye fatigue detection result; and match a corresponding eye fatigue warning according to the eye fatigue level; The display feedback component is further configured to display the visual fatigue warning to the target user.

Images