Digital eyestrain detection method based on multi-task learning and intelligent terminal

Abstract

The invention discloses a digital eyestrain detection method based on multi-task learning and an intelligent terminal. The method mainly solves the problems that the prior art depends on a single mode, only pays attention to a single task (for example, only the opening and closing state is detected), and comprehensive evaluation on the fatigue state of a user, especially the eye dryness condition is lacked. The invention provides a digital eye fatigue detection method based on multi-task learning and an intelligent terminal, and aims to realize high-precision fatigue and distance detection on low-cost hardware through fusion of multi-task learning and a monocular vision geometric model, solve the problems of limited precision of a single mode and redundancy of separate deployment calculation in the prior art, and improve the detection precision of the eye fatigue. The method is suitable for mobile terminals such as smart phones and tablet computers. After the scheme is deployed on the intelligent terminal, a user can be helped to keep a healthy screen use distance and a healthy screen display parameter, a healthy blinking habit is maintained, and dry eyes, astringent eyes and eyestrain are reduced.

Description

Technical Field

[0001] This invention relates to the field of smart device technology, specifically to a digital eye fatigue detection method and smart terminal based on multi-task learning. Background Technology

[0002] With the widespread adoption of smartphones and the continuous increase in usage time, visual health problems caused by prolonged close-range screen viewing are becoming increasingly prominent. These problems typically manifest in two main categories of symptoms: first, improper posture and excessively close viewing distance lead to increased accommodative strain and binocular convergence burden, resulting in eye fatigue, blurred vision, and even decreased vision; second, prolonged intense focus on screen content causes a significant decrease in natural blinking frequency, leading to excessively rapid tear evaporation and discomfort such as dry eyes, foreign body sensation, and stinging, which may accumulate over time and develop into clinical dry eye syndrome. In response to this phenomenon of "digital eye fatigue," the industry has begun exploring visual fatigue monitoring solutions based on smart terminals; however, most existing technologies still rely on single modalities or limited features, which have certain limitations in terms of accuracy, applicability, and user experience.

[0003] Several patented technologies have attempted to detect fatigue from different angles. For example, patent CN113591682B achieves multi-stage fatigue determination by fusing dynamic and static eye features, but its design is more suitable for scenarios such as driving fatigue and is insufficiently targeted at problems unique to digital eye fatigue, such as dryness and accommodative spasm. Patent CN103680465A uses the frequency of pupil diameter changes captured by a front-facing camera as a basis for fatigue judgment and adjusts screen parameters accordingly to alleviate fatigue, but this method has high requirements for device hardware and user cooperation, and the stability of a single pupil index is limited in complex lighting environments. Another patent, CN110784600A, focuses on detecting the distance between the human eye and the screen. Although it can help correct poor posture, it lacks the ability to directly judge and warn of fatigue states, and its function is relatively simple.

[0004] In addition, some technical solutions face other challenges in implementation. For example, the solution that uses an infrared emitter to detect blink frequency (such as invention patent CN202222101843) can achieve high-precision eye movement monitoring, but there are potential safety risks from prolonged direct exposure of infrared light to the human eye, and this hardware is not yet widely used in consumer electronic devices, making it difficult to achieve daily application.

[0005] In summary, the main drawbacks of existing technologies are: 1) Single-modal dependence: They typically focus on only a single task, such as detecting only the open / closed state or only distance, lacking a comprehensive assessment of user fatigue, especially eye dryness. 2) Insufficient collaboration: Existing technologies handle blink detection and distance detection independently, without learning shared features through multi-task learning, leading to computational redundancy and accuracy bottlenecks. Especially in mobile scenarios where computing resources are scarce, separate deployment will bring additional memory and power consumption overhead. 3) Complex network design: Existing patented solutions all first predict key points of the human eye, and then determine the open / closed state based on the geometric rules (aspect ratio) of the key points. This two-stage design exacerbates the complexity of the model, meaning that the result is only reliable after all key points are accurately predicted. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a digital eye fatigue detection method and intelligent terminal based on multi-task learning, which mainly solves the problem that the existing technology relies on a single modality, focuses only on a single task (such as only detecting the open and closed state), and lacks a comprehensive assessment of the user's fatigue state, especially the condition of dry eyes.

[0007] The technical solution of the present invention is as follows: A digital eye fatigue detection method based on multi-task learning includes the following steps: Step 1: Extraction and Assignment of User Behavior Scenarios: An important factor in the development of eye fatigue is the level of cognitive and visual demands required to perform the task, as well as the speed at which visual information is presented. Activities that require high cognitive and visual demands, such as video games, will reduce the blinking frequency because they require longer fixation time to complete the task. The system monitors user application usage after a single screen unlock on a smart terminal, identifies the application type (assigning a value of 1 for game applications and 0 for non-game applications), and records continuous usage time for each application in real-time, with statistics in hours. The system also performs real-time eye distance calculation, periodically (every 15 minutes of screen time) capturing facial images via the front-facing camera and calculating the average distance between the user's eyes and the smart terminal's screen. Finally, it extracts blink features in real-time, recording eye videos via the front-facing camera at preset intervals (every 15 minutes of screen time) and extracting blink features from the videos using image recognition and processing algorithms. These blink features include blink frequency, incomplete blink frequency, and incomplete blink rate. Step 5: Real-time prediction of digital eye fatigue: The application type obtained in Step 1, the usage duration obtained in Step 2, the average distance obtained in Step 3, and the blinking features obtained in Step 4 are input into the SANDE (SymptomAssessment iN Dry Eye, score range 0-100, higher values ​​indicate more severe symptoms) prediction model to calculate the SANDE prediction value; the SANDE prediction model is as follows:

[0008] Where time represents the duration of a single screen-on session, task represents the type of task used after unlocking the screen, distance represents the average user distance, IBR represents the incomplete blink rate, and β1, β2, β3, and β4 are regression coefficients.

[0009] is the interaction term, and ɛ is the residual (random effect); Step Six: Fatigue Intervention: Determine whether the SANDE prediction value obtained in Step Five exceeds the preset threshold T. If so, push proactive eye protection reminders to the user through the display screen, including healthy blink reminders, reminders for excessive close viewing distance, reminders for adjusting screen parameters, and short rest reminders.

[0010] The application type identification mentioned in step one is based on the application package name or activity name provided by the mobile operating system.

[0011] The real-time calculation of the human eye distance described in step three is based on the pinhole imaging principle and is specifically implemented using the formula d = W ×f / w, where d is the distance from the human eye to the camera, W is the preset statistical average interpupillary distance, f is the camera focal length pre-calibrated using the Zhang Zhengyou calibration method, and w is the pixel distance between the centers of the two pupils identified from the acquired face image.

[0012] The real-time extraction of blink features described in step four is specifically as follows: A preset acquisition frequency of once every 30 minutes is used, with each recording lasting 1 minute of eye video; the recorded video is processed by frame extraction, and face and eye region recognition is performed on each frame; complete and incomplete blink events are identified and statistically analyzed using an algorithm; the blink frequency is the sum of the frequencies of complete and incomplete blinks within each recording period; the incomplete blink rate is the ratio of the incomplete blink frequency to the total blink frequency within that period. The incomplete blink frequency is the number of times the upper and lower eyelids do not fully contact each other during a blink within that period.

[0013] The smart terminal is a mobile phone or a tablet computer.

[0014] It also includes step seven, the healthy blinking action: close your eyes for 2 seconds, then open them for 2 seconds, then close them for 2 seconds and then blink for 2 seconds.

[0015] A smart terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a digital eye fatigue detection method based on multi-task learning as described above.

[0016] The beneficial effects of this invention are as follows: This invention provides a digital eye fatigue detection method and smart terminal based on multi-task learning. By fusing multi-task learning with a monocular visual geometric model, high-precision fatigue (dry eye) and distance detection are achieved on low-cost hardware. This solves the problems of limited accuracy of single-modality detection and redundant computation in existing technologies, and is applicable to mobile terminals such as smartphones and tablets. After being deployed on a smart terminal, this solution can help users maintain a healthy screen usage distance and screen display parameters, maintain healthy blinking habits, and reduce dry eyes, eye strain, and eye fatigue. Attached Figure Description

[0017] Figure 1 The results showed that the SANDE dry eye symptom score was significantly correlated with all objective indicators, including tear film breakup time, tear river height, and redness index. Incomplete blinking was significantly correlated with both subjective and objective indicators, while blinking frequency, which is the focus of existing technologies, had a weak correlation with subjective and objective indicators related to eye fatigue.

[0018] Figure 2 This is a schematic diagram illustrating the principle of distance calculation according to an embodiment of the present invention. Based on the pinhole imaging principle, it is specifically implemented using the formula d = W × f / w, where d is the distance from the human eye to the camera, W is the preset statistical average interpupillary distance, f is the camera focal length pre-calibrated using the Zhang Zhengyou calibration method, and w is the pixel distance between the centers of the two pupils identified from the acquired face image.

[0019] Figure 3 This is a comparative diagram of the present invention and existing methods. Patent CN108205876A assesses eye fatigue by acquiring distance and continuous observation time, but based on the dataset of this patent, its model's explanatory power is approximately 69.6%, lower than the 79% of the model of the present invention. Furthermore, its method relies on specific smart glasses, limiting its application scenarios. Patent CN111699698A mainly calculates fatigue based on blinking frequency and eye-closing time. Based on the dataset of this patent, its model's explanatory power is only 13.8%, significantly lower than the model of the present invention. Because it only focuses on a single blinking behavior, its assessment dimensions for eye fatigue are relatively limited, making it difficult to comprehensively reflect the user's true fatigue state.

[0020] Figure 4The study revealed differences in blinking characteristics between gaming and non-gaming scenarios. Gaming scenarios, which involve high cognitive and visual demands, showed a significantly higher rate of incomplete blinking compared to non-gaming scenarios. Furthermore, the rate of incomplete blinking was significantly correlated with both subjective and objective fatigue indicators, suggesting that incorporating visual task types and their corresponding incomplete blinking rates into the model is crucial for accurately assessing eye fatigue.

[0021] Figure 5 This is a flowchart of the present invention. Detailed Implementation

[0022] The invention will be further described below with reference to the accompanying drawings. A digital eye fatigue detection method based on multi-task learning includes the following steps: Step 1: User Behavior Scene Extraction and Assignment: Monitor user application usage after a single screen unlock on a smart terminal, identify the application type, assigning a value of 1 to game applications and 0 to non-game applications. Specifically, the application type is identified based on the application package name or activity name provided by the smart terminal's operating system. If the currently foreground application is a game application, assign a value of 1; if it is a non-game application such as video, reading, or social media, assign a value of 0. This assignment reflects the different impacts of different task types on visual load. Frequently used apps can also be pre-labeled. See the appendix for differences in blink characteristics between game and non-game applications. Figure 4 Where G represents game-related categories, V represents non-game-related categories, and 20, 30, and 40 represent different device distances (in cm).

[0023] Step 2: Real-time Usage Duration Recording: Records the continuous usage time of users on various applications and statistically analyzes it in hours; this data quantifies the cumulative effect of visual load. Step 3: Real-time Eye Distance Calculation: During application use, facial images are periodically captured via the front-facing camera (every 15 minutes of screen time), and the average distance from the user's eyes to the screen of the smart terminal is calculated in real time. Step 4: Real-time Blink Feature Extraction: During application use, eye videos are recorded via the front-facing camera at preset intervals (every 15 minutes of screen time). Blink features are extracted from the videos using image recognition and processing algorithms. These features include blink frequency, incomplete blink frequency, and incomplete blink rate. Step 5: Real-time Prediction of Digital Eye Fatigue: The application type obtained in Step 1, the usage duration obtained in Step 2, the average distance obtained in Step 3, and the blink features obtained in Step 4 are input into the SANDE prediction model to calculate the SANDE prediction value. The SANDE prediction model is a multiple regression model containing second-order and third-order interaction terms, satisfying the following relationship:

[0024] where time is the duration of a single screen-on period, task is the type of task used after unlocking the screen during a single screen-on period, distance is the average usage distance of the user, IBR is the incomplete blink rate, and β1, β2, β3, and β4 are regression coefficients.

[0025] is an interaction term, and ɛ is the residual (random effect). Step Six: Fatigue Intervention: Determine whether the SANDE prediction value obtained in Step Five exceeds the preset threshold T. When S ≥ T, an active eye protection reminder is pushed to the user through the display screen. When S < T, no reminder is triggered or only background recording is performed. Further, to avoid frequent pop-ups affecting the user experience, a cooling time Δt (e.g., 5 minutes) is set after the reminder is triggered, and no repeated pop-up occurs even if S ≥ T during the cooling time; or the "continuous over-threshold trigger" strategy is adopted: the reminder is triggered only when n consecutive samples (e.g., n = 3) satisfy S ≥ T. The active eye protection reminder specifically includes a healthy blink reminder, a reminder of too close viewing distance, a reminder of screen parameter adjustment, a short break reminder, etc. The application type identification described in Step One is determined based on the application package name or activity name provided by the mobile phone operating system. The SANDE threshold T is 37 points.

[0026] The real-time calculation of the human eye distance described in Step Three is based on the principle of pinhole imaging and is specifically implemented based on the formula d = W × f / w, where d is the distance from the human eye to the camera, W is the preset statistical average pupil distance, f is the camera focal length pre-calibrated by the Zhang Zhengyou calibration method, and w is the pixel distance between the centers of two pupils identified from the captured face image.

[0027] The real-time extraction of blink features described in step four involves the following process: A preset acquisition frequency of once every 30 minutes, with each recording lasting 1 minute, is used to capture eye videos. The recorded videos are then processed by frame extraction, and face and eye region recognition is performed on each frame. An algorithm identifies and statistically analyzes complete and incomplete blink events. An incomplete blink event is defined as a blink in which the upper and lower eyelids do not fully contact each other during that time period. The blink frequency is the sum of the frequencies of complete and incomplete blinks within each recording period; the incomplete blink rate is the ratio of the incomplete blink frequency to the total blink frequency within that period. Specifically, eye images can be processed using a target recognition model based on a multi-task convolutional neural network architecture (e.g., CN120894816A). This model consists of a shared feature extraction backbone and multiple parallel branch modules, capable of simultaneously outputting parameters for eye occlusion status (eye confidence), pupil pixel coordinates, and multi-dimensional eye morphology parameters. The system is configured with n layers of 3×3 convolutional kernels and 256 output channels, and the feature map is compressed into a 1×1×256 feature vector through a global average pooling (GAP) layer. The subsequent five parallel branch modules (Branch 1-5) all adopt a fully connected (FC) layer structure, with a dimensionality transformation configuration of 256→128→output dimension (1 or 2). The output layer is activated by a sigmoid activation function to map the predicted values ​​to the (0,1) interval, thus simultaneously outputting the eye occlusion confidence score and pupil pixel coordinates (x,y). Then, a face eye morphology parameter recognition model is used, containing 21 convolutional layers (CONV_2D) and 16 depthwise separable convolutional layers (DEPTHWISE_CONV_2D), and the layers are connected through 16 residual fusion (ADD) operations. For the activation function, ReLU is applied after each residual ADD operation and after the initial convolutional layer of the model. The detection head uses 1×1 convolutions on two scales (16×16 and 8×8) for classification and regression output, without any activation function. Specifically, the eye morphology parameters include: eye angle parameters based on the ratio of the upper / lower eyelid angle to the inner / outer canthus angle, and parameters reflecting blinking characteristics such as the first and second distance ratios determined based on the interpupillary distance and corneal width, and the interpupillary distance and intercanthal distance, respectively.

[0028] When constructing blink feature parameters, the eye distance information is calculated using the identified pupil coordinates, and a highly robust eye-opening ratio is calculated. Finally, this eye-opening ratio is input into a state machine model, and dynamic thresholds are used to determine the open, closed, and partially closed eye states. By analyzing the peaks and troughs of the state change curves, the blink frequency is output.

[0029] To ensure the reliable implementation of the technical solution in complex scenarios, this application constructed a multi-dimensional sample dataset including different shooting distances, shooting angles, wearing glasses, facial occlusion, and strong / low light conditions, and trained the model accordingly. Under 300 lux illumination, eye video data of 21 subjects at three different distances (20cm, 30cm, and 40cm) were collected, totaling 1.134 million frames, with 8,971 complete blinks and 3,119 incomplete blinks annotated; under 100 lux illumination, eye video data of 20 subjects at a distance of 30cm was collected, totaling 540,000 frames, with 4,104 complete blinks and 2,019 incomplete blinks annotated.

[0030] Specifically, for strong light or low light environments, image enhancement algorithms (such as Retinex or CLAHE) are used before feature extraction to remove uneven lighting and enhance local contrast; for facial occlusion, an independent occlusion detection branch (Branch 5) is integrated into the model to output occlusion confidence in real time to filter invalid frames and prevent the accumulation of feature extraction errors.

[0031] The smart terminal is a mobile phone or a tablet computer.

[0032] It also includes step seven, the healthy blinking action: close your eyes for 2 seconds, then open them for 2 seconds, then close them for 2 seconds and then blink for 2 seconds.

[0033] A smart terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements a digital eye fatigue detection method based on multi-task learning as described above.

[0034] In addition, this application also includes a preliminary data processing step, the specific steps of which are as follows: Definition of fatigue coefficient: VDT (Video Display Terminal) Experimental Data Acquisition Participants were recruited and subjected to varying screen time (e.g., 30 minutes, 1 hour, 2 hours) and viewing distance (e.g., 20cm, 30cm, 40cm) in different scenarios (games, videos) to induce different levels of digital eye fatigue. Blink information was collected using the front-facing camera of a mobile phone, including the number of blinks per minute (blink frequency), incomplete blink frequency, and the proportion of incomplete blinks per minute (IBR). Objective indicators reflecting visual fatigue and tear film homeostasis were collected before and after the VDT experiment. Subjective evaluations of digital eye fatigue (Computer Vision Syndrome Questionnaire CVSQ) were also collected, with objective indicators including continuous functional visual acuity, accommodation ability, tear film breakup time, tear river height, red eye index, and corneal fluorescein staining score. The severity of subjective dry eye symptoms was collected periodically using the Dry Eye Symptom Assessment Scale (SANDE) to obtain dynamic changes in the subjects' subjective symptoms, thus obtaining a dataset.

[0035] Correlation analysis to quantify the impact of various factors on digital eye fatigue. First, we analyzed the changes in objective indicators and subjective symptoms of the subjects before and after the VDT experiment. We then conducted a correlation analysis between the subjective dry eye symptom score SANDE and various objective indicators. The results showed that SANDE was significantly correlated with all objective indicators, including tear film breakup time, tear river height, and redness index. Therefore, SANDE will be used as the dependent variable in subsequent modeling.

[0036] The fitting formula establishes a correspondence between objective indicators and visual fatigue. An optimal predictive model was established between data collected via mobile phone and subjective symptoms, and then an estimate of digital eye fatigue was obtained based on this optimal predictive model. Using the subjects' SANDE score as the predictor variable, and the proportion of incomplete blinking, usage duration, usage distance, and app type (games, non-games) as independent variables, a mixed linear model was established.

[0037] Where time represents the duration of a single screen-on session, task represents the type of task used after unlocking the screen, distance represents the average user distance, IBR represents the incomplete blink rate, and β1, β2, β3, and β4 are regression coefficients.

[0038] α is the interaction term, and ɛ is the residual (random effect). The parameters in the above formula are obtained by fitting a model based on data collected from the hospital.

[0039] To construct a robust mixed-effects model capable of predicting SANDE_diff, this study employed a strategy combining hypothesis testing-based model selection with cross-validation-based performance evaluation. First, an initial full model was constructed, containing all predictor variables (distance, incomplete blink rate, time, and task type) and their highest-order fourth-order interactions. Within the maximum likelihood estimation (ML) framework, a likelihood ratio test was performed using the drop1 function. Following the hierarchical principle, insignificant higher-order interactions were progressively eliminated to obtain a streamlined selected model. Subsequently, to validate the model's generalization ability and prevent overfitting, 10-fold cross-validation was implemented. Root mean square error (RMSE) and mean absolute error (MAE) were used as core evaluation metrics to compare the full model, the selected model, and the basic model containing only main effects. Finally, the screening model with the lowest RMSE in cross-validation and which preserved the significant interaction effect was selected as the optimal model, and the final parameter estimation was performed on it using restricted maximum likelihood estimation (REML).

[0040] Best-fit model selection process:

[0041] RMSE (Root Mean Square Error) is an indicator obtained by taking the square root of the average of the squared errors. It is more sensitive to larger prediction errors and thus highlights the model's performance under extreme bias conditions. The smaller the RMSE, the lower the overall prediction bias and the better the model's fit.

[0042] MAE (Mean Absolute Error) is the average of the absolute values ​​of prediction errors, reflecting the average bias of the model. Compared to RMSE, MAE is less sensitive to outliers and is a more robust error measure. The smaller the MAE, the smaller the average prediction error of the model and the higher its prediction stability.

[0043] NRMSE (Normalized RMSE) standardizes RMSE by its range or mean, making the error metric comparable and facilitating the evaluation of model performance across variables of different dimensions or scales. A lower NRMSE indicates a smaller relative prediction error and better predictive performance.

[0044] In a preferred embodiment, a mixed-effects model is constructed to characterize the relationship between each factor and the symptom score, and its specific form is as follows:

[0045] in, 0 represents the intercept term, used to indicate the baseline level; 1 is used to characterize the effect of usage time on symptom scores; as usage time increases, symptom scores worsen. 2. Used to characterize the impact of task type on symptom scores; 3 is used to characterize the effect of screen distance on symptom scores; the greater the screen distance, the less severe the symptom scores. 4 is used to characterize the effect of incomplete blink rate on symptom scores; the higher the incomplete blink rate, the more severe the symptom score. 5. Used to characterize the interaction effect between screen distance and incomplete blink rate; 6 is used to characterize the interaction effect between screen distance and usage time; 7 was used to characterize the interaction effect between usage time and incomplete blink rate; 8. Used to characterize the interaction effect between task type and incomplete blink rate; 9 is used to characterize the interaction effect between task type and usage time; 10 is used to characterize the third-order interaction effect of screen distance, incomplete blink rate, and usage time; 11 is used to characterize the third-order interaction effect of incomplete blink rate, usage time and task type; (1 | name) represents the random effect term introduced for different individuals to characterize the impact of individual differences on the model results.

[0046] The specific values ​​of the above parameters are not limited in this specification. Those skilled in the art can determine them by performing regression analysis, maximum likelihood estimation, or other methods on the experimental or collected data based on the statistical modeling methods described above.

[0047] The SANDE preset threshold T mentioned in step six is ​​determined based on the linear relationship between SANDE and the actual CVSQ score established from clinical trial data. The relationship between the two obtained by fitting the clinical trial data is: CVSQ = 0.1247 × SANDE + 1.355. It is known that the threshold for diagnosing computer vision syndrome in the CVSQ (Computer Vision Syndrome Questionnaire) is 6 points. Substituting this into the above formula, the SANDE threshold T is calculated to be 37 points.

[0048] Model adaptation and calibration (1) Design for applicability across screen sizes The SANDE prediction model in this application employs physical quantity / visual angle normalization in its input feature design to reduce the impact of screen size differences. Specifically: Features such as "viewing distance," "gazing angle," and "screen content viewing angle" are uniformly converted into quantities unrelated to the screen diagonal size, such as centimeters (cm), visual angles (deg), or rate of change per minute. Screen parameters (such as diagonal size, resolution, PPI, brightness, color temperature / eye protection mode status) are read by the terminal system interface and used as the device parameter vector d input to the model or for normalizing related features. Therefore, under the same eye-use behavior and environmental conditions, the impact of different screen sizes on the model output is limited to a controllable range, allowing the same set of basic coefficients to be reused on terminals of different sizes.

[0049] (2) Applicability design across camera parameters To address the differences caused by varying camera focal lengths, field of view, resolutions, and installation locations, this application employs the following mechanism to ensure the consistency of measurement features: The intrinsic parameters of the terminal camera (focal length / equivalent focal length, field of view, resolution) are obtained through the system interface or calibrated once at the factory / on first use, resulting in the camera parameter vector c. Features related to viewing distance, head pose, and gaze direction are converted into device-independent quantities such as centimeters or angles (e.g., converting pixel displacement into angles / distances), and the resolution is normalized. When a change in camera parameters is detected to exceed a threshold (e.g., a significant change in resolution / field of view), the system triggers rapid calibration on the device side to update c or correct the mapping relationship. Therefore, under different camera parameter conditions, the model can still maintain its feasibility and transferability through "parameter reading / calibration + physical quantity normalization".

[0050] (3) Structure of the base model and population correction In this embodiment, the SANDE model coefficients are fitted based on experimental data from specific young and middle-aged subjects. When the model shows significant deviations in other specific populations, it is necessary to calibrate / select population coefficients through a publicly available correction procedure: The SANDE predicted value S is output from the base model and a population correction term is added.

[0051] in: z represents the normalized eye-use / environment / device-related characteristics; The base coefficients obtained by fitting on the training set can be used directly for the general population; For population attributes (such as age group, refractive status, dry eye high-risk label, etc.). g( ; ) is the population correction function (which can be a piecewise constant, a linear term, or a lookup table term).

[0052] A more engineered and readily public implementation is a "grouping coefficient table": Preset at least three groups of population coefficients teen , adult , elder These correspond to teenagers, middle-aged people, and the elderly, respectively. The terminal selects the corresponding coefficient based on the age group for calculation.

[0053] In one implementation, the crowd correction function ( ; Take the piecewise constant lookup table entries. Assuming the SANDE predicted value has dimensions of 0–100, the age group correction terms are as follows: = 4. =0、 =+6; Additional correction when a user is flagged as high-risk for dry eye. =+8; Additional correction item when the user has high refractive error. =+3. The terminal, based on... Select the corresponding correction term and overlay it with the output of the base model.

[0054] Model application The above model is deployed on smart terminals such as mobile phones, and the front-facing camera is used to periodically detect distance and blinking and incomplete blinking; Based on the results of previous experiments, when the user's usage distance is less than the healthy distance, the display screen will remind the user to maintain the healthy distance. Based on the SANDE prediction value calculated by the model, if the SANDE is greater than the threshold, the display screen will remind the user to blink in a scientific way (close eyes for 2 seconds, then open eyes for 2 seconds, then close eyes for 2 seconds and then squeeze eyes for 2 seconds) to keep the surface of the eyes moist and reduce dry eyes.

[0055] Based on the SANDE prediction values ​​calculated by the model, the screen display parameters are dynamically adjusted to maximize eye protection while ensuring the best visual effect.

[0056] Based on the SANDE prediction value calculated by the model, the user is prompted to adopt a 20-20-20 rest strategy or close their eyes and rest for a few seconds.

[0057] The specific results are shown in Figure 3, which intuitively demonstrates the advantages of our predictive model compared to existing technologies: the model's Conditional R² (R²c) is significantly higher, indicating that the overall model (including fixed and random effects) has stronger explanatory power; while Marginal R² (R²m) represents the proportion of variance explained only by fixed effects, which is a measure of the contribution of fixed effects to the model.

[0058] The embodiments described with reference to the accompanying drawings are exemplary and intended to explain the invention, and should not be construed as limiting the invention. The embodiments should not be considered as limiting the invention, but any improvements made based on the spirit of the invention should be within the scope of protection of the invention.

[0059] Specific Implementation: A Complete Flowchart of Digital Eye Fatigue Detection Based on Multi-Task Learning on a Mobile Device When the screen is lit and unlocked, the system reads the foreground application type ∈ GameList (game category) → task=1.

[0060] Record the unlock time t0 and obtain the current time t in real time. now Calculate the screen-on duration: time raw = t now - t0=3.0 h.

[0061] Distance sampling: The distance is approximately 20 cm, calculated over multiple frames within the window (obtained by d=W_IPD×f / w and the average value is taken).

[0062] Trigger a 1-minute blink detection: Total blinks N_blink=10, incomplete blinks N_IB=6, BF=10 / min, IBF=6 / min, IBR=6 / 10=0.6.

[0063] Substituting into the optimal implementation formula, we get S≈39.56 (example calculation result). When a user activates the feature, the terminal imports / fills out a questionnaire and obtains the following information: Age Group: Middle-aged (adult) → ge =0, Dry eye diagnosis / High risk: Yes → =+8, High refractive error: No (normal refractive error) → φ refrac = 0 After correction, S = 47.56 Because S≥T, an eye protection reminder is triggered: a pop-up window prompts "too close to view" and "blinking too often", and provides a demonstration of healthy blinking actions or a one-click setting of recommended eye protection screen parameters (close for 2 seconds - open for 2 seconds - close for 2 seconds - squeeze for 2 seconds). At the same time, it enters the cooling Δt to prevent repeated interruptions.

Claims

1. A digital eye fatigue detection method based on multi-task learning, characterized in that: Comprising the following steps, Step one, user behavior scene extraction and assignment: monitor the application usage after single screen unlocking when the user uses the smart terminal, identify the application type, if it is a game application, assign it to 1, if it is a non-game application, assign it to 0; Step two, real-time recording of usage time: record the continuous usage time of the user on each application, and count it in units of hours; Step three, real-time calculation of eye distance: periodically collect face images through the front camera during the user's application usage, and calculate the average distance from the human eye to the screen of the smart terminal in real time; Step four, real-time extraction of blinking features: during the user's application usage, record eye videos at a predetermined period through the front camera, and extract blinking features from the video, the blinking features including blinking frequency, incomplete blinking frequency and incomplete blinking rate; Step five, real-time prediction of digital eye fatigue degree: input the application type assignment obtained in step one, the usage time obtained in step two, the average distance obtained in step three and the blinking features obtained in step four into the SANDE prediction model to calculate the SANDE prediction value; the SANDE prediction model is as follows: Wherein time is the single screen on time, task is the task type used after single screen unlocking, distance is the average use distance of the user, IBR is the incomplete blinking rate, β1, β2, β3, β4 are regression coefficients, is the interaction term, and ɛ is the residual error; Step six, fatigue intervention: judge whether the SANDE prediction value obtained in step five exceeds the preset threshold, if yes, remind the user.

2. The digital eye fatigue detection method based on multi-task learning according to claim 1, characterized in that: The application type identification in step one is based on the application package name or activity name provided by the mobile operating system.

3. The digital eye fatigue detection method based on multi-task learning according to claim 1, characterized in that: The real-time calculation of eye distance in step three is based on the pinhole imaging principle, specifically based on the formula d = W × f / w, wherein d is the distance from the eye to the camera, W is the preset statistical average interpupillary distance, f is the camera focal length pre-calibrated by Zhang Zhengyou calibration method, and w is the pixel distance between the two pupil centers identified from the collected face image.

4. The digital eye fatigue detection method based on multi-task learning according to claim 1, characterized in that: The real-time extraction of blinking features in step four has the following specific process: the preset collection frequency is to start once every 30 minutes, and each recording duration is 1 minute of eye video; frame extraction processing is performed on the recorded video, and face and eye region recognition is performed on each frame of image; a deep learning model is constructed based on multi-task network architecture through eye feature recognition algorithm to obtain eye confidence, pupil pixel coordinates and open eye ratio, and the blinking feature parameters are constructed to distinguish open eye and closed eye states, and finally based on the state machine model, the blinking frequency is output, and the complete blinking and incomplete blinking events are counted; the blinking frequency is the sum of the complete blinking and incomplete blinking frequencies in each recording period; the incomplete blinking rate is the ratio of the incomplete blinking frequency to the blinking frequency in the period.

5. The digital eye fatigue detection method based on multi-task learning according to claim 1, characterized in that: The smart terminal is a mobile phone or a tablet computer.

6. The digital eye fatigue detection method based on multi-task learning according to claim 1, characterized in that: It further includes step seven, healthy blinking action: close your eyes for 2 seconds, open your eyes for 2 seconds, close your eyes for 2 seconds, and then squeeze your eyes for 2 seconds.

7. An intelligent terminal comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the multi-task learning-based digital eye fatigue detection method according to any one of claims 1-6 when executing the program.

Images