Eye behavior monitoring method, smart glasses and readable storage medium

Abstract

This application discloses a method for monitoring eye use behavior, smart glasses, and a readable storage medium, relating to the field of wearable device technology and applied to smart glasses. The method includes: collecting user eye use behavior data through a multimodal sensor, and accumulating the near-field eye use time based on the eye use behavior data; outputting an alert signal when the accumulated near-field eye use time reaches a preset alert threshold; responding to the alert signal, monitoring the user's relaxation behavior through the multimodal sensor, including distance viewing, outdoor activities, and light activity; and resetting the accumulated near-field eye use time to zero when the duration of any relaxation behavior reaches a preset threshold, so as to re-monitor the user's eye use behavior. This application improves the accuracy of eye use behavior monitoring.

Description

Technical Field

[0001] This application relates to the field of wearable device technology, and in particular to a method for monitoring eye behavior, smart glasses, and a readable storage medium. Background Technology

[0002] Currently, with the widespread use of electronic devices and increasing academic pressure, the visual burden on children and adolescents is becoming increasingly heavy, and myopia is showing a serious trend of high incidence and younger age of onset. Prolonged close-range visual activities are widely recognized as a key cause of vision decline.

[0003] Traditional eye use behavior monitoring solutions typically accumulate near-field eye use time by collecting user status information. Once the time reaches a set threshold, an alert is issued. After detecting that the user has stopped near-field eye use, the accumulated time is directly reset to zero, thus completing a single monitoring and alert process.

[0004] The aforementioned monitoring methods cannot guarantee the accuracy of eye use behavior monitoring. When users briefly interrupt close-range eye use, the time spent is immediately reset and the monitoring is restarted, which can easily lead to discontinuous eye use statistics, inaccurate judgment of intervention timing, and an inability to fully and accurately reflect the user's actual eye use, thus affecting the effectiveness of eye use behavior monitoring and intervention. Therefore, improving the accuracy of eye use behavior monitoring has become an urgent technical problem to be solved. Summary of the Invention

[0005] The main objective of this application is to provide a method for monitoring eye use behavior, smart glasses, and a readable storage medium, aiming to solve the technical problem of how to improve the accuracy of eye use behavior monitoring.

[0006] To achieve the above objectives, this application provides an eye-use behavior monitoring method for use in smart glasses, the eye-use behavior monitoring method comprising the following steps:

[0007] The system collects user eye behavior data using multimodal sensors and accumulates the duration of close-range eye use based on the eye behavior data. When the accumulated near-field eye use time reaches a preset reminder threshold, a reminder signal is output; In response to the reminder signal, the user's relaxation behavior is monitored by the multimodal sensor, including looking into the distance, outdoor activities, and light activities. When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the accumulated near-field eye use time will be reset to zero in order to re-monitor the user's eye use behavior.

[0008] In one embodiment, the multimodal sensor includes a motion sensor, an ambient light sensor, a distance sensor, and an infrared imaging sensor. The step of monitoring the user's relaxation behavior through the multimodal sensor includes: The system acquires first motion data collected by the motion sensor, ambient light intensity collected by the ambient light sensor, first target distance data collected by the distance sensor, and eye infrared image data collected by the infrared imaging sensor. If the first motion data conforms to preset periodic gait characteristics and the ambient light intensity conforms to preset outdoor environment switching characteristics, then it is determined that the user's outdoor activity behavior has been detected. If the first motion data conforms to the preset non-periodic body displacement characteristics, and the change range of the first target distance data is less than or equal to the preset threshold, then it is determined that the user's slight activity behavior has been detected. If the first motion data matches the preset head posture characteristics for looking into the distance, the first target distance data is greater than the preset threshold, and the eye infrared image data matches the preset characteristics for switching to using the eyes at a distance, then it is determined that the user's looking-in-the-distance behavior has been detected.

[0009] In one embodiment, after the steps of acquiring the first motion data collected by the motion sensor, the ambient light intensity collected by the ambient light sensor, the first target distance data collected by the distance sensor, and the eye infrared image data collected by the infrared imaging sensor, the method further includes: Based on the data streams from each sensor, the logical flow of determining outdoor activity behavior, light activity behavior, and distant viewing behavior is executed in parallel. When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the parallel judgment process for other relaxation behaviors is terminated until the statistical value of near-field eye use duration is cleared to zero, and then the monitoring status of each relaxation behavior is reset.

[0010] In one embodiment, the step of collecting user eye behavior data through a multimodal sensor and accumulating the near-field eye use time based on the eye behavior data to obtain the near-field eye use duration includes: The motion sensor collects the user's second motion data. Based on the second motion data, determine the duration during which the head pitch angle is less than or equal to a preset angle threshold; If the duration exceeds a preset time threshold, the distance sensor is activated to collect distance data to the second target. If the user's viewing distance is less than or equal to a preset distance threshold based on the second target distance data, the user is determined to have entered a near-field viewing state, and the near-field viewing time is accumulated.

[0011] In one embodiment, the multimodal sensor further includes a visible light imaging sensor, and after the step of accumulating near-field eye use time based on the eye use behavior data, the method further includes: The visible light imaging sensor is activated to acquire RGB image data of the eye; During the accumulation of near-field eye use time, if the head tilt angle is detected to be greater than a preset threshold based on the motion data, or if the user is not in an effective eye use state based on the eye RGB image data, the accumulation of near-field eye use time is paused. The cumulative timing of near-field eye use will resume once the user is detected to be in an effective eye use state and the duration exceeds a preset threshold.

[0012] In one embodiment, the step of identifying that the user is not in an effective eye-use state based on the eye RGB image data includes: If the eyelids are found to be closed or the gaze is deviated from the preset target gaze area based on the eye RGB image data, it is determined that the user is not in an effective eye use state. If the eyelids are found to be in an open state based on the RGB image data of the eye, and the gaze is focused on a preset target area, then the user is determined to be in an effective eye use state.

[0013] In one embodiment, after the step of outputting a reminder signal when the accumulated near-field eye use time reaches a preset reminder threshold, the method further includes: If no relaxation behavior of the user is detected within the first preset time period, an enhanced reminder signal will be output; If no relaxation behavior of the user is detected within the second preset time period, an alert signal is sent to the preset monitoring terminal device, wherein the second preset time period is longer than the first preset time period.

[0014] In one embodiment, after the step of accumulating near-field eye use time based on the eye use behavior data, the method further includes: During the cumulative duration of near-field eye use, the stability of the user's gaze focus is monitored in real time; When the stability of eye focus is detected to be less than the preset stability threshold, it is determined that the user is in a state of non-focused eye use, and the cumulative rate of the near-field eye use time is reduced by the first compensation coefficient. When the detected gaze focus stability is greater than or equal to the preset stability threshold, the normal cumulative rate of the near-field eye use duration is restored. The gaze focus stability is determined based on the degree of gaze point dispersion collected by the eye-tracking sensor in the multimodal sensor.

[0015] In addition, to achieve the above objectives, this application also provides a smart glasses, which includes a multimodal sensor and a processor; The multimodal sensor is used to collect users' eye behavior data; The processor is used to perform the steps of the eye behavior monitoring method described above.

[0016] In addition, to achieve the above objectives, this application also provides a readable storage medium, which is a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps of the eye behavior monitoring method described above.

[0017] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the eye behavior monitoring method described above.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: This application embodiment collects user eye behavior data through multimodal sensors in smart glasses, cumulatively calculating the duration of near-field eye use and outputting a reminder signal when the cumulative duration reaches a preset reminder threshold. After issuing the reminder, instead of simply resetting the duration based on interrupted eye use, the multimodal sensors continue to monitor the user's relaxation behaviors. Only when effective relaxation behaviors advocated by public health guidelines, such as looking into the distance, outdoor activities, and light activity, are identified and the duration meets the threshold are the cumulative duration reset to zero and the monitoring process restarted. This solution, through multi-sensor fusion, overcomes the limitations of focusing only on a single rest dimension and identifying a single scenario. It can perform refined identification and judgment of relaxation behaviors in different scenarios, effectively avoiding misjudging effective rest due to brief interruptions in eye use, thereby reducing problems such as discontinuous duration statistics and imprecise monitoring logic. Through the above-mentioned relaxation behavior judgment mechanism, the authenticity and continuity of eye use duration statistics are ensured, the accuracy of eye behavior monitoring is improved, the timing of intervention is more accurate, and the overall monitoring and intervention mechanism is more in line with scientific eye use guidelines, effectively enhancing the practical effect of eye behavior management and myopia prevention. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the eye use behavior monitoring method of this application; Figure 2 This is a schematic diagram of the sensor setup involved in an embodiment of the eye behavior monitoring method of this application; Figure 3 This is a schematic diagram of the eye use behavior monitoring process in one embodiment of the eye use behavior monitoring method of this application; Figure 4 This is a schematic diagram of the system architecture of the smart glasses in this application; Figure 5 This is a schematic diagram of the hardware operating environment of the eye behavior monitoring method device in the embodiments of this application.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] With the widespread use of electronic devices and increasing academic burden, the myopia rate among children and adolescents is showing a year-on-year upward trend. The "Technical Guidelines for Comprehensive Public Health Intervention in the Prevention and Control of Myopia in Children and Adolescents" recommends that primary school students relax their eyes through one or more methods after 40 minutes of continuous reading and writing, such as looking into the distance, light activity, outdoor activities, or eye exercises. However, in practice, there is a lack of effective supervision methods, making it difficult for parents and teachers to monitor whether children's eye-use behavior is in compliance with regulations.

[0025] Some eye monitoring devices have emerged in the prior art, mainly including the following categories: (1) Poor posture reminder devices based on distance sensors, which only focus on eye distance and ignore eye duration; (2) Posture monitoring devices based on acceleration sensors, which can detect postures such as looking down but cannot distinguish between real relaxation behavior and short pauses; (3) Eye tracking systems based on cameras, which can accurately analyze the direction of gaze but have high power consumption and are difficult to use for a long time in the daily wear scenarios of children.

[0026] The main problems and shortcomings of the existing solutions include: (1) the monitoring dimensions are limited, with most devices only monitoring eye distance or head posture, making it impossible to comprehensively determine whether various relaxation methods meet the requirements of the guidelines; (2) the ability to recognize relaxation behaviors is insufficient, failing to distinguish between different relaxation types such as "looking into the distance," "outdoor activities," and "light activities"; and (3) the lack of a complete monitoring loop, with the process from eye usage timing and reminder triggering to relaxation confirmation and timing reset not yet forming an effective linkage. The technical difficulties in solving the above problems are: how to achieve reliable recognition of multiple types of relaxation behaviors with limited hardware resources, how to obtain sufficient eye usage behavior information without infringing on privacy, and how to achieve a balance between low power consumption and high accuracy.

[0027] To address the aforementioned issues, the main solution of this application is as follows: A multimodal sensor is used to collect user eye-use behavior data, and the duration of near-field eye use is accumulated based on this data. When the accumulated near-field eye use duration reaches a preset reminder threshold, a reminder signal is output. In response to the reminder signal, the multimodal sensor monitors the user's relaxation behaviors, including looking into the distance, outdoor activities, and light activities. When the duration of any of these relaxation behaviors reaches a preset threshold, the accumulated near-field eye use duration is reset to zero, and the user's eye-use behavior is monitored again.

[0028] This application collects user eye-use behavior data through multimodal sensors in smart glasses, enabling cumulative statistics of near-field eye use time and outputting an alert signal when the cumulative time reaches a preset threshold. After issuing the alert, instead of simply resetting the time based on interrupted eye use, the multimodal sensors continue to monitor the user's relaxation behaviors. Only when effective relaxation behaviors advocated by public health guidelines, such as looking into the distance, outdoor activities, and light activity, are identified and the duration meets the threshold, is the cumulative time reset to zero and the monitoring process restarted. This solution, through multi-sensor fusion, overcomes the limitations of existing technologies that focus only on a single rest dimension and identify only a single scenario. It can perform refined identification and judgment of relaxation behaviors in different scenarios, effectively avoiding misjudging effective rest due to brief interruptions in eye use, thereby reducing problems such as discontinuous time statistics and imprecise monitoring logic. Through the above-mentioned relaxation behavior judgment mechanism, the authenticity and continuity of eye use time statistics are ensured, the accuracy of eye use behavior monitoring is improved, the timing of intervention is more accurate, and the overall monitoring and intervention mechanism is more in line with scientific eye use guidelines, effectively enhancing the practical effect of eye use behavior management and myopia prevention.

[0029] It should be noted that the implementing entity of the various embodiments of the eye behavior monitoring method of this application can be a smart glasses capable of realizing the above functions.

[0030] Based on this, this application proposes a method for monitoring eye use behavior according to a first embodiment. In this embodiment, the method for monitoring eye use behavior is applied to smart glasses, as shown below. Figure 1 As shown, the eye use behavior monitoring method includes the following steps S10~S40: Step S10: Collect user eye behavior data through a multimodal sensor, and accumulate near-field eye use time based on the eye behavior data; This multimodal sensor may include, but is not limited to, distance sensors, motion sensors, ambient light sensors, cameras, etc., and is used to comprehensively perceive multi-dimensional information such as changes in the user's eye and head posture, viewing distance, head movement trajectory, ambient light intensity, and user activity status.

[0031] Based on the collected eye behavior data, it is determined whether the user is in a close-range eye use state. For example, the distance sensor detects whether the distance between the eye and the target object is less than a set threshold, and the motion sensor determines whether the user's head posture is still or in a low-motion state, thereby accurately identifying close-range eye use behavior and accumulating the duration of the state to obtain the close-range eye use time.

[0032] Step S20: When the accumulated near-field eye use time reaches a preset reminder threshold, a reminder signal is output; The system compares the cumulative duration of close-range screen time with a preset reminder threshold. This preset threshold can be personalized based on the user's age, eye habits, vision health status, or relevant public health guidelines, and can be customized by the user or dynamically adjusted by the system.

[0033] When the cumulative time reaches or exceeds the reminder threshold, the reminder mechanism is triggered. This can be achieved by outputting voice prompts through the built-in speaker of the smart glasses, generating tactile feedback through the vibration motor, or projecting visual prompts through the display module, so as to remind the user to stop close-range eye use and take appropriate rest, thus avoiding damage to vision caused by prolonged continuous eye use.

[0034] Step S30: In response to the reminder signal, monitor the user's relaxation behavior through the multimodal sensor. The relaxation behavior includes looking into the distance, outdoor activities, and light activities. In response to the output alert signal, the system switches to relaxation behavior monitoring mode. In this mode, multimodal sensors continue to collect subsequent behavioral data from the user, and a refined recognition algorithm based on multi-sensor fusion is used to determine in real time whether the user has performed relaxation behavior.

[0035] This relaxation behavior specifically includes looking into the distance, outdoor activities, and light activities, comprehensively covering the main effective rest methods advocated by public health guidelines.

[0036] Step S40: When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the accumulated near-field eye use time is cleared to zero, so as to re-monitor the user's eye use behavior.

[0037] During the relaxation behavior monitoring process, when the system detects that the user has performed any of the following behaviors: gazing into the distance, outdoor activities, or light activities, and the duration of this behavior reaches a preset threshold, the system determines that the user has completed a valid rest behavior.

[0038] The preset threshold can be set according to the public health guidelines' recommendations on the duration of a single effective rest session, such as 1 minute, 5 minutes, or 10 minutes, and supports personalized adjustments. At this point, the system will reset the accumulated near-vision time to zero and reset the timer to restart the accumulation and monitoring of near-vision time, thus entering the next cycle of eye behavior management and ensuring the continuity and scientific nature of eye monitoring.

[0039] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description and will not be repeated hereafter. Based on this, the multimodal sensor includes a motion sensor, an ambient light sensor, a distance sensor, and an infrared imaging sensor. The step of monitoring the user's relaxation behavior through the multimodal sensor includes: Step A10: Acquire the first motion data collected by the motion sensor, the ambient light intensity collected by the ambient light sensor, the first target distance data collected by the distance sensor, and the eye infrared image data collected by the infrared imaging sensor; This motion sensor is used to collect user motion state data, including acceleration, angular velocity, and head posture, to identify the user's gait characteristics, body displacement, and head pitch angle. For example, an IMU (Inertial Measurement Unit) sensor can be used.

[0040] The Ambient Light Sensor (ALS) is used to continuously monitor changes in the light intensity of the user's environment in order to determine whether the user is switching between indoor and outdoor environments.

[0041] This distance sensor is used to collect distance data between the user's eyes and the target object in front of them, in order to determine changes in the user's gaze distance. For example, a ToF (Time of Flight) distance sensor can be used.

[0042] This infrared imaging sensor is used to collect infrared image data of the user's eyes, and to calculate eye activity characteristics such as gaze direction and convergence angle, in order to analyze changes in the user's gaze state.

[0043] Step A20: If the first motion data conforms to the preset periodic gait characteristics and the ambient light intensity conforms to the preset outdoor environment switching characteristics, then it is determined that the user's outdoor activity behavior has been detected. When the first motion data matches a preset periodic gait characteristic, it indicates that the user is engaging in periodic rhythmic walking or similar movement. For example, this characteristic can be set to a movement frequency within the range of 1-3 Hz and a duration exceeding 10 seconds. Simultaneously, when the ambient light intensity matches a preset outdoor environment switching characteristic, it indicates that the user's environment has switched from indoors to outdoors, with a significant increase in ambient light intensity. For example, this characteristic can be set to a jump in light intensity from less than 500 lux to more than 1000 lux.

[0044] If both of the above conditions are met simultaneously, the user's outdoor activity behavior is confirmed. This identification logic can accurately distinguish between a user's non-periodic indoor activities and periodic walking outdoors, avoiding misjudgments.

[0045] Step A30: If the first motion data conforms to the preset non-periodic body displacement characteristics and the change amplitude of the first target distance data is less than or equal to the preset threshold, then it is determined that the user's slight activity behavior has been detected. When the first motion data matches the preset non-periodic body displacement characteristics, it indicates that the user is engaged in non-periodic physical activity, such as stretching, turning the head, twisting, or other low-intensity movements. For example, this characteristic can be determined through variance analysis of motion sensor data, and it is identified when the activity level exceeds a preset resting threshold. Simultaneously, when the first target distance data is less than or equal to a preset threshold, it indicates that the distance between the user and the object in front has not significantly increased, and the user is still in a non-close-range visual state. For example, this threshold can be set to 50cm.

[0046] If both of the above conditions are met, the system detects subtle physical activity in the user. This identification logic effectively captures low-intensity physical activity performed by the user during rest periods, recognizing it as a form of effective relaxation.

[0047] Step A40: If the first motion data matches the preset head posture characteristics for looking into the distance, the first target distance data is greater than the preset threshold, and the eye infrared image data matches the preset characteristics for switching to using the eyes at a distance, then it is determined that the user's looking-in-the-distance behavior has been detected.

[0048] When the first motion data matches the preset head posture characteristics for looking into the distance, it indicates that the user's head is in a stable or slightly upward posture, which is consistent with the typical head posture when looking into the distance. For example, this feature can be set to a head tilt angle greater than -5°. When the first target distance data is greater than a preset threshold, it indicates that the target object being looked at by the user is relatively far away, meeting the basic distance conditions for looking into the distance. For example, this threshold can be set to 2 meters. If the user is in an indoor environment, the determination can be mainly based on the direction of the line of sight.

[0049] When the infrared image data of the eye meets the preset feature of switching to distant vision, it indicates that the user's eyes tend to be parallel, which is consistent with the physiological characteristics of the eyes being relaxed when looking into the distance. As an example, this feature can be set to switch the line of sight from near to far, such as the convergence angle changing from more than 10° to less than 2°.

[0050] When all of the above conditions are met simultaneously, the user's gazing behavior is detected, thus achieving accurate identification of gazing relaxation behavior.

[0051] This embodiment achieves refined identification of different relaxation behaviors through a multimodal fusion mechanism using motion sensors, ambient light sensors, distance sensors, and infrared imaging sensors. For outdoor activities, the ambient light sensor monitors abrupt changes in indoor and outdoor light intensity, and the motion sensor detects periodic gait characteristics to accurately distinguish between outdoor walking and indoor activities. For light activity, the motion sensor captures low-intensity activity through variance analysis of non-periodic body displacement, while the distance sensor confirms that the distance between the user and objects in front has not significantly increased, thus accurately identifying short-term rest behaviors while seated. For gazing into the distance, the mechanism integrates motion sensor judgment of head posture, distance sensor measurement of gaze distance, and infrared imaging sensor analysis of eye activity based on convergence angle changes, comprehensively verifying the behavior from three dimensions: head posture, gaze distance, and eye physiological characteristics. This multi-sensor fusion mechanism fully leverages the unique advantages of various sensors, achieving comprehensive coverage and accurate identification of various relaxation behaviors advocated by public health guidelines, such as gazing into the distance, outdoor activities, and light activity. Compared to solutions that rely on a single sensor and have limited recognition dimensions, this method, through cross-validation of multi-dimensional data, effectively avoids the problem of invalid rest misidentification caused by misjudgment from a single sensor. This improves the accuracy and robustness of relaxation behavior recognition, providing solid technical support for the accuracy of eye use behavior monitoring and ensuring the authenticity and continuity of eye use duration statistics. In one possible implementation, after the steps of acquiring the first motion data collected by the motion sensor, the ambient light intensity collected by the ambient light sensor, the first target distance data collected by the distance sensor, and the eye infrared image data collected by the infrared imaging sensor, the method further includes: Step B10: Based on the data streams from each sensor, execute the logic flow of outdoor activity behavior determination, light activity behavior determination, and distant viewing behavior determination in parallel. After acquiring multi-dimensional data from each sensor, the logical processes for determining outdoor activity behavior, minor activity behavior, and distant viewing behavior are executed in parallel based on the data streams from each sensor.

[0052] Independent decision threads or modules are established for each of the three relaxation behaviors, and data from each sensor is analyzed and processed in real time. These decision processes execute in parallel without interference, ensuring that each relaxation behavior can be identified and determined promptly and independently.

[0053] Step B20: When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the parallel judgment process of other relaxation behaviors is terminated until the near-field eye use duration statistics are cleared to zero, and then the monitoring status of each relaxation behavior is reset.

[0054] Once a relaxation behavior is confirmed as effective rest, the continuous monitoring and assessment of the other two relaxation behaviors will cease to avoid unnecessary consumption of computational resources and logical conflicts. Subsequently, the accumulated near-vision time will be reset to zero, and the eye-use behavior monitoring process will be restarted.

[0055] After the statistics on near-field eye use duration are reset to zero, the monitoring status of each relaxation behavior is reset, including clearing intermediate data from each judgment module, resetting timers, and reinitializing judgment parameters, in preparation for the next eye use behavior management cycle. This mechanism ensures a logical closed loop between eye use monitoring and relaxation behavior judgment, achieving orderly connection and efficient operation of the monitoring process.

[0056] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, the step of collecting user eye behavior data through a multimodal sensor and accumulating the near-field eye use time based on the eye behavior data to obtain the near-field eye use time includes: Step C10: Collect the user's second motion data using the motion sensor; Step C20: Based on the second motion data, determine the duration during which the head pitch angle is less than or equal to a preset angle threshold. Based on the second motion data, the duration of head tilt angles less than or equal to a preset angle threshold is extracted. When a user lowers their head for close-range visual activities, the head tilt angle usually shows a significant negative change. By setting an angle threshold (e.g., -30°), it is possible to identify whether the user is in a head-down posture. Simultaneously, the duration of this posture is accumulated to obtain the overall duration.

[0057] Step C30: If the duration exceeds a preset time threshold, then activate the distance sensor to collect distance data of the second target. If the duration exceeds a preset time threshold, the distance sensor is activated to collect target distance data, thus obtaining the second target distance data. This preset time threshold is used to exclude brief head-down movements, avoiding frequent activation of the distance sensor due to instantaneous posture changes.

[0058] As an example, this time threshold can be set to 5 seconds or 10 seconds. When the continuous head-down time exceeds this threshold, it is determined that the user may be entering a close-range eye-use state, and the distance sensor is then activated for further verification.

[0059] Step C40: If the user's eye distance is less than or equal to a preset distance threshold based on the second target distance data, then the user is determined to have entered a near-field eye use state, and the near-field eye use time is accumulated.

[0060] A distance sensor collects distance data between the user's eyes and a target object in front of them. When this distance is less than or equal to a preset distance threshold (e.g., 30cm or 50cm), it indicates that the user is engaging in close-range visual activity. At this point, the duration of close-range visual activity begins to accumulate until the user exits this state.

[0061] Through the above-mentioned hierarchical judgment mechanism, a low-power motion sensor is first used for preliminary screening, and the distance sensor is only activated for accurate measurement after the head posture condition is met. This not only ensures the accuracy of close-range eye use status recognition, but also effectively reduces the power consumption caused by the continuous operation of multiple sensors.

[0062] In one possible implementation, the multimodal sensor further includes a visible light imaging sensor, and after the step of accumulating near-field eye use time based on the eye use behavior data, the method further includes: Step D10: Activate the visible light imaging sensor to acquire RGB image data of the eye; After accumulating near-vision time, the visible light imaging sensor is activated to collect RGB image data of the eye. This visible light imaging sensor is used to acquire color images of the user's eye area. For example, the visible light imaging sensor can take a quick picture of the eye every minute to facilitate subsequent analysis of the user's eye usage status, including the degree of eyelid opening and closing, gaze direction, etc., providing visual data support for determining whether the user is in an effective eye usage state.

[0063] Step D20: During the accumulation of near-field eye use time, if the head tilt angle is detected to be greater than a preset threshold based on the motion data, or if the user is not in an effective eye use state based on the eye RGB image data, then the accumulation of near-field eye use time is paused. During the accumulation of near-field eye use time, if the head tilt angle is detected to be greater than a preset threshold based on motion data, or if the user is not in an effective eye use state based on eye RGB image data, the accumulation of near-field eye use time will be paused.

[0064] A head tilt angle greater than a preset threshold (e.g., greater than -5°) indicates that the user has raised their head, potentially interrupting near-field visual activity. Simultaneously, based on RGB image data of the eyes, an image recognition algorithm performs refined verification of the user's visual state. For example, the image recognition algorithm determines whether the eyelids are closed or whether the gaze has been deviated from the desk area for an extended period. When these conditions are detected, it indicates that the user is not in an effective visual state.

[0065] If any of the above conditions are met, the time accumulation will be suspended to avoid including non-actual screen time in the screen time calculation and to ensure the accuracy of the time statistics.

[0066] Step D30: Until the user is detected to be in an effective eye use state and the duration exceeds a preset threshold, the cumulative timing of the near-field eye use duration is restored.

[0067] After pausing the cumulative timer, the system continuously monitors the user's eye usage status until it detects that the user is in an effective eye usage state and the duration exceeds a preset threshold. Then, the cumulative timer for the near-field eye usage duration is resumed.

[0068] When both motion data and eye RGB image data meet the conditions for effective eye use, it is determined that the user has re-entered a state of effective eye use.

[0069] Furthermore, to avoid frequent starts and stops caused by brief posture adjustments or gaze shifts, the duration of effective eye use can be required to exceed a preset threshold (e.g., 2 or 3 seconds) before time accumulation resumes. This mechanism effectively filters out brief interruptions during eye use, ensuring the continuity and rationality of eye use time statistics.

[0070] In addition, if no effective eye use is detected within the preset time period, the close-range eye use time can be reset to zero to restart the monitoring of eye use behavior.

[0071] This embodiment, based on the initial determination of near-field eye use status by a motion sensor, further introduces a visible light imaging sensor for refined verification of eye use status, and combines this with head posture analysis to achieve accurate accumulation of eye use duration. By periodically capturing and recognizing eye images, it effectively prevents users from circumventing monitoring by closing their eyes or lying on their side, ensuring the authenticity and reliability of eye use behavior data. Compared to solutions relying solely on a single sensor, it effectively excludes ineffective eye use periods such as resting with eyes closed, brief head tilts, and gaze deviations, avoiding miscalculations of eye use duration and thus improving the accuracy and scientific rigor of eye use behavior monitoring.

[0072] In one possible implementation, the step of identifying that the user is not in an effective eye-use state based on the eye RGB image data includes: Step E10: If the eyelids are found to be closed or the line of sight is deviated from the preset target gaze area based on the eye RGB image data, it is determined that the user is not in an effective eye use state. Image recognition algorithms are used to analyze RGB images of the eye captured by a visible light imaging sensor to detect the opening and closing state of the eyelids. When the eyelids are detected to be closed, it indicates that the user is blinking, resting with their eyes closed, or dozing off, and is not actually engaging in close-range visual activities.

[0073] Simultaneously, the system extracts the user's gaze direction through eye image analysis and compares it with a preset target gaze area. This preset target gaze area can be pre-defined based on the user's visual activity, such as a desk area, a book area, or an electronic screen area. When the system detects that the user's gaze has deviated from this preset area, it indicates that the user's gaze has left the target and they are not in an effective visual state.

[0074] If any of the above situations are detected, it is determined that the user is not in an effective eye-use state, and the logic of pausing the accumulation of close-range eye-use time is triggered.

[0075] Step E20: If the eyelids are found to be in an open state based on the RGB image data of the eye, and the gaze is focused on the preset target gaze area, then the user is determined to be in an effective eye use state.

[0076] When the image recognition algorithm detects that the eyelids are not closed (i.e., the eyes are open) and the user's gaze is focused on the preset target gaze area, it indicates that the user is looking at the target and has the basic conditions to carry out close-range visual activities.

[0077] If both of the above conditions are met simultaneously, the user is determined to be in an effective eye-use state. This determination mechanism combines dual verification of eye physiological state and gaze behavior, ensuring the accuracy and reliability of effective eye-use state identification.

[0078] In one possible implementation, after the step of accumulating near-vision time based on the eye-use behavior data, the method further includes: Step F10: If no relaxation behavior of the user is detected within the first preset time period, an enhanced reminder signal is output. Upon receiving the initial reminder signal, the relaxation behavior monitoring timer is activated. If, within the first preset duration (e.g., 5 minutes), the multimodal sensor fails to detect any relaxation behavior performed by the user, including gazing into the distance, outdoor activities, or light physical activity, it indicates that the user has not responded effectively to the initial reminder. In this case, an enhanced reminder signal is output to more strongly prompt the user to take a break.

[0079] This enhanced alert signal has a higher alert intensity than the initial alert signal. For example, it can be set to a louder voice prompt, stronger vibration feedback, or a more eye-catching visual cue, aiming to attract the user's full attention and prompt them to perform relaxation behavior.

[0080] Step F20: If no relaxation behavior of the user is detected within the second preset time period, a reminder signal is sent to the preset monitoring terminal device, wherein the second preset time period is longer than the first preset time period.

[0081] If no effective relaxation behavior is detected from the user within the second preset time period (e.g., 10 minutes), it indicates that the user may have ignored the reminder signal for an extended period and continued to overuse their eyes. In this case, a reminder signal can be sent to the preset monitoring terminal device to notify parents, teachers, or other guardians to pay attention to the user's eye use.

[0082] The preset monitoring terminal device can be a parent's smartphone, tablet, or other smart device that can receive reminder messages. The reminder signals can include information such as the user's screen time and reminders of unrelaxed behavior, so that the caregiver can intervene in a timely manner and guide the user to take necessary rest.

[0083] Through the aforementioned tiered reminder mechanism, this solution constructs a progressive intervention system, from user self-response to enhanced system reminders, and finally to guardian intervention. The first-level enhanced reminder guides users to actively perform relaxation behaviors with stronger prompts, while the second-level guardian reminder introduces external supervision when users continue to neglect eye health. This effectively avoids eye overuse problems caused by user non-cooperation and improves the reliability and effectiveness of eye behavior monitoring and intervention.

[0084] For example, to aid in understanding the technical concept or principle of the eye-use behavior monitoring method combined with the first and second embodiments described above, a specific embodiment is now provided. In this specific embodiment, the eye-use behavior monitoring method is applied to smart glasses, referring to... Figure 2 As shown, the smart glasses mainly include the glasses body and integrated sensor modules, processing units, vibration motors, and power modules. The sensor modules include: an inertial measurement unit (IMU) (for monitoring head posture and body movement), a ToF distance sensor (for monitoring eye distance), an ambient light sensor (ALS) (for determining lighting conditions and indoor / outdoor scenes), an infrared camera (for eye tracking), and an inward-facing RGB camera (for corneal reflection image analysis). Based on this, refer to... Figure 3 As shown, the eye use behavior monitoring process includes: (1) Determination and duration of continuous close-range visual use The processing unit continuously reads IMU sensor data in low-frequency mode to calculate the head pitch angle. When a head pitch angle of less than -30° (head-down posture) is detected for more than 10 seconds, the ToF distance sensor is triggered to detect the viewing distance. If the distance is consistently less than 30cm, it is determined that the user has entered a "close-range viewing state" and the timer begins to accumulate. If the user looks up (pitch angle greater than -5°) or the distance is greater than 50cm, the timer pauses; if the user returns to a close-range viewing state within 5 minutes, the timer continues to accumulate; otherwise, it is reset and restarts. To prevent users from avoiding monitoring by closing their eyes or lying on their side, the RGB camera takes a quick picture of the eyes every minute. The image recognition algorithm determines whether the eyelids are closed or whether the gaze has been deviated from the desk area for an extended period of time; if so, the timer is paused.

[0085] (2) Tiered reminders accumulated over 40 minutes When 40 minutes of close-range eye use accumulates, the processing unit controls the vibration motor to issue a reminder. If no relaxation behavior is detected within 5 minutes of the reminder, the vibration motor switches to a gradually increasing intensity mode until relaxation behavior is detected. If no relaxation behavior is detected after 10 minutes, a reminder is pushed to the parent's device via the app.

[0086] (3) Parallel monitoring of three types of relaxation behaviors: The system monitors the following three types of relaxation behaviors in parallel, and the specific identification logic is as follows: a. Outdoor activities The ALS sensor continuously monitors light intensity. When the value jumps from <500 lux to >1000 lux, it indicates that the user has moved from indoors to outdoors. At the same time, the IMU sensor detects periodic gait characteristics (such as walking frequency of 1-3Hz) that last for more than 10 seconds, which is determined to be outdoor activity behavior.

[0087] b. Minor activity The IMU sensor detected non-periodic body displacement (such as stretching, turning the head, or twisting), but the ToF sensor showed that the distance did not increase significantly (still within 50cm), indicating that the activity occurred while seated. Analysis of variance of the IMU data determined that activity levels exceeding the resting threshold were classified as mild activity.

[0088] c. Gazing into the distance The infrared camera captures eye images at 30fps and calculates the direction of gaze. When the gaze is detected to shift from near (convergence angle greater than 10°) to far (convergence angle less than 2°), and the ToF distance sensor detects a target distance greater than 2 meters (indoors, the gaze direction is mainly considered), combined with the head posture of maintaining a level or slightly tilted head (pitch angle greater than -5°), it is determined to be a distant gazing behavior.

[0089] (4) Relaxation of target confirmation and time reset For any of the above relaxation behaviors, the system continuously monitors its duration. When the behavior lasts for 1 minute continuously or cumulatively, it is considered a valid relaxation. The vibration motor issues a success signal, the processing unit resets the "near-field eye use timer" to zero, and monitoring resumes.

[0090] It should be noted that the above examples are only used to help understand this embodiment and do not constitute a limitation on the eye behavior monitoring process of this embodiment. Any simple modifications based on this technical concept are within the protection scope of this application.

[0091] Based on the first, second, and / or third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the above-described embodiments one, two, and three can be referred to the above description and will not be repeated hereafter. In addition, after the step of accumulating the near-field eye use time based on the eye use behavior data to obtain the near-field eye use time, the method further includes: Step G10: During the process of accumulating the near-field eye use time, monitor the stability of the user's gaze focus in real time; The system continuously collects the user's eye movement data, including the spatial location of the gaze point, gaze duration, and saccade trajectory, using an eye-tracking sensor within a multimodal sensor. Based on this data, the system calculates the dispersion of the gaze point to quantify the focusing stability of the user's gaze. The dispersion of the gaze point can be characterized by indicators such as the variance, standard deviation, or radius of dispersion of the gaze point coordinates. The smaller the dispersion, the more focused the user's gaze is, and the higher the focusing stability; the larger the dispersion, the more dispersed the user's gaze is, and the lower the focusing stability.

[0092] Step G20: When the detected gaze focus stability is less than the preset stability threshold, it is determined that the user is in a non-focused eye use state, and the cumulative rate of the near-field eye use time is reduced by the first compensation coefficient. When the dispersion of the gaze point exceeds a preset stability threshold, it indicates that the user's gaze is frequently drifting and their attention is scattered. Although they may still be in a near-field eye-use posture, they are not truly focused on the eye activity. At this time, the system determines that the user is in a state of unfocused eye use and uses a first compensation coefficient (e.g., 0.3 or 0.5) to attenuate the cumulative rate of eye use duration.

[0093] In other words, under this condition, the actual cumulative screen time is increased proportionally to the compensation coefficient at the normal rate, thereby avoiding the equating of inefficient and unfocused screen time with efficient screen time in the cumulative duration. The statistics on screen time more accurately reflect the user's effective screen load.

[0094] Step G30: When the detected gaze focus stability is greater than or equal to the preset stability threshold, restore the normal cumulative rate of the near-field eye use duration.

[0095] When the user's gaze refocuses and the dispersion of the fixation point decreases below the preset stability threshold, it indicates that the user has returned to a state of focused eye use. At this point, the compensation coefficient is discontinued, and the duration of focused eye use is accumulated at a normal rate to ensure that the focused eye use period is recorded completely and accurately.

[0096] Through the aforementioned dynamic cumulative rate adjustment mechanism based on gaze focus stability, this embodiment can effectively distinguish between a user's focused and unfocused eye use states, avoiding miscalculations of eye usage time caused by ineffective eye use behaviors such as distraction and daydreaming. Compared to schemes that rely solely on posture and distance for binary judgment, this embodiment introduces a refined assessment of eye use quality, making the accumulation of eye use time more scientific and reasonable, further improving the accuracy and reliability of eye use behavior monitoring.

[0097] Furthermore, embodiments of this application also propose a smart pair of glasses, referring to... Figure 4 As shown, the smart glasses include a multimodal sensor and a processor; The multimodal sensor is used to collect users' eye behavior data; The processor is used to perform the steps of the eye behavior monitoring method described above.

[0098] In addition, such as Figure 5 As shown, the smart glasses may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the smart glasses. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the smart glasses to communicate wirelessly or wiredly with other devices to exchange data. While the figures show smart glasses with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0099] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0100] The smart glasses provided in this application, employing the eye-use behavior monitoring method described in the above embodiments, can solve the technical problem of how to improve the accuracy of eye-use behavior monitoring. Compared with the prior art, the beneficial effects of the smart glasses provided in this application are the same as those of the eye-use behavior monitoring method provided in the above embodiments, and other technical features of the smart glasses are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0101] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0103] In addition, to achieve the above objectives, embodiments of this application also provide a readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the eye behavior monitoring method in the above embodiments.

[0104] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0105] The aforementioned computer-readable storage medium may be included in the smart glasses; or it may exist independently and not assembled into the smart glasses.

[0106] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by smart glasses, cause the smart glasses to perform the process steps of any embodiment of the above-described eye behavior monitoring method.

[0107] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the modules themselves.

[0110] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described eye behavior monitoring method, thereby solving the technical problem of how to improve the accuracy of eye behavior monitoring. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the eye behavior monitoring method provided in the above embodiments, and will not be repeated here.

[0111] Furthermore, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the eye behavior monitoring method described above.

[0112] The specific implementation of the computer program product in this application is basically the same as the embodiments of the above-described eye behavior monitoring method, and will not be repeated here.

[0113] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0114] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software sensor. This computer software sensor is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a smart glasses device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0116] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for monitoring eye use behavior, characterized in that, The eye behavior monitoring method, applied to smart glasses, includes the following steps: The system collects user eye behavior data using multimodal sensors and accumulates the duration of close-range eye use based on the eye behavior data. When the accumulated near-field eye use time reaches a preset reminder threshold, a reminder signal is output; In response to the reminder signal, the user's relaxation behavior is monitored by the multimodal sensor, including looking into the distance, outdoor activities, and light activities. When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the accumulated near-field eye use time will be reset to zero in order to re-monitor the user's eye use behavior.

2. The eye use behavior monitoring method as described in claim 1, characterized in that, The multimodal sensor includes a motion sensor, an ambient light sensor, a distance sensor, and an infrared imaging sensor. The step of monitoring the user's relaxation behavior through the multimodal sensor includes: The system acquires first motion data collected by the motion sensor, ambient light intensity collected by the ambient light sensor, first target distance data collected by the distance sensor, and eye infrared image data collected by the infrared imaging sensor. If the first motion data conforms to preset periodic gait characteristics and the ambient light intensity conforms to preset outdoor environment switching characteristics, then it is determined that the user's outdoor activity behavior has been detected. If the first motion data conforms to the preset non-periodic body displacement characteristics, and the change range of the first target distance data is less than or equal to the preset threshold, then it is determined that the user's slight activity behavior has been detected. If the first motion data matches the preset head posture characteristics for looking into the distance, the first target distance data is greater than the preset threshold, and the eye infrared image data matches the preset characteristics for switching to using the eyes at a distance, then it is determined that the user's looking-in-the-distance behavior has been detected.

3. The eye use behavior monitoring method as described in claim 2, characterized in that, After the steps of acquiring the first motion data collected by the motion sensor, the ambient light intensity collected by the ambient light sensor, the first target distance data collected by the distance sensor, and the eye infrared image data collected by the infrared imaging sensor, the method further includes: Based on the data streams from each sensor, the logical flow of determining outdoor activity behavior, light activity behavior, and distant viewing behavior is executed in parallel. When the duration of any of the relaxation behaviors is detected to reach a preset threshold, the parallel judgment process for other relaxation behaviors is terminated until the statistical value of near-field eye use duration is cleared to zero, and then the monitoring status of each relaxation behavior is reset.

4. The eye use behavior monitoring method as described in claim 2, characterized in that, The step of collecting user eye behavior data through a multimodal sensor and accumulating the near-field eye use time based on the eye behavior data to obtain the near-field eye use time includes: The motion sensor collects the user's second motion data. Based on the second motion data, determine the duration during which the head pitch angle is less than or equal to a preset angle threshold; If the duration exceeds a preset time threshold, the distance sensor is activated to collect distance data to the second target. If the user's viewing distance is less than or equal to a preset distance threshold based on the second target distance data, the user is determined to have entered a near-field viewing state, and the near-field viewing time is accumulated.

5. The eye use behavior monitoring method as described in claim 4, characterized in that, The multimodal sensor further includes a visible light imaging sensor, and after the step of accumulating near-field eye use time based on the eye use behavior data, the method further includes: The visible light imaging sensor is activated to acquire RGB image data of the eye; During the accumulation of near-field eye use time, if the head tilt angle is detected to be greater than a preset threshold based on the motion data, or if the user is not in an effective eye use state based on the eye RGB image data, the accumulation of near-field eye use time is paused. The cumulative timing of near-field eye use will resume once the user is detected to be in an effective eye use state and the duration exceeds a preset threshold.

6. The eye use behavior monitoring method as described in claim 4, characterized in that, The step of identifying that the user is not in an effective eye-use state based on the RGB image data of the eye includes: If the eyelids are found to be closed or the gaze is deviated from the preset target gaze area based on the eye RGB image data, it is determined that the user is not in an effective eye use state. If the eyelids are found to be in an open state based on the RGB image data of the eye, and the gaze is focused on a preset target area, then the user is determined to be in an effective eye use state.

7. The eye use behavior monitoring method as described in claim 1, characterized in that, After the step of outputting a reminder signal when the accumulated near-field eye use time reaches a preset reminder threshold, the method further includes: If no relaxation behavior of the user is detected within the first preset time period, an enhanced reminder signal will be output; If no relaxation behavior of the user is detected within the second preset time period, an alert signal is sent to the preset monitoring terminal device, wherein the second preset time period is longer than the first preset time period.

8. The eye use behavior monitoring method according to any one of claims 1 to 7, characterized in that, Following the step of accumulating near-field screen time based on the eye-use behavior data, the method further includes: During the cumulative duration of near-field eye use, the stability of the user's gaze focus is monitored in real time; When the stability of eye focus is detected to be less than the preset stability threshold, it is determined that the user is in a state of non-focused eye use, and the cumulative rate of the near-field eye use time is reduced by the first compensation coefficient. When the detected gaze focus stability is greater than or equal to the preset stability threshold, the normal cumulative rate of the near-field eye use duration is restored. The gaze focus stability is determined based on the degree of gaze point dispersion collected by the eye-tracking sensor in the multimodal sensor.

9. A type of smart glasses, characterized in that, The smart glasses include a multimodal sensor and a processor; The multimodal sensor is used to collect users' eye behavior data; The processor is configured to perform the steps of the eye behavior monitoring method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a device control program, which, when executed by a processor, implements the steps of the eye behavior monitoring method as described in any one of claims 1 to 8.

Images