Eye condition monitoring method and apparatus, electronic device, and storage medium

Abstract

This invention provides an eye condition detection method, device, electronic device, and storage medium, relating to the interdisciplinary fields of in-vehicle intelligent driving assistance and medical health. The eye condition detection method includes: acquiring an initial image sequence including the eye region of a target user; extracting eye feature parameters of the target user based on the initial image sequence; wherein the eye feature parameters include eye opening and closing degree and pupil visibility; determining the duration of a single eye closure based on the eye feature parameters; and calculating the effective blink frequency of the target user based on the duration of the single eye closure; if frequent blinking is detected based on the effective blink frequency, an in-vehicle medical auxiliary diagnosis mechanism is triggered. This invention can accurately detect driver's eye discomfort and provide accurate medical auxiliary diagnosis, improving the accuracy of eye condition detection results and ensuring driving safety and the user's eye health.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of vehicle intelligent driving assistance and medical health, and in particular to a method, device, electronic device and storage medium for detecting eye condition. Background Technology

[0002] With the rapid development of automotive intelligence and connectivity, the in-vehicle driver monitoring system (DMS) has become a core component of the smart cockpit. Its core function is to monitor driver fatigue and distraction to ensure driving safety.

[0003] The core application of existing DMS systems focuses on fatigue detection, primarily triggering warnings by recognizing extreme fatigue actions such as prolonged eye closure or yawning. However, they fail to address subtle changes in the driver's eye condition. When drivers experience eye discomfort while driving, it can lead to decreased concentration and visual acuity, thereby increasing driving risks.

[0004] Current in-vehicle systems fail to accurately capture and interpret the frequent blinking signal, a common eye discomfort signal, and cannot provide drivers with real-time medical diagnosis and improvement suggestions. Furthermore, existing eye discomfort detection systems largely rely on specialized medical equipment, which cannot be implemented in real-time while driving. This makes it difficult for drivers to promptly detect eye discomfort and obtain professional support while driving, creating a dual threat to driving safety and health.

[0005] Therefore, developing an in-vehicle technology solution that can accurately detect driver eye discomfort and provide accurate medical auxiliary diagnosis has become an urgent need in the field of in-vehicle intelligent health. Summary of the Invention

[0006] This invention provides a method, device, electronic device, and storage medium for detecting eye condition, aiming to solve the problem of the inability to accurately detect driver eye discomfort during driving in real time.

[0007] This invention provides a method for detecting eye condition, comprising: Obtain an initial image sequence including the target user's eye region; Based on the initial image sequence, eye feature parameters of the target user are extracted; wherein, the eye feature parameters include eye opening and pupil visibility; The duration of a single eye closure is determined based on the eye feature parameters, and the effective blinking frequency of the target user is calculated based on the duration of the single eye closure. If frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered.

[0008] According to an eye state detection method provided by the present invention, the step of determining the duration of a single eye closure based on the eye feature parameters, and calculating the effective blink frequency of the target user based on the duration of the single eye closure, includes: Based on the eye opening and closing degree and the pupil visibility, determine whether the target user is in a closed-eye state; If the eye opening degree is less than a preset eye-closing threshold and the pupil visibility is less than a preset pupil visibility threshold, then the target user is determined to be in a closed-eye state, and the duration of the closed-eye state is obtained as the single eye-closing duration. If the duration of a single blink is within a preset duration range, it is determined to be a valid blink action; wherein, the preset duration range is 100 milliseconds to 500 milliseconds; The effective blinking frequency of the target user is obtained by counting the number of effective blinking actions within a preset statistical period.

[0009] According to an eye state detection method provided by the present invention, before determining whether the target user is in a closed-eye state based on the eye opening and closing degree and the pupil visibility, the method further includes: Obtain the first facial image sequence of the target user after he / she is seated; The head pose of the target user is obtained by performing head pose detection on the first facial image sequence using a facial landmark detection model. Based on the head posture angle and the preset frontal viewing angle range, extract the eye opening and closing degree sample set from the first facial image sequence; Extract the mean value of the preset proportion of samples with the smallest value in the eye opening and closing degree sample set, and use it as the initial eye-closing calibration threshold. If the initial eye-closing calibration threshold is less than the preset threshold, then the preset eye-closing threshold is set to a preset fixed value; If the initial eye-closing calibration threshold is greater than or equal to the preset threshold, then the preset ratio of the initial eye-closing calibration threshold is used as the preset eye-closing threshold.

[0010] According to the present invention, an eye state detection method is provided, wherein extracting eye feature parameters of the target user based on the initial image sequence includes: The initial image sequence is used to locate faces using a face detection model to obtain a second facial image sequence; Eye feature point detection is performed on the second facial image sequence to obtain the coordinates of the eye feature points; Based on the coordinates of the eye feature points, calculate the first distance between the upper and lower eyelids and the second distance between the inner and outer corners of the eyes; The eye opening degree is determined based on the first distance and the second distance; The pupil detection model is used to detect pupils in the second facial image sequence, obtain pupil feature points and their corresponding confidence scores, and use the confidence scores as the pupil visibility.

[0011] According to an eye state detection method provided by the present invention, if frequent blinking is detected based on the effective blinking frequency, an in-vehicle medical auxiliary diagnosis mechanism is triggered, including: If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, and interactive query information is output. Obtain the historical symptom information of the target user based on the interactive inquiry information; The blinking frequency and the historical symptom information are input into the medical auxiliary diagnostic model to obtain the eye discomfort status assessment results and improvement suggestions output by the medical auxiliary diagnostic model.

[0012] According to the eye condition detection method provided by the present invention, after inputting the blink frequency and the historical symptom information into a large-scale medical auxiliary diagnostic model to obtain the eye discomfort condition assessment results and improvement suggestions output by the large-scale medical auxiliary diagnostic model, the method further includes: Obtain the humidity inside the vehicle and the direction of the air vents of the vehicle's air conditioning system; When the humidity inside the vehicle is detected to be lower than a preset humidity threshold, or when the air vent is directed toward the face of the target user, a corresponding device driving command is generated. The device drive commands control the in-vehicle environment adjustment equipment to perform corresponding adjustment operations.

[0013] According to an eye state detection method provided by the present invention, the step of acquiring an initial image sequence including the eye region of a target user includes: An initial image sequence including the target user's eye region is acquired using a vehicle driver monitoring system (DMS) camera or an occupant monitoring system (OMS) camera at a preset sampling frequency; wherein the preset sampling frequency is in the range of 10 frames / second to 20 frames / second.

[0014] The present invention also provides an eye condition detection device, comprising: The image acquisition module is used to acquire an initial image sequence including the target user's eye region; The parameter extraction module is used to extract eye feature parameters of the target user based on the initial image sequence; wherein, the eye feature parameters include eye opening and pupil visibility; The blinking statistics module is used to determine the duration of a single eye closure based on the eye feature parameters, and to count the effective blinking frequency of the target user based on the duration of the single eye closure. The status detection module is used to trigger the vehicle-mounted medical auxiliary diagnosis mechanism if frequent blinking is detected based on the effective blinking frequency.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the eye state detection method as described in any of the preceding claims.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the eye state detection method as described in any of the preceding claims.

[0017] The eye state detection method, device, electronic device, and storage medium provided by this invention first acquire an initial image sequence including the eye region of a target user; then, based on the initial image sequence, extract eye feature parameters of the target user, including eye opening and closing degree and pupil visibility; furthermore, determine the duration of a single eye closure based on the eye feature parameters, and statistically analyze the effective blink frequency of the target user based on the duration of a single eye closure. By extracting eye feature parameters in two dimensions—eye opening and closing degree and pupil visibility—the invention overcomes the shortcomings of traditional single-feature methods, which are easily interfered with by factors such as the user looking down and changes in viewing angle. This allows for the high-precision calculation of the duration of a single eye closure, and the statistically analyzed effective blink frequency effectively filters out irrelevant eye movements, accurately identifying the core eye discomfort signal—frequent blinking—that is undetectable by the DMS system. Finally, if frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered. This allows users to obtain professional medical diagnosis and improvement suggestions as soon as they experience discomfort such as dry eyes or eye fatigue that leads to frequent blinking. This further ensures the accuracy of the eye condition detection results and avoids distraction and decreased visual clarity caused by eye discomfort, effectively protecting driving safety and the user's eye health. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts of the eye state detection method provided by the present invention.

[0020] Figure 2 This is the second flowchart of the eye state detection method provided by the present invention.

[0021] Figure 3 This is the third flowchart of the eye state detection method provided by the present invention.

[0022] Figure 4 This is the fourth flowchart of the eye state detection method provided by the present invention.

[0023] Figure 5 This is a schematic diagram of the location of 68 key points of a human face provided by the present invention.

[0024] Figure 6 This is a schematic diagram of the eye condition detection device provided by the present invention.

[0025] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] With the rapid development of automotive intelligence and connectivity, the in-vehicle driver monitoring system (DMS) has become a core component of the smart cockpit. Its core function is to monitor driver fatigue and distraction to ensure driving safety.

[0028] The core application of existing DMS systems focuses on fatigue detection, primarily triggering warnings by recognizing extreme fatigue actions such as prolonged eye closure and yawning. However, they fail to address subtle changes in the driver's eye condition, resulting in undetected eye discomfort. This shortcoming stems from the fact that existing technologies are designed solely for driving safety, neglecting driver health and lacking the ability to perform refined analysis of eye parameters. Furthermore, current technologies only define "eye closure duration ≥ 1 second" as the fatigue criterion, failing to clearly differentiate between short-duration eye closure (blinking) and long-duration eye closure (fatigue-induced eye closure). This makes it impossible to accurately count blinks and thus determine the eye discomfort associated with frequent blinking. This shortcoming arises from the relatively simple algorithm design of existing technologies, focusing only on "whether the eyes are closed," without finely categorizing the duration of eye closure and failing to incorporate medical criteria for frequent blinking.

[0029] In other words, current in-vehicle systems fail to accurately capture and determine the driver's frequent blinking, a signal of eye discomfort, and cannot provide the driver with real-time medical diagnosis and improvement suggestions. Furthermore, current eye discomfort detection relies heavily on specialized medical equipment, which cannot be performed in real-time while driving. This makes it difficult for drivers to promptly detect their own eye discomfort and obtain professional support while driving, creating a dual threat to driving safety and health.

[0030] When drivers experience eye discomfort while driving, it can lead to decreased concentration and visual acuity, thereby increasing driving risks. Therefore, developing an in-vehicle technology solution that can accurately detect driver eye discomfort and provide medical assistance in diagnosis has become an urgent need in the field of in-vehicle intelligent health.

[0031] This invention proposes a method, device, electronic device, and storage medium for detecting eye condition, which are described below in conjunction with... Figures 1-7 Describe it.

[0032] Figure 1 This is one of the flowcharts of the eye state detection method provided by the present invention, such as... Figure 1 As shown, the eye condition detection method includes steps S110, S120, S130 and S140.

[0033] Step S110: Obtain an initial image sequence including the target user's eye region.

[0034] In this embodiment of the invention, the executing entity can be an in-vehicle processor.

[0035] The onboard processor continuously acquires image frames, including those of the target user's eye area, and records this as the initial image sequence. The target user can be either the driver or a passenger.

[0036] The initial image sequence can be acquired during the vehicle's movement.

[0037] Step S120: Based on the initial image sequence, extract the eye feature parameters of the target user. These eye feature parameters include eye opening and pupil visibility.

[0038] Then, based on the acquired initial image sequence, the eye feature parameters of the target user are extracted. These eye feature parameters include eye opening and pupil visibility.

[0039] Eye Aspect Ratio (EAR) is a geometric feature index used to quantify the degree of eye opening in humans. It is obtained by calculating the ratio of the vertical distance between feature points on the upper and lower eyelids to the horizontal distance between feature points on the inner and outer corners of the eyes.

[0040] Pupil visibility, a confidence score output by the pupil detection model, characterizes the degree to which the camera can clearly see the pupil (iris). A value close to 1 indicates complete visibility, while a value close to 0 indicates complete obstruction by the eyelid.

[0041] Specifically, a face detection model is used to locate faces in the initial image sequence to obtain a second facial image sequence. Eye feature points are detected in the second facial image sequence to obtain their coordinates. Based on these coordinates, a first distance between the upper and lower eyelids and a second distance between the inner and outer corners of the eyes are calculated. The eye opening / closing degree is determined based on these first and second distances. A pupil detection model is used to detect pupils in the second facial image sequence, obtaining pupil feature points and their corresponding confidence scores. The confidence scores are used as pupil visibility. The specific execution process can be found in the following embodiment, which will not be elaborated upon here.

[0042] Furthermore, before extracting eye feature parameters from the initial image sequence, each image frame in the initial image sequence can be preprocessed. Preprocessing includes denoising and / or grayscale conversion.

[0043] In one embodiment, a Gaussian filtering algorithm is first used to denoise each image frame in the initial image sequence to remove noise interference from changes in in-vehicle lighting and the camera itself; then grayscale processing is performed to reduce the amount of data processing.

[0044] Step S130: Determine the duration of a single eye closure based on the eye feature parameters, and calculate the effective blinking frequency of the target user based on the duration of the single eye closure.

[0045] Effective blinking refers to the eyelid undergoing a complete cycle of "fully open-close-fully open," with the closed state lasting within a specific time threshold range (e.g., between 100 milliseconds and 500 milliseconds). It is used to eliminate fatigue-induced eye closure (prolonged eye closure) and short-term high-frequency image noise (extremely short eye closure).

[0046] The duration of a single eye closure is determined based on the extracted eye feature parameters. Then, based on this duration, invalid actions are filtered out, and the effective blinking frequency of the target user is counted.

[0047] Specifically, based on eye opening and pupil visibility, it is determined whether the target user is in a closed-eye state. If the eye opening is less than a preset closed-eye threshold and the pupil visibility is less than a preset pupil visibility threshold, the target user is determined to be in a closed-eye state, and the duration of the closed-eye state is obtained as the duration of a single closed-eye action. If the duration of a single closed-eye action is within a preset duration range, it is determined to be a valid blink action. The preset duration range is 100 milliseconds to 500 milliseconds. The number of valid blink actions within a preset statistical period is counted to obtain the effective blink frequency of the target user. The specific execution process can be referred to in the following embodiment, which will not be elaborated here.

[0048] The preset eye-closing threshold can be a fixed value pre-set and stored in memory based on the baseline eye opening and closing degree of a typical user, or it can be an adaptive eye-closing threshold derived from the facial image sequence of the seated user. The preset pupil visibility threshold is an empirical value obtained from calibration experiments and pre-stored in memory. The preset duration range is a fixed range pre-set and stored in memory based on the blinking and eye-closing duration range of typical users. The preset statistical period is a fixed value pre-set and stored in memory for ease of statistical analysis; for example, it could be 1 minute.

[0049] Step S140: If frequent blinking is detected based on the effective blinking frequency, the vehicle-mounted medical auxiliary diagnosis mechanism is triggered.

[0050] If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, and at this time, the vehicle medical auxiliary diagnosis mechanism is triggered.

[0051] Based on the medical definition, when the effective blinking count is greater than 20 times per minute, it is preliminarily judged as frequent blinking, indicating eye discomfort (such as eye strain, dryness, mild inflammation, etc.). Therefore, the preset frequent blinking frequency can be set to 20 times / minute based on the medical definition of frequent blinking and pre-stored in memory.

[0052] In one embodiment, the in-vehicle medical auxiliary diagnosis mechanism is as follows: the blinking frequency is input into the medical auxiliary diagnosis model to obtain the eye discomfort assessment results and improvement suggestions output by the medical auxiliary diagnosis model.

[0053] In another embodiment, the in-vehicle medical auxiliary diagnosis mechanism is as follows: outputting interactive inquiry information; obtaining historical symptom information fed back by the target user based on the interactive inquiry information; inputting the blinking frequency and historical symptom information into the medical auxiliary diagnosis big model, and obtaining the eye discomfort status assessment results and improvement suggestions output by the medical auxiliary diagnosis big model.

[0054] In another implementation, the user's historical symptom information is retrieved from a medical application; the blinking frequency and historical symptom information are input into a large-scale medical auxiliary diagnostic model to obtain the eye discomfort assessment results and improvement suggestions output by the large-scale medical auxiliary diagnostic model.

[0055] The eye state detection method provided in this invention first acquires an initial image sequence including the eye region of a target user; then, it extracts eye feature parameters of the target user based on the initial image sequence, wherein the eye feature parameters include eye opening and closing degree and pupil visibility; furthermore, it determines the duration of a single eye closure based on the eye feature parameters, and statistically analyzes the effective blink frequency of the target user based on the duration of a single eye closure. By extracting eye feature parameters in two dimensions, eye opening and closing degree and pupil visibility, the method overcomes the shortcomings of traditional single feature methods that are easily interfered with by factors such as the user looking down and changes in viewing angle. This allows for the high-precision calculation of the duration of a single eye closure, and the statistically analyzed effective blink frequency effectively filters out irrelevant eye movements and accurately identifies the core eye discomfort signal of frequent blinking, which is undetectable by the DMS system. Finally, if frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered. This allows users to obtain professional medical diagnosis and improvement suggestions as soon as they experience discomfort such as dry eyes or eye fatigue that leads to frequent blinking. This further ensures the accuracy of the eye condition detection results and avoids distraction and decreased visual clarity caused by eye discomfort, effectively protecting driving safety and the user's eye health.

[0056] Based on any of the above embodiments Figure 2 This is the second flowchart of the eye state detection method provided by the present invention, as shown below. Figure 2 As shown, step S130 includes: step S131, step S132, step S133 and step S134.

[0057] Step S131: Based on the eye opening and closing degree and the pupil visibility, determine whether the target user is in a closed eye state.

[0058] The eye opening and closing degree is compared with the preset eye-closing threshold, and the pupil visibility is compared with the preset pupil visibility threshold. Based on the comparison results, it is determined whether the target user is in a closed-eye state.

[0059] The preset eye-closing threshold can be a fixed eye-closing threshold pre-set and stored in memory based on the baseline eye opening and closing degree of a regular user, or it can be an individual adaptive eye-closing threshold obtained by calibration based on the monitored facial image sequence of seated users. The preset pupil visibility threshold is an empirical value obtained based on calibration experiments and pre-stored in memory. The preset pupil visibility threshold is preferably set to 0.8.

[0060] Furthermore, the eye opening degree includes the eye opening degree of the left and right eyes. When the preset eye closing threshold is an individual adaptive eye closing threshold, it can correspond to the eye closing threshold of both the left and right eyes, or it can be the average of the eye closing thresholds of the left and right eyes. In this embodiment of the invention, the eye opening degree includes the eye opening degree of both the left and right eyes, and the preset eye closing threshold includes the preset eye closing thresholds of both the left and right eyes for explanation.

[0061] Furthermore, pupillary visibility includes the pupillary visibility of both the left and right eyes.

[0062] It should be noted that in real-world driving scenarios, relying solely on eye opening to determine whether the eyes are closed has a very high false alarm rate. Analysis shows the main reason is the interference from downward gaze (looking down / tilting the head). When a driver looks down at the central control screen or the instrument panel, the upper eyelid naturally droops, causing a sharp reduction in the distance between the key points of the upper and lower eyelids. The calculated eye opening at this time will be very small, easily falling below the eye-closing threshold, leading traditional algorithms to misjudge the driver as having closed eyes or dozing off. Therefore, this embodiment of the invention introduces pupil visibility. When the driver looks down at the instrument panel, although the eye opening decreases, the camera can still capture the complete pupil feature (pupil visibility is usually >0.8) because the condition of "pupil visibility <0.8" is not met. In this case, the system accurately identifies that the driver is looking down rather than having closed eyes, thus eliminating such false alarms and improving the accuracy of the eye-closing state determination. Furthermore, patients with dry eye syndrome or eye strain often experience incomplete blinking (i.e., the eyes close only halfway before reopening). During incomplete blinking, the eye opening and closing degree may hover around the edge of the eye-closing threshold, resulting in an ambiguous state. At this time, pupil visibility becomes an important indicator for judging the eye-closing state. As long as the eyelid droops down to cover the pupil (resulting in pupil visibility <0.8) and the eye opening and closing degree also shows a decreasing trend, the system can accurately detect this incomplete blinking or microspasmodic blinking. This is precisely the difference in accuracy between this invention and conventional fatigue monitoring.

[0063] Step S132: If the eye opening degree is less than a preset eye-closing threshold and the pupil visibility is less than a preset pupil visibility threshold, then the target user is determined to be in a closed-eye state, and the duration of the closed-eye state is obtained as the single eye-closing duration.

[0064] If the eye opening degree of the left eye is less than the preset eye-closing threshold of the left eye, the eye opening degree of the right eye is less than the preset eye-closing threshold of the right eye, and the pupil visibility of both the left and right eyes is less than the preset pupil visibility threshold, then the target user is determined to be in a closed-eye state, and the duration of this closed-eye state from beginning to end is recorded as the duration of a single closed-eye state.

[0065] Furthermore, blinking requires both eyes to be closed; if either eye opens first, the blink is considered complete. By statistically analyzing the duration of a single blink in this manner, the detection sensitivity can be improved.

[0066] Furthermore, the system determines an eye opening (i.e., the end of blinking) only after two consecutive frames of images have the eye open, thus effectively suppressing single-frame noise and micro-eye opening jitter.

[0067] Step S133: If the duration of a single eye closure is within a preset duration range, it is determined to be a valid blinking action; wherein, the preset duration range is 100 milliseconds to 500 milliseconds.

[0068] If the duration of a single blink is within the preset duration range, it is considered a valid blink action; the preset duration range is 100 milliseconds to 500 milliseconds.

[0069] It should be noted that, in accordance with the requirements of GB / T 39222-2020 and the existing DMS fatigue detection logic, a clear distinction is made between the blinking / eye-closing duration threshold in this embodiment of the invention and the existing fatigue eye-closing duration threshold. The existing fatigue eye-closing duration threshold is generally greater than or equal to 1 second. If the duration of a single eye-closing is greater than or equal to 1 second, it is classified as fatigue eye-closing and is not included in the valid blinking action. The preset duration range in this embodiment of the invention is a fixed range of 100 milliseconds to 500 milliseconds, pre-set and stored in memory based on the blinking / eye-closing duration range of regular users.

[0070] Furthermore, the variation pattern of eye opening and closing can be analyzed across multiple consecutive image frames to eliminate parameter fluctuations caused by momentary squinting (eye closure duration < 100 milliseconds) and external interference (such as wind or foreign objects entering the eye), ensuring the accuracy of parameter extraction. For example, if a driver squints for 50 milliseconds due to wind, this action is not considered as eye closure and is not included in the blink count.

[0071] Step S134: Count the number of effective blinking actions within a preset statistical period to obtain the effective blinking frequency of the target user.

[0072] The preset statistical period is a fixed value that is pre-set and stored in memory for ease of statistical analysis. For ease of analysis, the preset statistical period can be set to 1 minute. Of course, it can also be preset to other values ​​according to actual needs; no specific limitation is made here.

[0073] The system uses a sliding window to count the number of effective blinks and updates the results in real time to ensure data timeliness. By counting the number of effective blinks within a preset statistical period, the effective blink frequency of the target user is obtained.

[0074] Furthermore, when counting the number of effective blinks within a preset statistical period, it can be further determined whether the interval between two effective blinks is greater than a preset interval (a fixed value pre-set and stored in memory based on the blink interval of a regular user, for example, 500 milliseconds). If it is greater, it is included and the statistical results are updated. In this way, extremely high-frequency noise that is not physiologically normal or involuntary high-frequency tremors (microtwitches) of the eyelids are accurately filtered out, ensuring that every action finally included in the statistics is a physiologically independent, complete, and effective blink with dry eye compensation significance, thereby improving the accuracy of the effective blink frequency.

[0075] The eye state detection method provided in this invention combines eye opening and pupil visibility to determine whether a user is in a closed-eye state. It locks in the unique state of truly closed eyes from both mathematical and physical semantic dimensions, eliminating false positives caused by a downward head posture, thus improving the accuracy of both closed-eye state determination and frequent blinking determination. Simultaneously, it strictly limits the effective range to 100-500 milliseconds, eliminating high-frequency video noise from the temporal dimension and ensuring the absolute purity of the underlying data.

[0076] Based on any of the above embodiments Figure 3 This is the third flowchart of the eye state detection method provided by the present invention, as shown below. Figure 3 As shown, before step S131, the method further includes: steps S10, S20, S30, S40, S51, and S52.

[0077] Step S10: Obtain the first facial image sequence of the target user after he / she is seated.

[0078] Considering the significant differences in the eye baseline opening and closing degree among different users, a fixed eye-closing threshold cannot adapt to individual differences. Therefore, in this embodiment of the invention, an adaptive eye-closing threshold is obtained through prior calibration and used to determine the eye-closing state, which can improve the accuracy of the eye-closing state determination result.

[0079] Once the target user is identified as seated, a calibration program lasting 30 seconds is initiated to acquire the image sequence after the target user is seated, which is recorded as the calibration image sequence. Then, the face is located in the calibration image sequence using a face detection model to obtain the facial image sequence used for calibration, which is recorded as the first facial image sequence.

[0080] Step S20: Perform head pose detection on the first facial image sequence using a facial landmark detection model to obtain the head pose angle of the target user.

[0081] Then, the head pose angle of the target user is calculated using a facial landmark detection model. The head pose angle includes pitch angle and yaw angle.

[0082] Step S30: Based on the head posture angle and the preset frontal viewing angle range, extract the eye opening and closing degree sample set from the first facial image sequence.

[0083] Then, the image frames in the first facial image sequence are filtered. Only when the head posture angle is within the preset frontal viewing angle range, the eye opening and closing degree value of the corresponding image frame is extracted and stored in the eye opening and closing degree sample set for calibration, thereby eliminating deformation errors caused by side-facing or head-down states.

[0084] The preset viewing angle range is a fixed range that is pre-set and stored in memory based on the viewing angle range of a normal user under normal viewing conditions. For example, it can be set to have both pitch and yaw angles within ±15°.

[0085] Step S40: Extract the mean value of the preset proportion of samples with the smallest value in the eye opening and closing degree sample set, and use it as the initial eye-closing calibration threshold.

[0086] The preset ratio is a fixed value that is pre-set and stored in memory based on the statistical results of historical data, preferably 20%.

[0087] After the eye opening and closing degree sample set is constructed, the eye opening and closing degree values ​​in the sample set are sorted in ascending order, the top 20% of samples with the smallest values ​​are extracted and the average value is calculated. This average value is used as the initial eye-closing calibration threshold. This average value objectively reflects the extreme characteristics of eyelid closure in the user's natural state.

[0088] Furthermore, when calculating the average value, the average values ​​for the left and right eyes can be calculated separately as the initial closed-eye calibration thresholds for the left and right eyes.

[0089] Step S51: If the initial eye-closing calibration threshold is less than the preset threshold, then the preset eye-closing threshold is set to a preset fixed value.

[0090] Step S52: If the initial eye-closing calibration threshold is greater than or equal to the preset threshold, then the preset ratio of the initial eye-closing calibration threshold is used as the preset eye-closing threshold.

[0091] To ensure the basic availability of the system, the initial eye-closing calibration threshold is protected against extreme values: if the initial calibration threshold is less than the preset threshold (e.g., 0.4), it indicates that the data may be abnormal, and the preset eye-closing threshold is set to a preset fixed value (e.g., 0.02); otherwise, the initial eye-closing calibration threshold is multiplied by a preset ratio (e.g., 1 / 4) as the preset eye-closing threshold for the target user.

[0092] Among them, the preset threshold is a fixed value that is preset and stored in the memory based on the eye baseline opening and closing degree of a regular user; the preset fixed value is a fixed value that is preset and stored in the memory based on the eye closing threshold of a regular user; and the preset ratio value is a fixed value that is preset and stored in the memory based on the eye closing threshold and experience of a regular user.

[0093] Furthermore, if the initial eye-closing calibration threshold includes the initial eye-closing calibration thresholds for both the left and right eyes, the preset eye-closing thresholds for the left and right eyes can be determined separately according to the above rules when determining the preset eye-closing threshold.

[0094] The eye state detection method provided in this invention eliminates external posture interference by limiting the frontal viewing angle and calculating the mean of the minimum value of the sample set, and obtains a judgment benchmark that closely matches the user's real closed eye characteristics. This solves the problem of missed detection or frequent false alarms that easily occur when using fixed thresholds for users with different eye shapes.

[0095] Based on any of the above embodiments Figure 4 This is the fourth flowchart of the eye state detection method provided by the present invention, as shown below. Figure 4 As shown, step S120 includes: step S121, step S122, step S30, step S40, step S51 and step S52.

[0096] Step S121: Use a face detection model to locate faces in the initial image sequence to obtain a second facial image sequence.

[0097] The face detection model is used to locate faces in the initial image sequence and output face bounding boxes. A second image sequence is then obtained from the initial image sequence using the face bounding boxes, and is denoted as the second face image sequence.

[0098] Step S122: Perform eye feature point detection on the second facial image sequence to obtain the coordinates of the eye feature points.

[0099] The second facial image sequence is input into a facial landmark detection model, such as a regression model containing 68 feature points, and the model outputs high-precision feature point coordinates for the eye region, denoted as eye feature point coordinates. Figure 5 As shown, the eye feature points include feature points numbered 36-46, where feature points numbered 36-39 are the set of right eye feature points of the person in the image, and feature points numbered 42-47 are the set of left eye feature points of the person in the image.

[0100] Step S123: Based on the coordinates of the eye feature points, calculate the first distance between the upper and lower eyelids and the second distance between the inner and outer corners of the eyes.

[0101] Step S124: Determine the eye opening degree based on the first distance and the second distance.

[0102] Using the coordinates of eye feature points, the sum of the vertical Euclidean distances between corresponding feature points on the upper and lower eyelids is calculated as the first distance, and the horizontal Euclidean distance between corresponding feature points on the inner and outer corners of the eye is calculated as the second distance. Then, based on the first and second distances, the eye opening degree is determined.

[0103] like Figure 5 As shown, the feature points corresponding to the upper eyelid of the right eye include 37 and 38, and their ordinates are denoted as P. 37 and P 38 The feature points corresponding to the lower eyelid of the right eye include 41 and 40, and their ordinates are denoted as P. 41 and P 40 The feature point corresponding to the inner corner of the right eye is 39, and its coordinates are marked as follows: The feature point corresponding to the outer corner of the right eye is 36, and its coordinate is marked as... The feature points corresponding to the upper eyelid of the left eye include 43 and 44, and their ordinates are denoted as P. 43 and P 44 The feature points corresponding to the lower eyelid of the left eye are 47 and 46, and their ordinates are denoted as P. 47 and P 46 The feature point corresponding to the inner corner of the left eye is 42, and its coordinate is marked as... The feature point corresponding to the outer corner of the left eye is 45, and its coordinate is marked as... .

[0104] The specific calculation method for the opening and closing degree of the right eye is as follows: ; in, Indicates the degree of eye opening and closing of the right eye; This represents the sum of the vertical Euclidean distances between corresponding feature points on the upper and lower eyelids of the right eye; | represents the horizontal Euclidean distance between the corresponding feature points at the inner and outer corners of the right eye.

[0105] The specific calculation method for the opening and closing degree of the left eye is as follows: ; in, Indicates the degree of eye opening / closing of the left eye; This represents the sum of the vertical Euclidean distances between corresponding feature points on the upper and lower eyelids of the left eye; | represents the horizontal Euclidean distance between the corresponding feature points at the inner and outer corners of the left eye.

[0106] Step S125: Perform pupil detection on the second facial image sequence using a pupil detection model to obtain pupil feature points and their corresponding confidence scores, and use the confidence scores as the pupil visibility.

[0107] A pupil detection model constructed using a deep learning network is used to extract features from each frame of the facial image sequence in the second facial image sequence. The coordinates of the pupil feature points and their corresponding confidence scores are obtained from the model output, and the confidence scores are used as pupil visibility.

[0108] The eye state detection method provided in this embodiment of the invention calculates the eye opening degree based on the coordinates of eye feature points. This set calculation method ensures the computational efficiency and low latency of the algorithm, while the pupil visibility directly calls the confidence output of the pupil detection model, making full use of the feature extraction capability of the neural network under complex lighting conditions (such as backlight and dark light), effectively balancing the computing power consumption of the vehicle processor and the detection accuracy requirements.

[0109] Based on any of the above embodiments, step S140 includes: step S141, step S142 and step S143.

[0110] Step S141: If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, and interactive query information is output.

[0111] Based on medical definitions, a blink rate exceeding 20 times per minute is preliminarily considered frequent blinking, indicating potential eye discomfort (such as eye strain, dryness, or mild inflammation). Therefore, a preset frequent blinking frequency can be set to 20 times per minute.

[0112] If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, indicating eye discomfort. In this case, an interactive query message is output.

[0113] The output of interactive inquiry information may include, but is not limited to, sending inquiry information to the target user simultaneously through two methods: display on the in-vehicle central control screen and / or voice prompts.

[0114] The inquiry could be something like, "We've detected that you're blinking frequently, which may indicate eye discomfort. Would you like to get a medical diagnosis and advice?" The voice prompt should be clear and gentle to avoid interfering with the driver's normal driving.

[0115] Target users can confirm via touch on the central control screen or respond with "yes" or "no" via voice. Both response methods are supported, improving ease of operation.

[0116] Further, you can ask, "Please provide more details about your symptoms."

[0117] Target users can further answer questions about recent symptoms via voice, such as, "My eyes have been a bit dry, a bit itchy, and a bit sore lately."

[0118] Furthermore, to avoid misjudgments, a fault-tolerance mechanism can be added: If the blinking frequency exceeds 20 times per minute for two out of three consecutive minutes (two minutes in total), the in-vehicle medical auxiliary diagnostic mechanism is officially triggered, and interactive inquiry information is output. If only a single instance of a blinking frequency exceeding the preset frequent blinking frequency is detected, it is judged as an accidental phenomenon (such as a momentary foreign object stimulus), and subsequent processes are not triggered, ensuring the accuracy of the judgment result. For example, if the driver blinks 22 times in the first minute, 18 times in the second minute, and 19 times in the third minute, the inquiry is not triggered because the exceedance only occurs for one minute. However, if the exceedance occurs in the first and second minutes but not in the third minute, the inquiry process is triggered.

[0119] Step S142: Obtain the historical symptom information fed back by the target user based on the interactive inquiry information.

[0120] Then, obtain the target user's historical symptom information based on the interactive inquiry information.

[0121] Historical symptom information can come from the target user's voice feedback or be retrieved from medical applications.

[0122] Step S143: Input the blinking frequency and the historical symptom information into the medical auxiliary diagnosis model to obtain the eye discomfort status assessment results and improvement suggestions output by the medical auxiliary diagnosis model.

[0123] The medical auxiliary diagnostic model is a natural language processing and logical reasoning model deployed on the vehicle's local controller or cloud server. Based on the input objective detection data (blinking frequency) and subjective symptom text, it outputs the corresponding assessment results and improvement suggestions for the eye condition.

[0124] Blink frequency data is combined with historical symptom information from subjective feedback to create structured prompts, which are then input into a large-scale medical auxiliary diagnostic model, either locally or in the cloud. This model uses a built-in medical knowledge base to perform logical reasoning, outputting an assessment of eye discomfort (such as eye strain, mild dry eye, or excessive eye use) and improvement suggestions. For example, if eye strain is suspected, it might suggest stopping and looking into the distance for 5 minutes, avoiding prolonged periods of focused driving, and using artificial tears to relieve dryness.

[0125] The assessment results and improvement suggestions for eye discomfort can be fed back to the target user through two methods: display on the in-vehicle central control screen and voice broadcast. The feedback time is ≤3 seconds to ensure that it does not affect the driver's normal driving. At the same time, the system will clearly indicate in the feedback results that "this suggestion is for auxiliary reference only. If eye discomfort persists, please seek medical attention in time" to avoid medical liability disputes.

[0126] Furthermore, if the target user does not respond within 10 seconds, the system defaults to "not needed for now," records the test result, continues real-time monitoring, and does not trigger subsequent diagnostic processes.

[0127] Furthermore, if the driver continues to drive after receiving the assessment results and improvement suggestions for their eye condition and the eye discomfort does not improve, for example, if the blinking frequency is greater than 20 times / minute for 5 consecutive minutes, the system will issue a warning to remind the driver that "the eye discomfort has not been relieved, please stop and rest in time." At the same time, the system will record the test data, inquiry results and diagnostic suggestions for the driver to review later or provide them to professional medical personnel for reference.

[0128] The eye condition detection method provided in this invention improves the scientific rigor and relevance of auxiliary diagnostic results by integrating objective blink frequency and subjective symptom descriptions as input to a large model. This enables the in-vehicle system to provide users with practical health guidance information and achieves closed-loop health management from objective data monitoring to subjective symptom confirmation.

[0129] Based on any of the above embodiments, after step S143, the eye state detection method further includes steps S151 and S152.

[0130] Step S151: When the humidity of the in-vehicle environment is detected to be lower than a preset humidity threshold, or when the air vent is directed toward the face area of ​​the target user, a corresponding device drive command is generated.

[0131] After obtaining the evaluation results and improvement suggestions output by the medical auxiliary diagnostic model, the sensor data in the cabin is retrieved to obtain the current humidity value of the vehicle interior and the physical orientation data of the air vents of the vehicle air conditioner.

[0132] If the humidity inside the vehicle is detected to be lower than a preset humidity threshold, or if the air conditioning vents are pointing directly at the user's face, the system determines that the current environment will accelerate the evaporation of moisture from the surface of the eyeballs, and then generates a corresponding device drive command. The preset humidity threshold is a critical humidity value between eye comfort and discomfort, obtained through experimental testing and pre-stored in memory. As a specific example of this critical value, the preset humidity threshold can be set to 30%.

[0133] Step S152: Control the in-vehicle environment adjustment device to perform the corresponding adjustment operation through the device drive command.

[0134] Then, through the device's drive commands, the in-vehicle environment control equipment can be directly controlled to perform corresponding adjustment operations. Specifically, it can control the stepper motor of the vehicle's air conditioning vents to change the angle of the louvers, so that the airflow vector direction avoids the target user's facial area; and / or, it can activate the in-vehicle humidification module. Furthermore, it can also reduce the backlight brightness of the instrument panel and central control screen.

[0135] The eye condition detection method provided in this invention automatically adjusts the air conditioning direction and environmental parameters, thereby physically eliminating external environmental factors that aggravate eye discomfort and providing a health intervention method with substantial physical effects, significantly improving cabin comfort.

[0136] Based on any of the above embodiments, step S110 includes: step S111.

[0137] Step S111: Using the in-vehicle driver monitoring system (DMS) camera or occupant monitoring system (OMS) camera, an initial image sequence including the target user's eye area is acquired at a preset sampling frequency; wherein the preset sampling frequency is in the range of 10 frames / second to 20 frames / second.

[0138] In this embodiment of the invention, a deployed vehicle-mounted DMS camera or OMS (Occupant Monitoring System) camera is used as the data acquisition device, and an initial image sequence including the target user's eye region is acquired using a preset sampling frequency. The preset sampling frequency is a range of sampling frequencies determined through experimental testing that balances blink detection accuracy with the vehicle's computing resources, and is pre-stored in the memory.

[0139] By using existing cameras, no additional hardware is needed, reducing deployment costs.

[0140] When the target user is the driver, a DMS camera is preferred. This camera is fixedly installed in front of the driver's seat (such as above the dashboard or directly in front of the steering wheel), with the lens angle adjusted to 30° to the driver's face to ensure clear capture of the driver's eye area and avoid parameter extraction errors caused by angular deviation.

[0141] Furthermore, the camera's sampling frequency is set to 10-20 frames per second, preferably 15 frames per second. If the frame rate is lower than 10 frames per second, the interval between adjacent frames exceeds 100 milliseconds, making it easy to miss brief, effective blinks; if the frame rate is higher than 30 frames per second, it will cause excessive strain on the vehicle's SOC (System on Chip) computing power, resulting in vehicle system lag. The above sampling frequency can accurately capture every subtle movement of the eye opening and closing.

[0142] The eye state detection method provided in this invention collects the initial image sequence of the target user in the above manner, and can achieve high-precision capture of eye features without increasing any additional hardware costs or affecting the smoothness of the vehicle system.

[0143] The eye condition detection device provided by the present invention will be described below. The eye condition detection device described below can be referred to in correspondence with the eye condition detection method described above.

[0144] Figure 6 This is a schematic diagram of the eye condition detection device provided by the present invention, as shown below. Figure 6 As shown, the device includes an image acquisition module 610, a parameter extraction module 620, a blink statistics module 630, and a state detection module 640; wherein: Image acquisition module 610 is used to acquire an initial image sequence including the eye region of the target user; The parameter extraction module 620 is used to extract eye feature parameters of the target user based on the initial image sequence; wherein, the eye feature parameters include eye opening and closing degree and pupil visibility; The blinking statistics module 630 is used to determine the duration of a single eye closure based on the eye feature parameters, and to count the effective blinking frequency of the target user based on the duration of the single eye closure. The status detection module 640 is used to trigger the vehicle-mounted medical auxiliary diagnosis mechanism if frequent blinking is detected based on the effective blinking frequency.

[0145] The eye state detection device provided in this invention first acquires an initial image sequence including the eye region of a target user; then, it extracts eye feature parameters of the target user based on the initial image sequence, wherein the eye feature parameters include eye opening and closing degree and pupil visibility; furthermore, it determines the duration of a single eye closure based on the eye feature parameters, and statistically analyzes the effective blink frequency of the target user based on the duration of a single eye closure. By extracting eye feature parameters in two dimensions, eye opening and closing degree and pupil visibility, it overcomes the shortcomings of traditional single feature methods that are easily interfered with by factors such as the user looking down and changes in viewing angle. Thus, it can calculate the duration of a single eye closure with high accuracy. The effective blink frequency calculated based on this effectively filters out irrelevant eye movements and accurately identifies the core eye discomfort signal of frequent blinking, which cannot be detected by the DMS system. Finally, if frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered. This allows users to obtain professional medical diagnosis and improvement suggestions as soon as they experience discomfort such as dry eyes or eye fatigue that leads to frequent blinking. This further ensures the accuracy of the eye condition detection results and avoids distraction and decreased visual clarity caused by eye discomfort, effectively protecting driving safety and the user's eye health.

[0146] According to the present invention, an eye condition detection device is provided, wherein the parameter extraction module 620 is specifically used for: Based on the eye opening and closing degree and the pupil visibility, determine whether the target user is in a closed-eye state; If the eye opening degree is less than a preset eye-closing threshold and the pupil visibility is less than a preset pupil visibility threshold, then the target user is determined to be in a closed-eye state, and the duration of the closed-eye state is obtained as the single eye-closing duration. If the duration of a single blink is within a preset duration range, it is determined to be a valid blink action; wherein, the preset duration range is 100 milliseconds to 600 milliseconds; The effective blinking frequency of the target user is obtained by counting the number of effective blinking actions within a preset statistical period.

[0147] An eye condition detection device according to the present invention further includes: Image acquisition module 610 is used to acquire the first facial image sequence of the target user after the user has taken a seat; The pose detection module is used to perform head pose detection on the first facial image sequence using a facial key point detection model to obtain the head pose angle of the target user. The sample extraction module is used to extract a sample set of eye opening and closing angles from the first facial image sequence based on the head posture angle and a preset frontal viewing angle range. The data extraction module is used to extract the mean value of the preset proportion of samples with the smallest value in the eye opening and closing degree sample set, as the initial eye-closing calibration threshold. The first setting module is used to set the preset eye-closing threshold to a preset fixed value if the initial eye-closing calibration threshold is less than the preset threshold. The second setting module is used to set a preset ratio of the initial eye-closing calibration threshold as the preset eye-closing threshold if the initial eye-closing calibration threshold is greater than or equal to the preset threshold.

[0148] According to the present invention, an eye condition detection device is provided, wherein the parameter extraction module 620 is specifically used for: The initial image sequence is used to locate faces using a face detection model to obtain a second facial image sequence; Eye feature point detection is performed on the second facial image sequence to obtain the coordinates of the eye feature points; Based on the coordinates of the eye feature points, calculate the first distance between the upper and lower eyelids and the second distance between the inner and outer corners of the eyes; The eye opening degree is determined based on the first distance and the second distance; The pupil detection model is used to detect pupils in the second facial image sequence, obtain pupil feature points and their corresponding confidence scores, and use the confidence scores as the pupil visibility.

[0149] According to the present invention, an eye condition detection device is provided, wherein the condition detection module 640 is specifically used for: If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, and interactive query information is output. Obtain the historical symptom information of the target user based on the interactive inquiry information; The blinking frequency and the historical symptom information are input into the medical auxiliary diagnostic model to obtain the eye discomfort status assessment results and improvement suggestions output by the medical auxiliary diagnostic model.

[0150] An eye condition detection device according to the present invention further includes: The information acquisition module is used to acquire the humidity inside the vehicle and the direction of the air vents of the vehicle's air conditioning system. The instruction generation module is used to generate corresponding device drive instructions when the humidity of the in-vehicle environment is detected to be lower than a preset humidity threshold, or when the air outlet is directed toward the face area of ​​the target user. The equipment adjustment module is used to control the in-vehicle environment adjustment equipment to perform corresponding adjustment operations through the equipment drive commands.

[0151] According to an eye state detection device provided by the present invention, the image acquisition module 610 is specifically used for: An initial image sequence including the target user's eye region is acquired using a vehicle driver monitoring system (DMS) camera or an occupant monitoring system (OMS) camera at a preset sampling frequency; wherein the preset sampling frequency is in the range of 10 frames / second to 20 frames / second.

[0152] It should be noted that the eye state detection device provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned eye state detection method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0153] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute an eye state detection method, which includes: acquiring an initial image sequence including the eye region of a target user; extracting eye feature parameters of the target user based on the initial image sequence; wherein the eye feature parameters include eye opening and closing degree and pupil visibility; determining the duration of a single eye closure based on the eye feature parameters; calculating the effective blink frequency of the target user based on the duration of the single eye closure; and triggering an in-vehicle medical auxiliary diagnostic mechanism if frequent blinking is detected based on the effective blink frequency.

[0154] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0155] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the eye state detection method provided in the above embodiments. The method includes: acquiring an initial image sequence including the eye region of a target user; extracting eye feature parameters of the target user based on the initial image sequence; wherein the eye feature parameters include eye opening and closing degree and pupil visibility; determining the duration of a single eye closure based on the eye feature parameters; calculating the effective blinking frequency of the target user based on the duration of the single eye closure; and triggering an in-vehicle medical auxiliary diagnosis mechanism if frequent blinking is detected based on the effective blinking frequency.

[0156] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An ocular condition detection method characterized by, include: Obtain an initial image sequence including the target user's eye region; Based on the initial image sequence, eye feature parameters of the target user are extracted; wherein, the eye feature parameters include eye opening and pupil visibility; The duration of a single eye closure is determined based on the eye feature parameters, and the effective blinking frequency of the target user is calculated based on the duration of the single eye closure. If frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered.

2. The eye condition detection method of claim 1, wherein The process of determining the duration of a single eye closure based on the eye feature parameters, and calculating the effective blinking frequency of the target user based on the duration of the single eye closure, includes: Based on the eye opening and closing degree and the pupil visibility, determine whether the target user is in a closed-eye state; If the eye opening degree is less than a preset eye-closing threshold and the pupil visibility is less than a preset pupil visibility threshold, then the target user is determined to be in a closed-eye state, and the duration of the closed-eye state is obtained as the single eye-closing duration. If the duration of a single blink is within a preset duration range, it is determined to be a valid blink action; wherein, the preset duration range is 100 milliseconds to 500 milliseconds; The effective blinking frequency of the target user is obtained by counting the number of effective blinking actions within a preset statistical period.

3. The eye condition detection method of claim 2, wherein, Before determining whether the target user is in a closed-eye state based on the eye opening and pupil visibility, the method further includes: Obtain the first facial image sequence of the target user after he / she is seated; The head pose of the target user is obtained by performing head pose detection on the first facial image sequence using a facial landmark detection model. Based on the head posture angle and the preset frontal viewing angle range, extract the eye opening and closing degree sample set from the first facial image sequence; Extract the mean value of the preset proportion of samples with the smallest value in the eye opening and closing degree sample set, and use it as the initial eye-closing calibration threshold. If the initial eye-closing calibration threshold is less than the preset threshold, then the preset eye-closing threshold is set to a preset fixed value; If the initial eye-closing calibration threshold is greater than or equal to the preset threshold, then the preset ratio of the initial eye-closing calibration threshold is used as the preset eye-closing threshold.

4. The ocular condition detection method of claim 1, wherein, The step of extracting the eye feature parameters of the target user based on the initial image sequence includes: The initial image sequence is used to locate faces using a face detection model to obtain a second facial image sequence; Eye feature point detection is performed on the second facial image sequence to obtain the coordinates of the eye feature points; Based on the coordinates of the eye feature points, calculate the first distance between the upper and lower eyelids and the second distance between the inner and outer corners of the eyes; The eye opening degree is determined based on the first distance and the second distance; The pupil detection model is used to detect pupils in the second facial image sequence, obtain pupil feature points and their corresponding confidence scores, and use the confidence scores as the pupil visibility.

5. The ocular condition detection method according to any one of claims 1 to 4, characterized in that, If frequent blinking is detected based on the effective blinking frequency, the in-vehicle medical auxiliary diagnosis mechanism is triggered, including: If the effective blinking frequency is greater than the preset frequent blinking frequency, it is determined to be frequent blinking, and interactive query information is output. Obtain the historical symptom information of the target user based on the interactive inquiry information; The blinking frequency and the historical symptom information are input into the medical auxiliary diagnostic model to obtain the eye discomfort status assessment results and improvement suggestions output by the medical auxiliary diagnostic model.

6. The eye condition detection method of claim 5, wherein, After inputting the blinking frequency and historical symptom information into the medical auxiliary diagnostic model to obtain the eye discomfort assessment results and improvement suggestions output by the medical auxiliary diagnostic model, the method further includes: Obtain the humidity inside the vehicle and the direction of the air vents of the vehicle's air conditioning system; When the humidity inside the vehicle is detected to be lower than a preset humidity threshold, or when the air vent is directed toward the face of the target user, a corresponding device driving command is generated. The device drive commands control the in-vehicle environment adjustment equipment to perform corresponding adjustment operations.

7. The ocular condition detection method of any one of claims 1 to 4, wherein, The acquisition of the initial image sequence including the target user's eye region includes: An initial image sequence including the target user's eye region is acquired using a vehicle driver monitoring system (DMS) camera or an occupant monitoring system (OMS) camera at a preset sampling frequency; wherein the preset sampling frequency is in the range of 10 frames / second to 20 frames / second.

8. An ocular condition detection apparatus, characterized by, include: The image acquisition module is used to acquire an initial image sequence including the target user's eye region; The parameter extraction module is used to extract eye feature parameters of the target user based on the initial image sequence; wherein, the eye feature parameters include eye opening and pupil visibility; The blinking statistics module is used to determine the duration of a single eye closure based on the eye feature parameters, and to count the effective blinking frequency of the target user based on the duration of the single eye closure. The status detection module is used to trigger the vehicle-mounted medical auxiliary diagnosis mechanism if frequent blinking is detected based on the effective blinking frequency.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the eye state detection method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the eye state detection method as described in any one of claims 1 to 7.

Images