DRIVER CONDITION MONITORING USING CORNEAL REFLECTION DETECTION

Abstract

A method includes: sampling light information reflected from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; measuring a plurality of eye parameters based on the sampled light information by a control unit installed in the vehicle; calculating, by the control unit, a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determining, by the control unit, whether the calculated hazard score exceeds a hazard threshold; and executing, by the control unit, a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.

Description

TECHNICAL FIELD

[0001] The present disclosure generally relates to monitoring a condition of a driver of a vehicle, and more particularly to monitoring a driver condition using corneal reflection sensing. BACKGROUND

[0002] Many vehicle accidents occur due to distractions that impair a driver's ability to focus on nearby traffic and the road ahead. For example, a driver's focus may be diverted when searching for a destination, looking at a cell phone, or talking to passengers. Additionally, a driver's gaze may stray from the road if they are daydreaming, or their eyes may close completely if they become drowsy.

[0003] In an attempt to prevent driver distraction, some vehicles employ devices that monitor the driver's eyes. However, it can be difficult to accurately assess when a driver's distraction level becomes problematic. For example, the required level of concentration may be heightened when driving at night, at high speeds, on winding roads, in urban areas, etc. Analyzing driver activity within the vehicle can also be complicated without knowing the driver's unique behavioral history. SUMMARY

[0004] The present disclosure provides techniques for tracking a driver's eye activity using an optical sensor mounted in a vehicle. If the tracked eye activity indicates that the driver is not adequately concentrating on the road ahead, corrective action can be taken to correct and train the driver to develop safer driving habits. A variety of eye parameters can be measured to determine if the driver has an adequate level of concentration. It can be determined that adequate concentration for driving is not present if, for example, the driver's cornea is not aligned with the direction of oncoming traffic or the road ahead, or the cornea is either too still oris motionless (indicating that the driver is daydreaming), or moving too fast (indicating that the driver is distracted), or the cornea is less than usual exposed because the driver's eyelids are closing for long periods of time. Furthermore, a hazard score or rating, taking into account the parameters described above or other parameters, can be calculated, and the calculated hazard score can be compared to a hazard threshold to determine whether the driver's distraction level has reached unsafe levels.

[0005] According to embodiments of the present disclosure, a method includes: sampling light information reflected from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; measuring a plurality of eye parameters based on the sampled light information; calculating a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determining whether the calculated hazard score exceeds a hazard threshold; and executing a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.

[0006] The optical sensor can be mounted on a vehicle's rearview mirror. A light source can be coupled to the optical sensor.

[0007] Measuring the plurality of eye parameters may include, for example, measuring a gaze orientation indicating an alignment of the driver's gaze with a road ahead of the vehicle based on the sampled light information. Measuring the gaze orientation may include determining whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level.

[0008] Measuring the plurality of eye parameters may further include, for example, measuring a corneal immobility indicating a corneal immobility of the driver during a period of time based on the sampled light information. Measuring the corneal immobility may further include determining a number of movements of the driver's cornea during the period of time.

[0009] Measuring the plurality of eye parameters may further include, for example, measuring a blink rate indicating an amount of closure of a driver's eyelid during a period of time based on the sampled light information. Measuring the blink rate may include determining a maximum amount of time the driver's eyelid is closed during the period of time.

[0010] Measuring the plurality of eye parameters may further include, for example: determining a current speed of the vehicle; and measuring the plurality of eye parameters further based on the determined current vehicle speed.

[0011] Calculating the hazard score may include: measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information; measuring a corneal immobility indicating a motionlessness of a driver's cornea during a period of time based on the sampled light information; measuring a blink closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information; and calculating the hazard score using the measured gaze orientation, the corneal immobility, and the blink closure.

[0012] Additionally, the method may further include controlling the hazard threshold according to a desired sensitivity. Accordingly, the method may further include adjusting the hazard threshold in real time according to one or more factors selected from a group consisting of: a direction of travel, a speed of travel, a time of day, a duration of travel, a starting point of the trip, a destination of the trip, a number of occupants in the vehicle, an amount of noise in the vehicle, and use of a mobile device. The method may further include determining predetermined hazard thresholds; and setting the hazard threshold based on the predetermined hazard thresholds.

[0013] The method may further include applying a filter to the sampled light information to remove noise therefrom.

[0014] Implementing the corrective action may include, for example, providing the driver with a visual, audible or tactile warning.

[0015] Carrying out the corrective action may further include, for example, reducing a vehicle speed.

[0016] Executing the corrective action may further include, for example, determining whether the calculated hazard score continues to exceed the hazard threshold after reducing the vehicle speed; and further reducing the vehicle speed if the calculated hazard score continues to exceed the hazard threshold.

[0017] According to embodiments of the present disclosure, a system further includes: an optical sensor mounted in a vehicle and configured to sample light information reflecting from an eye of a driver of the vehicle; and a controller disposed in the vehicle and configured to: measure a plurality of eye parameters based on the sampled light information; calculate a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determine whether the calculated hazard score exceeds a hazard threshold; and perform a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.

[0018] Furthermore, according to embodiments of the present disclosure, a non-transitory computer-readable medium includes program instructions executable by a control unit implemented in a vehicle, the program instructions, when executed, causing the control unit to: measure a plurality of eye parameters based on light information reflected from an eye of a driver of the vehicle and sensed by an optical sensor mounted in the vehicle; calculate a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determine whether the calculated hazard score exceeds a hazard threshold; and execute a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold. Character list

[0019] The embodiments herein may be better understood by reference to the following description in conjunction with the accompanying drawings, in which like reference numerals indicate identical or functionally similar elements and of which: the Fig. 1A and Fig. 1B illustrates an exemplary optical sensor system for tracking eye activity; Fig. 2 illustrates an example of the driver's eye movements; the Fig. 3A and Fig. 3B illustrates an exemplary simplified method for monitoring a driver condition using corneal reflection sensing; Fig. 4 illustrates an exemplary simplified method for performing a corrective action in response to determining that the driver is driving in an unsafe manner.

[0020] It should be understood that the above-referenced drawings are not necessarily to scale and present a somewhat simplified representation of various preferred features illustrative of the basic principles of the disclosure. The specific embodiments of the present disclosure, including, for example, particular dimensions, orientations, locations, and shapes, will be determined in part by the particular intended application and environment of use. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. As will be appreciated by those skilled in the art, the described embodiments may be modified in many different ways without departing from the spirit or scope of the present disclosure. Furthermore, like reference numerals refer to like elements throughout the specification.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a" and "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of the recited features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed elements.

[0023] It is to be understood that the term "vehicle" or "vehicular" or other similar term used herein includes motor vehicles in general, such as passenger cars, including off-road vehicles (SUVs), buses, trucks, various business cars, watercraft, including a variety of boats and vessels, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from feedstocks other than petroleum). As referred to herein, a hybrid vehicle is a vehicle that has two or more power sources, such as both gasoline-powered and electric-powered vehicles.

[0024] Furthermore, it is to be understood that one or more of the following methods, or aspects thereof, may be performed by at least one control unit. The term "control unit" may refer to a hardware device including a memory and a processor. The memory is configured to store program instructions, and the processor is specifically programmed to execute the program instructions to perform one or more processes described further below. Furthermore, it is to be understood that the following methods may be performed by a device having the control unit in conjunction with one or more other components, as would be understood by someone of ordinary skill in the art.

[0025] Furthermore, the control unit of the present disclosure may be embodied as non-transitory computer-readable media containing executable program instructions executed by a processor, controller, or the like. Examples of computer-readable media include, but are not limited to, read-only memory, random access memory, compact disc read-only memory (CD-ROMs), magnetic tape, floppy disks, flash drives, smart cards, and optical data storage devices. The computer-readable recording medium may also be distributed throughout a computer network such that the program instructions are stored and executed in a distributed manner, e.g., by a telematics server or a controller area network (CAN).

[0026] Turning now to embodiments of the present disclosure, the disclosed techniques utilize a vehicle-mounted optical sensor to track a driver's eye activity. Using the information acquired by the optical sensor, a variety of eye parameters are measured to determine whether the driver is maintaining an appropriate level of concentration. The measured eye parameters may include, for example (without limitation), gaze alignment, which indicates alignment of a driver's gaze with a road ahead of the vehicle, corneal stillness, which indicates stillness of a driver's cornea for a period of time, eyelid closure, which indicates an amount of closure of an eyelid of the driver for a period of time, and so on.A hazard score that considers the measured parameters can be calculated to quantify the hazard level associated with the driver's attention. The calculated hazard score can be compared to a hazard threshold to determine whether the driver's distraction level has reached unsafe levels. Furthermore, if the hazard score exceeds the hazard threshold (controlled according to a desired sensitivity), indicating that the driver is not adequately focused on the road ahead, corrective action can be taken to correct and train the driver to develop safer driving habits.

[0027] The Fig. 1A and Fig. 1B illustrate an exemplary optical sensor system for tracking eye activity. As in Fig. As shown in Figure 1A, an optical sensor 100be mounted in a vehicle. The optical sensor 100 can be implemented appropriately within the passenger compartment of a vehicle. The optical sensor 100 can be attached to a rearview mirror 110 of the vehicle (see Fig. 1A), a windshield, a dashboard, or the like. The optical sensor 100 can be mounted externally or in the rearview mirror 110 , the windshield, the dashboard, etc.

[0028] The optical sensor 100may be any sensor capable of detecting / sensing light (referred to herein as "light information") and converting the light into electronic signals, such as a video camera, an infrared camera, or another specially designed optical sensor, as would be understood by someone of ordinary skill in the art. In this regard, corneal reflection techniques, which are generally known in the art, may be employed to detect the eye activity of a driver of the vehicle in which the optical sensor 100 is mounted. In particular, the optical sensor may 100 Scan light (i.e. “light information”) reflected from the driver’s cornea, as in Fig. 1B. The light reflected from the driver's cornea can, for example, be detected by a sensor connected to the optical 100(not shown) coupled light source, a light source arranged inside the vehicle, the environment outside the vehicle and the like.

[0029] Using corneal reflection techniques as described above, the optical sensor 100 capable of sensing light reflected from the driver's cornea and constantly transmitting such information to a control unit (not shown) for processing, thus enabling the position and movements of the cornea to be continuously tracked in real time. For example, Fig. 2 an exemplary representation of the eye movements of a driver in relation to the road / environment in front of the vehicle, which are recorded by the optical sensor 100were tracked. In the event that excessive noise exists, e.g., due to vehicle-related movements, one or more filters, such as a Kalman filter, a low-pass filter, or the like, or other signal conditioning may be applied to the sampled light information to eliminate such noise, as would be understood by someone of ordinary skill in the art.

[0030] The optical sensor 100Sampled light information can be used by the control unit to measure a variety of eye parameters. As an example, gaze alignment, which indicates alignment of the driver's gaze with a road ahead of the vehicle, can be measured to detect whether the driver's gaze is misaligned with the road ahead and / or oncoming traffic. In particular, using the sampled light information, it can be determined whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level (e.g., 5 to 30 degrees).As another example, corneal stillness, indicating a lack of movement of a driver's cornea, may be measured for a predefined period of time to detect whether the driver's eyes are either too still, indicating that the driver is daydreaming, or moving too quickly, indicating that the driver is distracted. To this end, a number of movements of the driver's cornea during the predefined period of time may be determined using the sampled light information. As yet another example, eyelid closure, indicating an amount of closure of an eyelid of the driver during a predefined period of time, may be measured to detect whether the driver is drowsy.For this purpose, a maximum period of time that the driver's eyelid is closed during the predefined period can be determined using the sensed light information (for example, if the cornea is temporarily not detected, it can be inferred that the eyelid is partially or completely closed). It should be understood that any number and / or type of eye parameters can be determined based on the information received from the optical sensor. 100 sampled light information can be measured and consequently the parameters described above are provided for demonstration purposes only.

[0031] Once a variety of eye parameters indicating activity of the driver's eye(s) have been measured, the measured parameters can be used to determine whether the driver is distracted and failing to pay adequate attention to the road ahead and / or nearby traffic. For this purpose, a hazard score quantifying a driver's driving condition—i.e., a degree of danger posed by the driver due to a lack of attention—can be calculated based on the measured parameters. In some cases, the calculation of the hazard score may depend on the measured eye parameters as well as the current speed of the vehicle.

[0032] In an exemplary case where three eye parameters listed above are measured according to the light information sensed by the optical sensor 100, a hazard score (I E) can be calculated according to the following equation 1: Ι E = i A u s r i c h t u n g × i B e w e g u n g s l o s i g k e i t × i A u g e n l i d

[0033] A calculation of the risk assessment (I E ), as shown in Equation 1, can be determined from three eye parameters (or “indices”):

[0034] Gaze orientation (i Ausrichtung ), corneal immobility (i Bewegungslosigkeit ) and eyelid closure (i Augenlid). In a non-limiting example implementation—provided for demonstration purposes only—each index may be assigned a "score" between 0 and 3, reflecting a level of hazard associated with the respective parameter, with a lower score indicating a lower degree of distraction and resulting hazard, while a higher score indicates a higher degree of distraction and resulting hazard. This example implementation will be referenced throughout the example calculations provided below.

[0035] As mentioned above, a calculation of the hazard rating (I E) also depend on the current speed of the vehicle. In other words, the measured eye parameters can be interpreted in relation to the vehicle speed. For example, the time period (T) during which the driver's eye tracks and the hazard rating (I E ) directly depend on the vehicle speed, whereby the time period (T) is equal to 180 / speed (in miles per hour (mph)). This means that the hazard rating (I E ) can be calculated according to Equation 1 every 180 / speed (mph) seconds. For example, for a vehicle traveling at 30 mph (approximately 48 km / h), T = 6 seconds. For a vehicle traveling at 60 mph (approximately 97 km / h), T = 3 seconds. Consequently, the eye tracking time (T) decreases as the vehicle speed increases.

[0036] Again referring to equation 1, the first index used in calculating the hazard score (I E ) is involved, the gaze orientation index (i Ausrichtung ), which is the ratio of the summation of changes in the driver's gaze angle normalized to a predefined acceptable degree or amount of deviation (in degrees (θ)) from the road ahead or oncoming traffic. For example, the predefined acceptable amount of deviation may be 10 degrees (this amount is merely exemplary and may vary according to the desired sensitivity of the system). Consequently, if the deviation of the driver's gaze angle with respect to the road ahead or oncoming traffic is never greater than 10 degrees, the index will be small (e.g., 1 or less). The gaze orientation index (i Ausrichtung) increases, however, while the deviation of the driver's viewing angle from the road ahead begins to exceed the predefined acceptable level (e.g., 2 or 3). For example, the orientation index (i Ausrichtung ) can be calculated according to equation 2 below: i A u s r i c h t u n g = ∫ T i T + T i d θ t d t / ∫ T i T i + T d θ = 10 d t

[0037] The second index used in calculating the risk assessment (I E ) is involved, the corneal motionlessness index (i Bewegungslosigkeit ), which measures, for example, the state of daydreaming. The corneal immobility index (i Bewegungslosigkeit ) is a function of the vehicle speed and the number of eyeball / corneal movements per unit time. For example, the corneal motionlessness index (i Bewegungslosigkeit ) can be calculated according to equation 3 below: i Bewegungslosigkeit = { 1, f ü r 4 + H o r n h a u t b e w e g u n g e n i n T s − D a u e r 2, f ü r 2 o d e r 3 H o r n h a u t b e w e g u n g e n i n T s − D a u e r 3, f ü r 1 o d e r 0 H o r n h a u t b e w e g u n g e n i n T s − D a u e r

[0038] According to equation 3, in a case where only one or zero corneal movements are recorded over a given period of time (e.g., a deep daydream state), the value of this index will be high (e.g., 3). If two or three corneal movements are recorded over the given period of time, the value of this index decreases (e.g., 2). Meanwhile, if four or more corneal movements are recorded over the given period of time, the value of this index decreases even further (e.g., 1 or less). In particular, a very low corneal motionlessness index (i Bewegungslosigkeit ), which represents a high number of eye movements over a given period of time, indicate that the driver is distracted. This can be done, for example, by assigning a high (i.e., poor) score to the corneal motionlessness index (i Bewegungslosigkeit ) should be taken into account if the number of eye movements detected exceeds a predefined number that is considered unsafe.

[0039] The third index used in calculating the risk assessment (I E ) is involved, the eyelid closure index (i Augenlid ), which tracks the driver's fatigue / drowsiness. Similar to the Corneal Immobility Index (i Bewegungslosigkeit ) is the eyelid closure index (i Augenlid ) is a function of the vehicle speed. For example, the eyelid closure index (i Augenlid ) can be calculated according to equation 4 below: i Augenlid = { 1, f ü r w e n i g e r a l s 15 G e s c h w i n d i g k e i t i n T s − D a u e r g e s c h l o s s e n e s A u g e n l i d 2, f ü r 15 G e s c h w i n d i g k e i t b i s 30 G e s c h w i n d i g k e i t i n T s − D a u e r g e s c h l o s s e n e s A u g e n l i d 3, f ü r m e h r a l s 30 G e s c h w i n d i g k e i t i n T s − D a u e r g e s c h l o s s e n e s A u g e n l i d

[0040] As an example, in a case where the vehicle is traveling at a speed of 30 mph, the tracking duration (T) is six seconds ( 180 / 30 mph = 6 seconds). According to equation 4, if the driver is sufficiently alert and the driver's eyelid is closed for a maximum time of 0.5 seconds ( 15 / 30mph = 0.5 seconds) during the six-second period, the index must be low (e.g. 1 or less). If the driver's eyelid closes for a maximum time between 0.5 seconds ( 15 / 30 mph = 0.5 seconds) and 1.0 seconds ( 30 / 30 mph = 1.0 seconds) during the six-second period, the index will increase (e.g., 2). If the driver's eyelid closes for a maximum time exceeding 1.0 seconds ( 30 / 30 mph = 1.0 seconds) during the period of six seconds, the index will be high (e.g. 3).

[0041] Meanwhile, if the vehicle is traveling at a speed of 60 mph, the tracking time (T) is set to three seconds ( 180 / 60mph = 3 seconds). Consequently, the duration (T) decreases as the vehicle speed increases. Similarly, the eyelid closure benchmarks change as the vehicle speed changes. For example, for a vehicle traveling at 60 mph, the eyelid closure index (i Augenlid ) may only be small (e.g. 1 or less) if the driver's eyelid is open for a maximum time of 0.25 seconds ( 15 / 60 mph = 0.25 seconds) for a period of three seconds.

[0042] It should be understood that the hazard score calculations described above are provided for demonstration purposes only, and the scope of the present disclosure is not limited thereto. That is, the hazard score calculations described above represent only one possible way to configure a hazard score that indicates a driver's driving condition according to a variety of measured ocular parameters.

[0043] Once a hazard score has been calculated, indicating a driver's driving condition, the score can be compared to a hazard threshold to determine whether the driver is operating the vehicle in an unsafe manner (e.g., distracted, drowsy, daydreaming, etc.). More specifically, the driver may be determined to be operating the vehicle in an unsafe manner if the calculated hazard threshold exceeds the hazard threshold. In response to determining that the driver is performing an unsafe operation, a corrective action may be executed, as explained in more detail below.

[0044] The hazard threshold may be set considering the range of possible hazard scores. As a non-limiting example, the hazard threshold may be set equal to 0.2 * the maximum possible hazard score. Furthermore, the hazard threshold may be controlled according to a desired sensitivity level. For example, lowering the hazard threshold increases hazard detection sensitivity by increasing the probability of detecting that the driver is driving unsafely. Conversely, increasing the hazard threshold decreases hazard detection sensitivity by decreasing the probability of detecting that the driver is driving unsafely.

[0045] The hazard threshold may change in real time according to environmental or contextual conditions. For example, if the vehicle is traveling straight at 20 mph (approximately 32 km / h), the threshold may be different compared to if the vehicle is traveling at 60 mph on winding roads. Other factors may affect the hazard threshold, such as a direction of travel, a travel speed, a time of day, a trip duration, a trip origin, a trip destination, a number of occupants in the vehicle, an amount of noise in the vehicle, use of a mobile device, and the like. Furthermore, a history of previously established hazard thresholds may be retrieved and used to establish the hazard threshold. In one example, the hazard threshold may be initialized to be equal to the most recently established hazard threshold.

[0046] If the calculated hazard score exceeds the hazard threshold, a corrective action can be taken. For example, a warning can be provided to the driver. The warning can be visual or audible, depending on the time of day, the noise level in the vehicle, etc. The warning can also be tactile, i.e., a physical stimulus such as a nudge or "stickshaker."

[0047] Additionally, the vehicle's speed may be automatically reduced in response to determining that the hazard score exceeds the hazard threshold. In some cases, the reduction in vehicle speed may be based on the driver receiving the warning indicating that the driver is driving unsafely and ignoring the warning, continuing to drive unsafely according to the calculated hazard score.

[0048] Furthermore, if the driver continues to ignore the warnings provided by the warning and speed reduction, the vehicle speed may be further reduced by continuing to drive unsafely. In other words, the vehicle speed may be reduced a second time if the calculated hazard score continues to exceed the hazard threshold after the vehicle speed has been reduced the first time. These corrective actions can work to correct and even train the driver to practice safer driving habits.

[0049] The Fig. 3A and Fig. 3B illustrate an exemplary simplified method for monitoring a driver condition using corneal reflection sensing. The method 300 can step 310 start and with the step 315continue, as described in more detail herein, by detecting light information reflected from a driver's eye by an optical sensor mounted in a vehicle, measuring a plurality of eye parameters based on the sensed light information, calculating a hazard score according to the measured parameters, and executing a corrective action in response to determining that the calculated hazard score exceeds a hazard threshold.

[0050] In step 315 Input data can be collected after system initialization. For example, light reflected from a driver's cornea can be collected by an optical sensor mounted in a vehicle 100 be scanned, as in the Fig. 1A and Fig. 1B. The sensed light information can be converted into electrical signals and transmitted to a control unit for processing. Furthermore, vehicle speed can be determined and used as additional input for calculating the hazard score.

[0051] In step 320 A variety of eye parameters can be measured using the sampled light information. For example, in the step 325 shown, a gaze orientation (i Ausrichtung), which indicates an alignment of the driver's gaze with a road ahead of the vehicle, can be measured to detect whether the driver's gaze is angularly misaligned with the road ahead and / or oncoming traffic. In particular, using the sampled light information, it can be determined whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level (e.g., 5 to 30 degrees). As another example, corneal immobility (i Bewegungslosigkeit, which indicates a motionlessness of the driver's cornea, can be measured during a predefined period of time in order to determine whether the driver's eyes are either too motionless, indicating that the driver is daydreaming, or moving too quickly, indicating that the driver is distracted. For this purpose, a number of movements of the driver's cornea during the predefined period of time can be determined using the sampled light information. As yet another example, a blink (i Augenlid), which indicates a closure amount of a driver's eyelid during a predefined period of time, can be measured to detect whether the driver is drowsy. For this purpose, a maximum period of time that the driver's eyelid is closed during the predefined period of time can be determined using the sampled light information (for example, if the cornea is temporarily not detected, it can be inferred that the eyelid is partially or completely closed). It should be understood that any number and / or type of eye parameter can be determined based on the data detected by the optical sensor. 100 sampled light information can be measured, and consequently the parameters described above are provided for demonstration purposes only.

[0052] In step 330The hazard score can be calculated based on the measured eye parameters. As shown in step 335, a hazard score (I E ) can be calculated according to the following equation 1: Ι E = i A u s r i c h t u n g × i B e w e g u n g s l o s i g k e i t × i A u g e n l i d

[0053] A calculation of the risk assessment (I E ) can be determined, as shown in equation 1, from three eye parameters (or “indices”) determined in step 325 be measured, depend on: gaze orientation (i Ausrichtung ), corneal immobility (i Bewegungslosigkeit ) and eyelid closure (i Augenlid ). In a non-limiting exemplary implementation - intended for demonstration purposes only - each index can be given a "score" and the hazard score (I E) is equal to the product of all ratings. For example, a lower rating may indicate a lower level of distraction and a lower resulting hazard, while a higher rating may indicate a higher level of distraction and a higher resulting hazard.

[0054] In step 340 Additional input data may be required for the purpose of calculating the hazard threshold (step 345) are collected. As explained above, the hazard threshold can be set considering the range of possible hazard scores. As a non-limiting example, the hazard threshold can be set equal to 0.2 * maximum possible hazard score. Furthermore, the hazard threshold can be controlled according to a desired sensitivity level. The hazard threshold can also change in real time according to environmental or contextual conditions determined in step 340 For example, the threshold may change according to factors (input data) such as a traveling direction, a traveling speed, a time of day, a traveling duration, a traveling starting point, a traveling destination, a number of occupants in the vehicle, an amount of noise in the vehicle, a use of a mobile device, and the like.

[0055] In step350 the calculated hazard rating (I E ) are compared with the hazard threshold. If the hazard assessment (I E ) exceeds the hazard threshold, it can be determined that the driver is driving unsafely (e.g. due to a lack of attention), and a corrective action can be carried out (step 355 ) to correct and even train the driver to adopt safe driving habits. If the hazard rating (I E ) does not exceed the hazard threshold, the procedure 300 return to an earlier step to recalculate the hazard score and / or hazard threshold.

[0056] The procedure 300 ends illustratively at the step 355 The techniques by which the steps of the procedure 300 can be performed, as well as additional procedures and parameters are described in detail herein.

[0057] It should be noted that the Fig. 3A and Fig. 3B are merely illustrative examples, and certain other steps may be included or excluded as desired. While a particular order of steps is shown, this order is merely illustrative, and any suitable arrangement of steps may be used without departing from the scope of the embodiments herein. Furthermore, the illustrated steps may be modified in any suitable manner within the scope of the present claims.

[0058] Fig. 4 illustrates an exemplary simplified method for performing a corrective action in response to determining that the driver is driving in an unsafe manner. The method 400 can be started with the step 410 start and with the step 415proceed in which, as described in more detail herein, a series of corrective actions are carried out depending on the response or reaction elicited by the driver and, in particular, on whether the driver pays attention to the warning signals.

[0059] In step 410 the calculated hazard rating (I E ) are compared with the hazard threshold. If the hazard assessment (I E ) exceeds the hazard threshold, it may be determined that the driver is driving unsafely (e.g., due to a lack of attention), and an initial corrective action may be taken (step 415) to correct and even train the driver to adopt safer driving habits.

[0060] In step 415An initial corrective action may be executed in response to determining that the calculated hazard score exceeds the hazard threshold. For example, the initial corrective action may include a warning provided to the driver. The warning may be visual or audible, depending on the time of day, the noise level in the vehicle, etc. The warning may also be tactile, i.e., a physical stimulus such as a nudge or "stickshaker."

[0061] After the driver has been given the warning in step 415 provided, the calculated hazard rating (I E ) are compared again with the hazard threshold (step 420) to verify whether the driver has changed his driving state (e.g., whether the driver is less distracted, more alert, more concentrated, etc.). If the calculated hazard score no longer exceeds the hazard threshold, which means that the driver has responded appropriately to the hazard threshold determined in step 410 has responded to the warning provided, the procedure can 400 to step 410 return.

[0062] If the calculated hazard score exceeds the hazard threshold in step 420 which means that the driver has to exceed the speed limit 415 On the other hand, if the driver has ignored the warning provided, the vehicle speed can be reduced automatically (step 425). In one example, the vehicle speed may be reduced by an initial amount, such as 5%, 10%, etc. The automatic reduction in vehicle speed should warn the driver that their driving behavior is unsafe.

[0063] After reducing the vehicle speed by a first amount in step 425 the calculated hazard rating (I E ) must be compared again with the hazard threshold (step 430 ) to verify once again whether the driver has changed his driving state (e.g. whether the driver is less distracted, more alert, more concentrated, etc.). If the calculated hazard score no longer exceeds the hazard threshold, which means that the driver has responded to the changes made in step 425 If the driver has reacted appropriately to the reduction in vehicle speed by changing his driving behaviour, the procedure can be 400 to step410 return.

[0064] On the other hand, if the calculated hazard score exceeds the hazard threshold in step 430 which means that the driver has to exceed the speed limit 425 If the driver has once again ignored the corrective action taken, the vehicle speed can be reduced automatically a second time (step 435 ). In one example, the vehicle speed may be reduced by a second amount, such as 15%, 20%, etc., that is greater than the first reduction amount in step 425 The subsequent automatic reduction in vehicle speed should further warn the driver that their driving behavior is unsafe. As explained above, these corrective actions can work to correct and even train the driver to practice safer driving habits.

[0065] The procedure 400 ends illustratively at the step435 The techniques by which the steps of the procedure 400 can be performed, as well as additional procedures and parameters are described in detail herein.

[0066] It should be noted that the Fig. 4 are merely illustrative examples, and certain other steps may be included or excluded as desired. Further, while a particular order of steps is shown, this order is merely illustrative, and any suitable arrangement of steps may be used without departing from the scope of the embodiments herein. Furthermore, the illustrated steps may be modified in any suitable manner within the scope of the present claims.

[0067] Consequently, techniques are described herein that address driving behavior and reduce the likelihood of collisions that occur due to a driver's lack of concentration on the road ahead or nearby traffic. The system described herein dynamically adapts to a driver's given normal driving pattern. It can also consider cultural, social, and geographical constraints when calculating a danger score and / or a danger threshold. The effect is a reduction in vehicle accidents and an increase in driving safety. Furthermore, drivers can be trained to pay greater attention to their respective driving conditions; otherwise, corrective actions are executed, such as an automatic reduction in vehicle speed.

[0068] While illustrative embodiments have been shown and described that provide driver condition monitoring using corneal reflection sensing, it should be understood that various other adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, while reference is primarily made herein to corneal reflection techniques for the purpose of eye tracking, the claimed embodiments are also applicable to other eye tracking techniques. Therefore, the embodiments of the present disclosure may be appropriately modified within the scope of the present claims.

[0069] The foregoing description has been directed to embodiments of the present disclosure. However, it will be apparent that other variations and modifications may be made to the described embodiments with the attainment of some or all of the advantages thereof. Accordingly, this description is to be taken only as an example and not to otherwise limit the scope of the embodiments herein. Therefore, it is the object of the appended claims to cover all such variations and modifications that come within the true spirit and scope of the embodiments herein.

Claims

[1] Method comprising: Sensing light information reflected from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; Measuring a plurality of eye parameters based on the sampled light information by a control unit installed in the vehicle; Calculating a hazard score indicating a driving condition of the driver according to the plurality of eye parameters by the control unit; Determining by the control unit whether the calculated hazard score exceeds a hazard threshold; and Execution of a corrective action by the control unit in response to determining that the calculated hazard score exceeds the hazard threshold. [2] The method of claim 1, wherein the optical sensor is mounted on a rearview mirror of the vehicle. [3] The method of claim 2, wherein a light source is coupled to the optical sensor. [4] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information by the control unit. [5] The method of claim 4, wherein measuring gaze orientation comprises: Determination by the control unit whether the driver's gaze deviates from the road in front of the vehicle beyond a predefined acceptable level. [6] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit. [7] The method of claim 6, wherein measuring corneal immobility comprises: Determining a number of movements of the driver's cornea during the time period by the control unit. [8] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sensed light information by the control unit. [9] The method of claim 8, wherein measuring eyelid closure comprises: Determining by the control unit a maximum period of time that the driver's eyelid is closed during the period. [10] The method of claim 1, wherein calculating the hazard score comprises: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sensed light information by the control unit; Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit; Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information by the control unit; and Calculate the hazard score using the measured gaze orientation, corneal immobility and eyelid closure by the control unit. [11] The method of claim 1, further comprising: Control of the hazard threshold according to a desired sensitivity by the control unit. [12] The method of claim 1, further comprising: Adjustment of the hazard threshold in real time by the control unit according to one or more factors selected from a group consisting of: a direction of travel, a speed of travel, a time of day, a duration of travel, a starting point of the trip, a destination of the trip, a number of occupants in the vehicle, an amount of noise in the vehicle and use of a mobile device. [13] The method of claim 1, further comprising: Determination of previously defined hazard thresholds by the control unit; and Setting of the hazard threshold by the control unit based on the previously defined hazard thresholds. [14] The method of claim 1, further comprising: Applying a filter to the sampled light information by the control unit to remove noise from it. [15] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Determining a current vehicle speed by the control unit; and Measuring the plurality of eye parameters further based on the determined current vehicle speed by the control unit. [16] The method of claim 1, wherein performing the corrective action comprises: Providing the driver with a visual, audible or tactile warning. [17] The method of claim 1, wherein performing the corrective action comprises: Reducing a vehicle speed by the control unit. [18] The method of claim 16, wherein performing the corrective action comprises: Determining by the control unit whether the calculated hazard score continues to exceed the hazard threshold after reducing the vehicle speed; and further reduction of the vehicle speed by the control unit if the calculated hazard score continues to exceed the hazard threshold. [19] System comprising: an optical sensor mounted in a vehicle and configured to sense light information reflected from an eye of the driver of the vehicle; and a control unit installed in the vehicle and configured to: Measuring a variety of eye parameters based on the sampled light information; Calculating a hazard score indicating a driving condition of the driver according to the plurality of eye parameters; Determine whether the calculated hazard score exceeds a hazard threshold; and Implementing a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold. [20] A non-transitory computer-readable medium containing program instructions executable by a control unit installed in a vehicle, the program instructions, when executed, causing the control unit to: measures a plurality of eye parameters based on light information reflected from an eye of a driver of the vehicle and sensed by an optical sensor mounted in the vehicle; a hazard score indicating a driver's driving condition is calculated according to the plurality of eye parameters; determines whether the calculated hazard score exceeds a hazard threshold; and takes a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.