DRIVER CONDITION MONITORING USING CORNEAL REFLECTION DETECTION

Abstract

Method comprising: Sensing light information reflected from an eye of a driver of a vehicle using an optical sensor (100) mounted in the vehicle; Measuring a plurality of eye parameters based on the sampled light information by a control unit installed in the vehicle; Calculating a hazard score indicating a driving condition of the driver according to the plurality of eye parameters by the control unit; Determining by the control unit whether the calculated hazard score exceeds a hazard threshold; and Execution of a corrective action by the control unit in response to determining that the calculated hazard score exceeds the hazard threshold, where the calculation of the risk assessment comprises the following: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information by the control unit; Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit; Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information by the control unit; and Calculate the hazard score using the measured gaze orientation, corneal immobility and eyelid closure by the control unit.

Description

TECHNICAL FIELD

[0001] The present disclosure generally relates to monitoring a condition of a driver of a vehicle, and more particularly to monitoring a driver condition using corneal reflection sensing. BACKGROUND

[0002] Many vehicle accidents occur due to distractions that impair a driver's ability to focus on nearby traffic and the road ahead. For example, a driver's focus may be diverted when searching for a destination, looking at a cell phone, or talking to passengers. Additionally, a driver's gaze may stray from the road if they are daydreaming, or their eyes may close completely if they become drowsy.

[0003] In an attempt to prevent driver distraction, some vehicles employ devices that monitor the driver's eyes. However, it can be difficult to accurately assess when a driver's distraction level becomes problematic. For example, the required level of concentration may be heightened when driving at night, at high speeds, on winding roads, in urban areas, etc. Analyzing driver activity within the vehicle can also be complicated without knowing the driver's unique behavioral history.

[0004] DE 600 09 057 T2, DE 10 2005 058 678 A1, DE 10 2008 043 200 A1, DE 10 2014 100 352 A1, DE 10 2014 220 759 A1 and DE 11 2016 007 124 T5 each disclose a method comprising: sampling light information reflected from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; measuring a plurality of eye parameters based on the sampled light information by a control unit installed in the vehicle; calculating a hazard score indicative of a driving state of the driver according to the plurality of eye parameters by the control unit; Determining, by the control unit, whether the calculated hazard score exceeds a hazard threshold; and executing, by the control unit, a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold. OVERVIEW

[0005] It is the object of the present disclosure to provide techniques for tracking a driver's eye activity using an optical sensor mounted in a vehicle. This object is achieved by a method having the features of claim 1 and a system having the features of claim 18. Advantageous developments can be found in the subclaims.

[0006] If the tracked eye activity indicates that the driver is not adequately concentrating on the road ahead, corrective action can be taken to correct and train the driver to develop safer driving habits. A variety of eye parameters can be measured to determine whether the driver has an adequate level of concentration. It may be determined that adequate concentration for driving is not present if, for example, the driver's cornea is not aligned with the direction of oncoming traffic or the road ahead, or the cornea is either too still or motionless (indicating that the driver is daydreaming), or moving too quickly (indicating that the driver is distracted), or the cornea is being scanned less than usual because the driver's eyelids are closing for long periods of time. Furthermore, a hazard score ora hazard score taking into account the parameters described above or other parameters is calculated and the calculated hazard score is compared with a hazard threshold to determine whether the driver's distraction level has reached unsafe levels.

[0007] According to embodiments of the present disclosure, a method includes: sampling light information reflected from an eye of a driver of a vehicle using an optical sensor mounted in the vehicle; measuring a plurality of eye parameters based on the sampled light information; calculating a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determining whether the calculated hazard score exceeds a hazard threshold; and executing a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.Calculating the hazard score comprises: measuring, by the control unit, a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information; measuring, by the control unit, a corneal immobility indicating a motionlessness of a driver's cornea during a period of time based on the sampled light information; measuring, by the control unit, a blink closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information; and calculating, by the control unit, the hazard score using the measured gaze orientation, corneal immobility, and blink closure.

[0008] The optical sensor can be mounted on a vehicle's rearview mirror. A light source can be coupled to the optical sensor.

[0009] Measuring the plurality of eye parameters may include, for example, measuring a gaze orientation indicating an alignment of the driver's gaze with a road ahead of the vehicle based on the sampled light information. Measuring the gaze orientation may include determining whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level.

[0010] Measuring the plurality of eye parameters may further include, for example, measuring a corneal immobility indicating a corneal immobility of the driver during a period of time based on the sampled light information. Measuring the corneal immobility may further include determining a number of movements of the driver's cornea during the period of time.

[0011] Measuring the plurality of eye parameters may further include, for example, measuring a blink rate indicating an amount of closure of a driver's eyelid during a period of time based on the sampled light information. Measuring the blink rate may include determining a maximum amount of time the driver's eyelid is closed during the period of time.

[0012] Measuring the plurality of eye parameters may further include, for example: determining a current speed of the vehicle; and measuring the plurality of eye parameters further based on the determined current vehicle speed.

[0013] Additionally, the method may further include controlling the hazard threshold according to a desired sensitivity. Accordingly, the method may further include adjusting the hazard threshold in real time according to one or more factors selected from a group consisting of: a direction of travel, a speed of travel, a time of day, a duration of travel, a starting point of the trip, a destination of the trip, a number of occupants in the vehicle, an amount of noise in the vehicle, and use of a mobile device. The method may further include determining predetermined hazard thresholds; and setting the hazard threshold based on the predetermined hazard thresholds.

[0014] The method may further include applying a filter to the sampled light information to remove noise therefrom.

[0015] Implementing the corrective action may include, for example, providing the driver with a visual, audible or tactile warning.

[0016] Carrying out the corrective action may further include, for example, reducing a vehicle speed.

[0017] Executing the corrective action may further include, for example, determining whether the calculated hazard score continues to exceed the hazard threshold after reducing the vehicle speed; and further reducing the vehicle speed if the calculated hazard score continues to exceed the hazard threshold.

[0018] According to embodiments of the present disclosure, a system further includes: an optical sensor mounted in a vehicle and configured to sample light information reflecting from an eye of a driver of the vehicle; and a controller disposed in the vehicle and configured to: measure a plurality of eye parameters based on the sampled light information; calculate a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determine whether the calculated hazard score exceeds a hazard threshold; and perform a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.Calculating the hazard score comprises: measuring, by the control unit, a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information; measuring, by the control unit, a corneal immobility indicating a motionlessness of a driver's cornea during a period of time based on the sampled light information; measuring, by the control unit, a blink closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information; and calculating, by the control unit, the hazard score using the measured gaze orientation, corneal immobility, and blink closure.

[0019] Furthermore, according to embodiments of the present disclosure, a non-transitory computer-readable medium includes program instructions executable by a control unit implemented in a vehicle, the program instructions, when executed, causing the control unit to: measure a plurality of eye parameters based on light information reflected from an eye of a driver of the vehicle and sensed by an optical sensor mounted in the vehicle; calculate a hazard score indicative of a driving condition of the driver according to the plurality of eye parameters; determine whether the calculated hazard score exceeds a hazard threshold; and execute a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold.Calculating the hazard score comprises: measuring, by the control unit, a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information; measuring, by the control unit, a corneal immobility indicating a motionlessness of a driver's cornea during a period of time based on the sampled light information; measuring, by the control unit, a blink closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information; and calculating, by the control unit, the hazard score using the measured gaze orientation, corneal immobility, and blink closure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments herein may be better understood by reference to the following description in conjunction with the accompanying drawings, in which like reference numerals indicate identical or functionally similar elements and of which: the Fig. 1A and Fig. 1B illustrates an exemplary optical sensor system for tracking eye activity; Fig. 2 illustrates an example of the driver's eye movements; the Fig. 3A and Fig. 3B illustrates an exemplary simplified method for monitoring a driver condition using corneal reflection sensing; Fig. 4 illustrates an exemplary simplified method for performing a corrective action in response to determining that the driver is driving in an unsafe manner.

[0021] It should be understood that the above-referenced drawings are not necessarily to scale and present a somewhat simplified representation of various preferred features illustrative of the basic principles of the disclosure. The specific embodiments of the present disclosure, including, for example, particular dimensions, orientations, locations, and shapes, will be determined in part by the particular intended application and environment of use. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. As will be appreciated by those skilled in the art, the described embodiments may be modified in many different ways without departing from the spirit or scope of the present disclosure. Furthermore, like reference numerals refer to like elements throughout the specification.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a" and "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of the recited features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed elements.

[0024] It is to be understood that the term "vehicle" or "vehicular" or other similar term used herein includes motor vehicles in general, such as passenger cars, including off-road vehicles (SUVs), buses, trucks, various business cars, watercraft, including a variety of boats and vessels, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from feedstocks other than petroleum). As referred to herein, a hybrid vehicle is a vehicle that has two or more power sources, such as both gasoline-powered and electric-powered vehicles.

[0025] Furthermore, it is to be understood that one or more of the following methods, or aspects thereof, may be performed by at least one control unit. The term "control unit" may refer to a hardware device including a memory and a processor. The memory is configured to store program instructions, and the processor is specifically programmed to execute the program instructions to perform one or more processes described further below. Furthermore, it is to be understood that the following methods may be performed by a device having the control unit in conjunction with one or more other components, as would be understood by someone of ordinary skill in the art.

[0026] Furthermore, the control unit of the present disclosure may be embodied as non-transitory computer-readable media containing executable program instructions executed by a processor, controller, or the like. Examples of computer-readable media include, but are not limited to, read-only memory, random access memory, compact disc read-only memory (CD-ROMs), magnetic tape, floppy disks, flash drives, smart cards, and optical data storage devices. The computer-readable recording medium may also be distributed throughout a computer network such that the program instructions are stored and executed in a distributed manner, e.g., by a telematics server or a controller area network (CAN).

[0027] Turning now to embodiments of the present disclosure, the disclosed techniques utilize a vehicle-mounted optical sensor to track a driver's eye activity. Using the information acquired by the optical sensor, a variety of eye parameters are measured to determine whether the driver is maintaining an appropriate level of concentration. The measured eye parameters may include, for example (without limitation), gaze alignment, which indicates alignment of a driver's gaze with a road ahead of the vehicle, corneal stillness, which indicates stillness of a driver's cornea for a period of time, eyelid closure, which indicates an amount of closure of an eyelid of the driver for a period of time, and so on.A hazard score that considers the measured parameters can be calculated to quantify the hazard level associated with the driver's attention. The calculated hazard score can be compared to a hazard threshold to determine whether the driver's distraction level has reached unsafe levels. Furthermore, if the hazard score exceeds the hazard threshold (controlled according to a desired sensitivity), indicating that the driver is not adequately focused on the road ahead, corrective action can be taken to correct and train the driver to develop safer driving habits.

[0028] The Fig. 1A and Fig. 1B illustrate an exemplary optical sensor system for tracking eye activity. As in Fig. 1A, an optical sensor 100 may be mounted in a vehicle. The optical sensor 100 may be implemented within the passenger compartment of a vehicle in a suitable manner. The optical sensor 100 may, for example, be mounted on a rearview mirror 110 of the vehicle (see Fig. 1A), a windshield, a dashboard, or the like. The optical sensor 100 can be mounted externally or integrated into the rearview mirror 110, the windshield, the dashboard, etc.

[0029] The optical sensor 100 may be any sensor suitable for detecting / sensing light (referred to herein as "light information") and converting the light into electronic signals, such as a video camera, an infrared camera, or other specially designed optical sensor, as would be understood by someone of ordinary skill in the art. In this regard, corneal reflection techniques, which are well known in the art, may be employed to detect the eye activity of a driver of the vehicle in which the optical sensor 100 is mounted. In particular, the optical sensor 100 may sample light (i.e., "light information") reflected from the driver's cornea, as in Fig. 1B. The light reflected from the driver's cornea may be emitted, for example, from a light source coupled to the optical sensor 100 (not shown), a light source located within the vehicle, the environment outside the vehicle, and the like.

[0030] Using corneal reflection techniques as described above, the optical sensor 100 is capable of sensing light reflected from the driver's cornea and constantly transmitting such information to a control unit (not shown) for processing, thus enabling the position and movements of the cornea to be continuously tracked in real time. For example, Fig. 2 is an exemplary representation of a driver's eye movements relative to the road / environment in front of the vehicle, as tracked by the optical sensor 100. In the event that excessive noise exists, e.g., due to vehicle-related movements, one or more filters, such as a Kalman filter, a low-pass filter, or the like, or other signal conditioning may be applied to the sampled light information to eliminate such noise, as would be understood by someone of ordinary skill in the art.

[0031] The light information sampled by the optical sensor 100 can be used by the controller to measure a variety of eye parameters. As an example, gaze alignment, which indicates an alignment of the driver's gaze with a road ahead of the vehicle, can be measured to detect whether the driver's gaze is misaligned with the road ahead and / or oncoming traffic. In particular, using the sampled light information, it can be determined whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level (e.g., 5 to 30 degrees).As another example, corneal stillness, indicating a lack of movement of a driver's cornea, may be measured for a predefined period of time to detect whether the driver's eyes are either too still, indicating that the driver is daydreaming, or moving too quickly, indicating that the driver is distracted. To this end, a number of movements of the driver's cornea during the predefined period of time may be determined using the sampled light information. As yet another example, eyelid closure, indicating an amount of closure of an eyelid of the driver during a predefined period of time, may be measured to detect whether the driver is drowsy.For this purpose, a maximum period of time that the driver's eyelid is closed during the predefined period can be determined using the sampled light information (for example, if the cornea is temporarily not detected, it can be inferred that the eyelid is partially or completely closed). It should be understood that any number and / or type of eye parameter can be measured based on the light information sampled by the optical sensor 100, and thus, the parameters described above are provided for demonstration purposes only.

[0032] Once a variety of eye parameters indicating activity of the driver's eye(s) have been measured, the measured parameters can be used to determine whether the driver is distracted and failing to pay adequate attention to the road ahead and / or nearby traffic. For this purpose, a hazard score quantifying a driver's driving condition—i.e., a degree of danger posed by the driver due to a lack of attention—can be calculated based on the measured parameters. In some cases, the calculation of the hazard score may depend on the measured eye parameters as well as the current speed of the vehicle.

[0033] In an exemplary case where three eye parameters listed above are measured according to the light information sampled by the optical sensor 100, a hazard score (I E) can be calculated according to the following equation 1: lE=iAlignment×iImmobility×iEyelid

[0034] A calculation of the risk assessment (I E ), as shown in Equation 1, can be determined from three eye parameters (or “indices”): gaze orientation (i Ausrichtung ), corneal immobility (i Bewegungslosigkeit ) and eyelid closure (i Augenlid). In a non-limiting example implementation—provided for demonstration purposes only—each index may be assigned a "score" between 0 and 3, reflecting a level of hazard associated with the respective parameter, with a lower score indicating a lower degree of distraction and resulting hazard, while a higher score indicates a higher degree of distraction and resulting hazard. This example implementation will be referenced throughout the example calculations provided below.

[0035] As mentioned above, a calculation of the hazard rating (I E) also depend on the current speed of the vehicle. In other words, the measured eye parameters can be interpreted in relation to the vehicle speed. For example, the time period (T) during which the driver's eye tracks and the hazard rating (I E ) directly depend on the vehicle speed, whereby the time period (T) is equal to 180 / speed (in miles per hour (mph)). This means that the hazard rating (I E ) can be calculated according to Equation 1 every 180 / speed (mph) seconds. For example, for a vehicle traveling at 30 mph (approximately 48 km / h), T = 6 seconds. For a vehicle traveling at 60 mph (approximately 97 km / h), T = 3 seconds. Consequently, the eye tracking time (T) decreases as the vehicle speed increases.

[0036] Again referring to equation 1, the first index used in calculating the hazard score (I E ) is involved, the gaze orientation index (i Ausrichtung ), which is the ratio of the summation of changes in the driver's gaze angle normalized to a predefined acceptable degree or amount of deviation (in degrees (θ)) from the road ahead or oncoming traffic. For example, the predefined acceptable amount of deviation may be 10 degrees (this amount is merely exemplary and may vary according to the desired sensitivity of the system). Consequently, if the deviation of the driver's gaze angle with respect to the road ahead or oncoming traffic is never greater than 10 degrees, the index will be small (e.g., 1 or less). The gaze orientation index (i Ausrichtung) increases, however, while the deviation of the driver's viewing angle from the road ahead begins to exceed the predefined acceptable level (e.g., 2 or 3). For example, the orientation index (i Ausrichtung ) can be calculated according to equation 2 below: iAlignment=∫TiTi+Tdθtdt / ∫TiTi+Tdθ=10dt

[0037] The second index used in calculating the risk assessment (I E ) is involved, the corneal motionlessness index (i Bewegungslosigkeit ), which measures, for example, the state of daydreaming. The corneal immobility index (i Bewegungslosigkeit ) is a function of the vehicle speed and the number of eyeball / corneal movements per unit time. For example, the corneal motionlessness index (i Bewegungslosigkeit ) can be calculated according to equation 3 below: iMotionlessness={1,for 4+corneal movements in Ts−duration 2,for 2 or 3 corneal movements in Ts−duration 3,for 1 or 0 corneal movements in Ts−duration

[0038] According to equation 3, in a case where only one or zero corneal movements are recorded over a given period of time (e.g., a deep daydream state), the value of this index will be high (e.g., 3). If two or three corneal movements are recorded over the given period of time, the value of this index decreases (e.g., 2). Meanwhile, if four or more corneal movements are recorded over the given period of time, the value of this index decreases even further (e.g., 1 or less). In particular, a very low corneal motionlessness index (i Bewegungslosigkeit), which represents a high number of eye movements over a given period of time, indicate that the driver is distracted. This can be done, for example, by assigning a high (i.e., poor) score to the corneal motionlessness index (i Bewegungslosigkeit ) should be taken into account if the number of eye movements detected exceeds a predefined number that is considered unsafe.

[0039] The third index used in calculating the risk assessment (I E ) is involved, the eyelid closure index (i Augenlid ), which tracks the driver's fatigue / drowsiness. Similar to the Corneal Immobility Index (i Bewegungslosigkeit ) is the eyelid closure index (i Augenlid ) is a function of the vehicle speed. For example, the eyelid closure index (i Augenlid ) can be calculated according to equation 4 below: iEyelid={1, for less than 15 speed in Ts−permanently closed eyelid 2, for 15 speed up to 30 speed in Ts−permanently closed eyelid 3, for more than 30 speed in Ts−permanently closed eyelid

[0040] As an example, in a case where the vehicle is traveling at a speed of 30 mph, the tracking duration (T) is six seconds (180 / 30 mph = 6 seconds). According to Equation 4, if the driver is sufficiently alert and the driver's eyelid closes for a maximum time of 0.5 seconds (15 / 30 mph = 0.5 seconds) during the six-second period, the index will be low (e.g., 1 or less). If the driver's eyelid closes for a maximum time between 0.5 seconds (15 / 30 mph = 0.5 seconds) and 1.0 seconds (30 / 30 mph = 1.0 seconds) during the six-second period, the index will increase (e.g., 2). If the driver's eyelid closes for a maximum time exceeding 1.0 seconds (30 / 30 mph = 1.0 seconds) during the six-second period, the index will be high (e.g., 3).

[0041] Meanwhile, if the vehicle is traveling at 60 mph, the tracking duration (T) is reduced to three seconds (180 / 60 mph = 3 seconds). Consequently, the duration (T) decreases as the vehicle speed increases. Similarly, the eyelid closure benchmarks change as the vehicle speed changes. For example, for a vehicle traveling at 60 mph, the eyelid closure index (i Augenlid ) may only be small (e.g. 1 or less) if the driver's eyelid closes for a maximum time of 0.25 seconds (15 / 60 mph = 0.25 seconds) during the three-second period.

[0042] It should be understood that the hazard score calculations described above are provided for demonstration purposes only, and the scope of the present disclosure is not limited thereto. That is, the hazard score calculations described above represent only one possible way to configure a hazard score that indicates a driver's driving condition according to a variety of measured ocular parameters.

[0043] Once a hazard score has been calculated, indicating a driver's driving condition, the score can be compared to a hazard threshold to determine whether the driver is operating the vehicle in an unsafe manner (e.g., distracted, drowsy, daydreaming, etc.). More specifically, the driver may be determined to be operating the vehicle in an unsafe manner if the calculated hazard threshold exceeds the hazard threshold. In response to determining that the driver is performing an unsafe operation, a corrective action may be executed, as explained in more detail below.

[0044] The hazard threshold may be set considering the range of possible hazard scores. As a non-limiting example, the hazard threshold may be set equal to 0.2 * maximum possible hazard score. Furthermore, the hazard threshold may be controlled according to a desired sensitivity level. For example, lowering the hazard threshold increases hazard detection sensitivity by increasing the likelihood of detecting that the driver is driving unsafely. Conversely, increasing the hazard threshold decreases hazard detection sensitivity by decreasing the likelihood of detecting that the driver is driving unsafely.

[0045] The hazard threshold may change in real time according to environmental or contextual conditions. For example, if the vehicle is traveling straight at 20 mph (approximately 32 km / h), the threshold may be different compared to if the vehicle is traveling at 60 mph on winding roads. Other factors may affect the hazard threshold, such as a direction of travel, a travel speed, a time of day, a trip duration, a trip origin, a trip destination, a number of occupants in the vehicle, an amount of noise in the vehicle, use of a mobile device, and the like. Furthermore, a history of previously established hazard thresholds may be retrieved and used to establish the hazard threshold. In one example, the hazard threshold may be initialized to be equal to the most recently established hazard threshold.

[0046] If the calculated hazard score exceeds the hazard threshold, a corrective action can be taken. For example, a warning can be provided to the driver. The warning can be visual or audible, depending on the time of day, the noise level in the vehicle, etc. The warning can also be tactile, i.e., a physical stimulus such as a nudge or "stickshaker."

[0047] Additionally, the vehicle's speed may be automatically reduced in response to determining that the hazard score exceeds the hazard threshold. In some cases, the reduction in vehicle speed may be based on the driver receiving the warning indicating that the driver is driving unsafely and ignoring the warning, continuing to drive unsafely according to the calculated hazard score. Furthermore, if the driver continues to ignore the warnings provided by the warning and the reduction in speed, the vehicle's speed may be further reduced by continuing to drive unsafely. In other words, the vehicle's speed may be reduced a second time if the calculated hazard score continues to exceed the hazard threshold after the vehicle's speed has been reduced a first time.These corrective measures can work to correct and even train the driver to practice safer driving habits.

[0048] The Fig. 3A and Fig. 3B illustrate an exemplary simplified method for monitoring a driver condition using corneal reflection sensing. The method 300 may begin at step 310 and continue to step 315, where, as described in more detail herein, light information reflected from a driver's eye is sensed by a vehicle-mounted optical sensor, a plurality of eye parameters are measured based on the sensed light information, a hazard score is calculated according to the measured parameters, and corrective action is performed in response to determining that the calculated hazard score exceeds a hazard threshold.

[0049] In step 315, after system initialization, input data may be collected. For example, light reflected from a driver's cornea may be sampled by a vehicle-mounted optical sensor 100, as shown in FIGS. Fig. 1A and Fig. 1B. The sensed light information can be converted into electrical signals and transmitted to a control unit for processing. Furthermore, vehicle speed can be determined and used as additional input for calculating the hazard score.

[0050] In step 320, a variety of eye parameters may be measured using the sampled light information. For example, as shown in step 325, a gaze orientation (i Ausrichtung), which indicates an alignment of the driver's gaze with a road ahead of the vehicle, can be measured to detect whether the driver's gaze is angularly misaligned with the road ahead and / or oncoming traffic. In particular, using the sampled light information, it can be determined whether the driver's gaze deviates from the road ahead of the vehicle beyond a predefined acceptable level (e.g., 5 to 30 degrees). As another example, corneal immobility (i Bewegungslosigkeit), which indicates a motionlessness of the driver's cornea, can be measured during a predefined period of time in order to determine whether the driver's eyes are either too motionless, indicating that the driver is daydreaming, or moving too quickly, indicating that the driver is distracted. For this purpose, a number of movements of the driver's cornea during the predefined period of time can be determined using the sampled light information. As yet another example, a blink (i Augenlid), which indicates a closure amount of a driver's eyelid during a predefined period of time, may be measured to detect whether the driver is drowsy. For this purpose, a maximum period of time that the driver's eyelid is closed during the predefined period of time may be determined using the sampled light information (for example, if the cornea is temporarily not detected, it may be inferred that the eyelid is partially or completely closed). It should be understood that any number and / or type of eye parameter may be measured based on the light information sampled by the optical sensor 100, and thus, the parameters described above are provided for demonstration purposes only.

[0051] In step 330, the hazard score may be calculated based on the measured eye parameters. As shown in step 335, a hazard score (IE ) can be calculated according to the following equation 1: lE=iAlignment×iImmobility×iEyelid

[0052] A calculation of the risk assessment (I E ) may depend, as shown in Equation 1, on three eye parameters (or “indices”) measured in step 325: Gaze orientation (i Ausrichtung ), corneal immobility (i Bewegungslosigkeit ) and eyelid closure (i Augenlie ). In a non-limiting exemplary implementation - intended for demonstration purposes only - each index can be given a "score" and the hazard score (I E ) is equal to the product of all ratings. For example, a lower rating may indicate a lower level of distraction and a lower resulting hazard, while a higher rating may indicate a higher level of distraction and a higher resulting hazard.

[0053] In step 340, additional input data may be collected for the purpose of calculating the hazard threshold (step 345). As explained above, the hazard threshold may be set considering the range of possible hazard scores. As a non-limiting example, the hazard threshold may be set equal to 0.2 * maximum possible hazard score. Furthermore, the hazard threshold may be controlled according to a desired sensitivity level. The hazard threshold may also change in real time according to environmental or contextual conditions detected in step 340.For example, the threshold may change according to factors (input data) such as a traveling direction, a traveling speed, a time of day, a traveling duration, a traveling starting point, a traveling destination, a number of occupants in the vehicle, an amount of noise in the vehicle, a use of a mobile device, and the like.

[0054] In step 350, the calculated hazard rating (I E ) are compared with the hazard threshold. If the hazard assessment (I E ) exceeds the hazard threshold, it can be determined that the driver is driving unsafely (e.g., due to a lack of attention), and a corrective action can be taken (step 355) to correct and even train the driver to adopt safe driving habits. If the hazard score (I E) does not exceed the hazard threshold, the method 300 may return to an earlier step to recalculate the hazard score and / or the hazard threshold.

[0055] The method 300 illustratively ends at step 355. The techniques by which the steps of the method 300 may be performed, as well as additional methods and parameters, are described in detail herein.

[0056] It should be noted that the Fig. 3A and Fig. 3B are merely illustrative examples, and certain other steps may be included or excluded as desired. While a particular order of steps is shown, this order is merely illustrative, and any suitable arrangement of steps may be used without departing from the scope of the embodiments herein. Furthermore, the illustrated steps may be modified in any suitable manner within the scope of the present claims.

[0057] Fig. 4 illustrates an exemplary simplified method for performing a corrective action in response to determining that the driver is driving in an unsafe manner. Method 400 may begin with step 410 and continue with step 415, where, as described in more detail herein, a series of corrective actions are performed depending on the response elicited by the driver, and in particular, whether the driver heeds the warning signals.

[0058] In step 410, the calculated hazard rating (I E ) are compared with the hazard threshold. If the hazard assessment (I E) exceeds the hazard threshold, it may be determined that the driver is driving unsafely (e.g., due to a lack of attention), and an initial corrective action may be taken (step 415) to correct and even train the driver to adopt safer driving habits.

[0059] At step 415, an initial corrective action may be executed in response to determining that the calculated hazard score exceeds the hazard threshold. For example, the initial corrective action may include a warning provided to the driver. The warning may be visual or audible depending on the time of day, the noise level in the vehicle, etc. The warning may also be tactile, i.e., a physical stimulus, such as a nudge or "stickshaker."

[0060] After the warning is provided to the driver in step 415, the calculated hazard score (I E ) may be compared again with the hazard threshold (step 420) to verify whether the driver has changed their driving state (e.g., whether the driver is less distracted, more alert, more focused, etc.). If the calculated hazard score no longer exceeds the hazard threshold, meaning that the driver has appropriately responded to the warning provided in step 410 by changing their driving behavior, the method 400 may return to step 410.

[0061] On the other hand, if the calculated hazard score still exceeds the hazard threshold in step 420, indicating that the driver has ignored the warning provided in step 415, the vehicle speed may be automatically reduced (step 425). In one example, the vehicle speed may be reduced by a first amount, such as 5%, 10%, etc. The automatic reduction in vehicle speed should warn the driver that their driving behavior is unsafe.

[0062] After reducing the vehicle speed by a first amount in step 425, the calculated hazard rating (I E) may be compared again with the hazard threshold (step 430) to verify once again whether the driver has changed their driving state (e.g., whether the driver is less distracted, more alert, more focused, etc.). If the calculated hazard score no longer exceeds the hazard threshold, meaning that the driver has appropriately responded to the reduction in vehicle speed performed in step 425 by changing their driving behavior, the method 400 may return to step 410.

[0063] On the other hand, if the calculated hazard score still exceeds the hazard threshold in step 430, meaning the driver has once again ignored the corrective action taken in step 425, the vehicle speed may be automatically reduced a second time (step 435). In one example, the vehicle speed may be reduced by a second amount, such as 15%, 20%, etc., that is greater than the first reduction amount in step 425. The subsequent automatic reduction in vehicle speed should further warn the driver that their driving behavior is unsafe. As explained above, these corrective actions can work to correct and even train the driver to practice safer driving habits.

[0064] The method 400 illustratively ends at step 435. The techniques by which the steps of the method 400 may be performed, as well as additional methods and parameters, are described in detail herein.

[0065] It should be noted that the Fig. 4 are merely illustrative examples, and certain other steps may be included or excluded as desired. Further, while a particular order of steps is shown, this order is merely illustrative, and any suitable arrangement of steps may be used without departing from the scope of the embodiments herein. Furthermore, the illustrated steps may be modified in any suitable manner within the scope of the present claims.

[0066] Consequently, techniques are described herein that address driving behavior and reduce the likelihood of collisions that occur due to a driver's lack of concentration on the road ahead or nearby traffic. The system described herein dynamically adapts to a driver's given normal driving pattern. It can also consider cultural, social, and geographical constraints when calculating a danger score and / or a danger threshold. The effect is a reduction in vehicle accidents and an increase in driving safety. Furthermore, drivers can be trained to pay greater attention to their respective driving conditions; otherwise, corrective actions are executed, such as an automatic reduction in vehicle speed.

[0067] While illustrative embodiments have been shown and described that provide driver condition monitoring using corneal reflection sensing, it should be understood that various other adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, while reference is primarily made herein to corneal reflection techniques for the purpose of eye tracking, the claimed embodiments are also applicable to other eye tracking techniques. Therefore, the embodiments of the present disclosure may be appropriately modified within the scope of the present claims.

Claims

[1] Method comprising: Sensing light information reflected from an eye of a driver of a vehicle using an optical sensor (100) mounted in the vehicle; Measuring a plurality of eye parameters based on the sampled light information by a control unit installed in the vehicle; Calculating a hazard score indicating a driving condition of the driver according to the plurality of eye parameters by the control unit; Determining by the control unit whether the calculated hazard score exceeds a hazard threshold; and Execution of a corrective action by the control unit in response to determining that the calculated hazard score exceeds the hazard threshold, where the calculation of the risk assessment comprises the following: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sensed light information by the control unit; Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit; Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information by the control unit; and Calculate the hazard score using the measured gaze orientation, corneal immobility and eyelid closure by the control unit. [2] The method of claim 1, wherein the optical sensor (100) is mounted on a rearview mirror (110) of the vehicle. [3] The method of claim 2, wherein a light source is coupled to the optical sensor (100). [4] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information by the control unit. [5] The method of claim 4, wherein measuring gaze orientation comprises: Determination by the control unit whether the driver's gaze deviates from the road in front of the vehicle beyond a predefined acceptable level. [6] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit. [7] The method of claim 6, wherein measuring corneal immobility comprises: Determining a number of movements of the driver's cornea during the time period by the control unit. [8] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sensed light information by the control unit. [9] The method of claim 8, wherein measuring eyelid closure comprises: Determining by the control unit a maximum period of time that the driver's eyelid is closed during the period. [10] The method of claim 1, further comprising: Control of the hazard threshold according to a desired sensitivity by the control unit. [11] The method of claim 1, further comprising: Adjustment of the hazard threshold in real time by the control unit according to one or more factors selected from a group consisting of: a direction of travel, a speed of travel, a time of day, a duration of travel, a starting point of the journey, a destination of the trip, a number of occupants in the vehicle, an amount of noise in the vehicle and use of a mobile device. [12] The method of claim 1, further comprising: Determination of previously defined hazard thresholds by the control unit; and Setting of the hazard threshold by the control unit based on the previously defined hazard thresholds. [13] The method of claim 1, further comprising: Applying a filter to the sampled light information by the control unit to remove noise from it. [14] The method of claim 1, wherein measuring the plurality of eye parameters comprises: Determining a current vehicle speed by the control unit; and Measuring the plurality of eye parameters further based on the determined current vehicle speed by the control unit. [15] The method of claim 1, wherein performing the corrective action comprises: Providing the driver with a visual, audible or tactile warning. [16] The method of claim 1, wherein performing the corrective action comprises: Reducing a vehicle speed by the control unit. [17] The method of claim 15, wherein performing the corrective action comprises: Determining by the control unit whether the calculated hazard score continues to exceed the hazard threshold after reducing the vehicle speed; and further reduction of the vehicle speed by the control unit if the calculated hazard score continues to exceed the hazard threshold. [18] System comprising: an optical sensor (100) mounted in a vehicle and configured to sense light information reflected from an eye of the driver of the vehicle; and a control unit installed in the vehicle and configured to: Measuring a variety of eye parameters based on the sampled light information; Calculating a hazard score indicating a driving condition of the driver according to the plurality of eye parameters; Determine whether the calculated hazard score exceeds a hazard threshold; and Implementing a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold, where the calculation of the risk assessment comprises the following: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information by the control unit; Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit; Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information by the control unit; and Calculate the hazard score using the measured gaze orientation, corneal immobility and eyelid closure by the control unit. [19] A non-transitory computer-readable medium containing program instructions executable by a control unit installed in a vehicle, the program instructions, when executed, causing the control unit to: measures a plurality of eye parameters based on light information reflected from an eye of a driver of the vehicle and sensed by an optical sensor (100) mounted in the vehicle; a hazard score indicating a driver's driving condition is calculated according to the plurality of eye parameters; determines whether the calculated hazard score exceeds a hazard threshold; and takes a corrective action in response to determining that the calculated hazard score exceeds the hazard threshold, where the calculation of the risk assessment comprises the following: Measuring a gaze orientation indicating an alignment of a driver's gaze with a road ahead of the vehicle based on the sampled light information by the control unit; Measuring a corneal immobility indicating immobility of a cornea of ​​the driver during a period of time based on the sampled light information by the control unit; Measuring an eyelid closure indicating an amount of closure of an eyelid of the driver during a period of time based on the sampled light information by the control unit; and Calculate the hazard score using the measured gaze orientation, corneal immobility and eyelid closure by the control unit.

Images