Contactless Driver Alertness Monitoring Device

Abstract

Device (1) for contactless monitoring of the state of alertness of a driver (10) of a rolling, flying or maritime means of transport comprising a millimeter radar (3) for measuring and monitoring the HR cardiac signals and the RR respiratory cycles, an optical sensor (2) for characterizing the posture, the size of the driver (10) and the movements in order to validate drowsiness and distraction activities during driving, a piezoelectric sensor (5) having the capacity to measure the intrinsic vibrations of the means of transport and the occasional vibrations of the road in order to adjust the frequency bands of the digital filters carried out by the microprocessor (9) as well as the adjustment of the equivalent surface of the millimeter radar (3) and an alarm management circuit (8) comprising a light signal as well as an audible, mechanical and visual signal in order to restore certain modes of non-alertness of a driver (10).

Description

Title of the invention: Contactless Driver Alertness Monitoring Device

[0001] The invention relates to the monitoring of the state of vigilance of drivers of a rolling, air or maritime means of transport and the precautions and tools allowing the detection of states of non-vigilance and the reestablishment of the state of optimal control in order to eliminate dangerous situations which could lead to accidents.

[0002] Loss of alertness can be caused by various visual, cognitive, physical or auditory distractions while driving, but drowsiness is considered to be the most restrictive state for safety during maneuvers. Indeed, drowsiness is an unavoidable biological phenomenon having drastic effects on the driver's mind and body. It can be induced by health problems such as apnea, medications, prolonged sleep deprivation or natural circadian rhythm leading to cognitive and behavioral changes and giving rise to probable accidents. In the road transport sector, drowsiness at the wheel is accompanied by the weakening of reaction time, mental processing, judgment and the accuracy of decisions. In addition, accidents related to drowsiness can occur at any time but most often, late in the evening, in the early hours of dawn or in the middle of the afternoon.Despite the seriousness of this condition, the detection of drowsiness while driving or using a controller is an elusive event and difficult to measure objectively. Consequently, several methods have been proposed to detect different states of drowsiness to better understand the risks and incidence of related collisions and their prevention. One of the proposed methods for determining the driver's state of alertness is based on a visual analysis by one or more cameras to assess yawning frequency, blinking frequency, gaze movement, head movement and facial expressions.Document KR102306358B1 is known proposing a method for detecting driver drowsiness by means of optimized preprocessing for eye blink classification using a support vector machine consisting of detecting the image of a driver's face by camera in real time or through an offline video stream to extract the facial features, compare them to predefined data using a classification unit, and subsequently assess the state of drowsiness by determining the state of the eyes. The method proposed by document KR102306358B1 is dependent on a certain level of illumination around the . driver's cab seat, wearing medical glasses and / or sunglasses, but in general, these methods rely on video recording methods that are not generally accepted because they represent intrusions into private or professional life. Detecting the driver's state of alertness can also involve measurements and analyses of vital signals using an electrocardiogram (ECG) placed in the seat belts or on the steering wheel or a jaw pulse oximeter placed on the middle or index finger as well as similar devices often called wrist oximeters, considering certain recommendations related to the fact that the heart rate decreases dramatically during sleep.Also known are documents US2021015382A1 and CN111354157A promulgating an ECG having one or more electrodes placed on the steering wheel to correlate heartbeats with an initial value. Apart from the bulk and complexity of the installations and operation of these devices, the electrodes, especially those placed on the steering wheel, are not effective when the driver wears gloves or has hands moistened by care lotions or sweat, not to mention the degradation of these electrodes by repetitive friction on the active surface. Thanks to advances in wireless or contactless technology, it has become possible to capture cardiac and respiratory activity without contact in ballistocardiographic (BCG) mode.Among the possible ways to obtain information related to physiological data also called ballistocardiography, BCG or contactless vital signals are methods based on millimeter radars capable of detecting small mechanical oscillations of the human chest resulting from heartbeats and respiratory cycles. Document KR20170055352A proposes a system for measuring a biological signal of a driver using a radar operating on an ultra-wide frequency band and comprising a pulse detector, a conversion module for processing a reflected radio signal indicating the frequencies of the driver's heartbeats and respiratory cycles and a controller allowing the processing of these signals over predetermined value ranges and a bio-information analysis module for comparing data and detecting anomalies in order to generate countermeasure alarms.Unfortunately, BCG based on millimeter radar technology is limited in accuracy and repeatability due to sensitivity to environmental conditions such as vibration and temperature as well as driver tilt and posture.

[0003] The present invention relates to a device and methods for monitoring a driver's vigilance in general and in particular fatigue inducing drowsiness in the driver's seat using an RF sensor equipped with a millimeter antenna, hereinafter called millimeter radar, and an optical sensor. characterized in that the physiological parameters of the driver captured by the millimeter radar are compared with the measurements of the spatial coordinates X, Y and Z of a region of interest (RDI) located on the forehead of the face, cheek, ear, shoulder, arm and or chin depending on the position and posture of the driver and the characteristics of the skin perfusion pressure and the useful facial surfaces of the driver.

[0004] In the embodiments, the measurements of the vital signals by the BCG technique by millimeter radar, the device for contactless monitoring of the driver's state of vigilance, is equipped with an ambient temperature sensor allowing the calibration of certain physiological data including the surface temperature of the RDI region as well as the estimation of the internal body temperature of the driver, a piezoelectric sensor allowing the characterization of the ambient or intrinsic vibration of the means of transport and the vibrations generated by the journey characterized in that the vibration signals are processed and evaluated from the start in order to adjust the angles of the waves reflected and received by the millimeter radar based on an estimation of the micro-displacements generated by these vibrations.

[0005] The millimeter radar can be installed in the control cabin of the means of transport considering the requirement of having an optimal viewing angle and range with respect to the surface of the chest in order to measure the mechanical displacements caused by heartbeats and breathing. In the same way, the optical sensor can be installed in the control cabin of the means of transport taking into consideration the ideal position allowing to obtain the best viewing angle in order to locate the forehead or any other region of interest characterized in that the posture and movements of a driver are analyzed and processed in real time. Depending on the position and posture of a driver and following the RDI coordinates, an estimate of the initial heart position is established and considered as a reference point.

[0006] The technologies adopted by the present invention include a millimeter radar emitting short wavelength signals. It emits signals towards the driver's chest along the field of vision which are subsequently reflected and captured by the two-dimensional antenna. These types of radar systems are relatively small and operate on frequencies between 60 and 64 GHz with a wavelength of approximately 4 mm and having the property of detecting the movements of objects down to one fifteenth of a millimeter. The waves received and emitted require a specific transceiver called an RF sensor integrating several receiver and transmitter channels to improve the accuracy of detection of movements, particularly those of the chest, in order to carry out ballistocardiographic measurements.After transmission and reception of the millimeter radar waves, the data is processed by a microprocessor which triggers the filtration sequence in order to separate the different signals of the heartbeat (HR), respiratory rate (RR), on one side and . the random movements of the driver generated by vibrations and posture measured respectively by a piezoelectric sensor and an optical sensor, on the other side. The analysis of the curves of the vital signals HR, RR also called BCG, physiological or ballistocardiographic data and RDI data presents a major indicator allowing the detection of the state of drowsiness and the vigilance in general of a driver.

[0007] The optical sensor sends waves allowing the detection and localization of the regions of interest RDI of a driver based on the distribution and granulation of the temperature. In the embodiments, a step of processing and localizing the regions of interest is characterized in that the data obtained by the optical sensor are sent to a microprocessor and converted into a thermal image followed by a subtraction of the background from a pixelated table. After a Gaussian filtration for a finer processing of the point clouds, a conversion to the gray spectrum level is completed in order to localize the RDI. The coordinates of the RDI, (X, Y, Z), make it possible to follow the movements in translation (X', Y' and Z') and in rotation (a', [3' and y') of the driver in synchronization with the BCG signals in order to estimate the state of vigilance of a driver.

[0008] Contactless monitoring of the state of vigilance of a driver operating a means of transport consists of triggering an initiation phase allowing the verification of the presence of a driver simultaneously validated by the millimeter radar and the optical sensor followed by the measurement or reading of the speed of the means of transport by means of a piezoelectric sensor or communicated by a computer on board the vehicle. If the speed of the means of transport is greater than zero Km / h, the procedure for determining the initial reference values ​​HR0, RRo and RDIo is triggered. When the values ​​remain constant over a predefined period and characterized in that one of the regions of interest RDIo is located relative to the center of the control device such as a steering wheel, a navigation joystick, etc., and the position of the optical sensor located in the control cabin of a means of transport, these data are stored and used as reference values. Thus, an estimation of the coordinates of the core is carried out based on predefined values ​​ZF0 - ZCo = AZFCo and YF0 - YCo = AYFCo where ZF0 and YF0 are the initial reference coordinates of the frontal region of interest RDIF0, and YCo, ZCo are the initial reference coordinates of the core. A validation step allowing the characterization of the reference power of the equivalent surface of the millimeter radar, hereinafter designated SER0 and thus defined by [Math 1] is carried out when the values ​​AZFCo and AYFCo vary.

[0009] Knowing that the constant k is equal to 2ir divided by the wavelength of the millimeter radar, b is the characteristic length of the equivalent surface of the millimeter radar- SER, = 1 z metric and 00 is the initial angle of incidence, [Math 1] becomes:

[0010] [Math.l] CI?” _ ^1 / F Sin(^fow0o)) ' “ ? I î / ï - / j \ u Z“ L Wbsmffd

[0011] [Math.2] sin( kbj ( siu( ) ] kh^iniO 0+^,+i5ji

[0012] where bi is the instantaneous characteristic length of the equivalent surface of the radar, 0; is the instantaneous angle of incidence proportional to the driver's posture, and ô; is the algebraic average angle caused by the ambient vibration and that generated by the road.

[0013] Consequently, and considering that the chest is a rectangular flat surface, a new posture or movement of the driver results in a variation of the angle of incidence 0i, inducing a fluctuation of the SER designated SER; thus specifically described by [Math.2]. In the case of a decrease in the heart rate and respiratory cycles, the stability of the instantaneous SER may indicate that the driver has entered a drowsiness mode signaling a lack of movement. Otherwise, a rapid fluctuation of the SER automatically generates the processing of several HR and RR measurements to assess the state of drowsiness in accordance with the coordinates of the regions of interest provided by the optical sensor.

[0014] In addition to the data sent by the optical sensor allowing the location of the frontal reference points, in this case XF0, YF0 and ZF0, other points defining the upper left part of the driver and that of the upper right part, which can be used to monitor his movements, for example, telephone conversations, ingestion or action on the steering wheel or the steering and navigation lever. The distinctive reference coordinates of normal driving are processed by the microprocessor and recorded in the memory of the device and instantly compared to the new coordinates corresponding to actions classified in relation to distraction during driving. According to the embodiments, the coordinates instantaneous location of any of the regions of interest in the presence of an event captured by the optical sensor, the contactless vigilance monitoring device records it in the memory. If the event is significantly repetitive and impacts safety, alarms that can be audible, mechanical, luminous, visual or by specific display on the human-machine interface of the control instrument also called dashboard, are triggered and simultaneously the duration, the category of distraction and the time stamps are recorded and can only be erased from the memory by a user with certain access rights to this memory. In addition to distraction events and in the case of va- significant nations of the HR and RR values, a sample of measurements is plotted and then derived to the first order with respect to time to determine the slope of the variation characterized in that predefined limit thresholds are transposed to the slope of variation. In the case where the slope exceeds the minimum negative threshold indicating a doubtful zone of non-vigilance, caused by the driver's drowsiness, a validation phase of this doubtful state is triggered by using data from the optical sensor which continuously evaluates the driver's movements. A static movement state confirming the state of non-vigilance characterized in that a percentage of the drowsiness rate is assigned and prepared with a time stamp and at the same time, audible, mechanical, luminous or visual alarms are triggered in a progressive manner following the physiological and physical response of the driver.Otherwise, if the optical sensor data confirms that the driver is performing ordinary movements, apart from the distracting gestures mentioned above, the state is considered vigilant. Consequently, a new sampling of BCG data is captured for an instantaneous assessment of his state knowing that any measurement having values ​​close to the minimum threshold can also trigger a precautionary alarm. The effectiveness of the audible, mechanical, luminous or visual alarms in these cases is continuously evaluated in relation to the physical and physiological reactions of the driver. In all cases, if the driver is vigilant or has regained his vigilance probably linked to the effectiveness of the alarms, and in certain embodiments, he can stop the alarms at any time.The percentage of drowsiness assigned at any time just after the recording of the reference values ​​provides a decreasing drowsiness curve having a value HRv determined from the first deflection of the surface function of drowsiness Sn described in [Math 3] and a minimum value HRS which can be expressed by HRS = a.HRO where a is a coefficient for estimating the descent on the sinusoidal curve of drowsiness, over a period varying between tv and h. The percentage Sn of drowsiness is divided into seven levels according to [Table 1] characterized in that the “very alert” state varies from 0 to 25% and requires the monitoring of BCG data and the analysis of the data received by the optical sensor. When the percentage Sn is between 26 and 55%, corresponding to the “alert” state, a flashing green light signal will be initiated while directing the measurements and analyses of the HR, RR and RDI parameters.The doubtful state of vigilance between 56 and 73%, having the same conditions of processing of optical and physiological data except that the light signal becomes solid yellow. If the percentage Sn is between 74 and 81%, designated by a very doubtful state of vigilance during which the viewing by the optical sensor intensifies and characterized in that a mechanical alarm is triggered at low intensity with a solid orange light signal as well as a visual signal indicated on the display of the human-interface. machine located in the control cabin while analyzing the driver's movements with reference to the coordinates of the RDI0. The drowsiness states corresponding to the percentages 82 up to 88% and from 89 to 95% respectively indicating that the driver is in probable or very probable drowsiness requiring the increase of the intensity of the mechanical signal accompanied by a red light signal and a visual display, characterized in that all these events are recorded and time stamped. The designated critical state, "Sleep State" corresponding to a percentage varying between 96 and 100%, characterized in that the audible and mechanical alarm signals are intensified to the maximum to ensure the awakening of the driver who can be identified by his physical and physiological reactions captured by the optical sensor and the millimeter radar.At any time, the driver can stop the alarms by manually pressing a button dedicated to this function installed in the device or remotely, proving that vigilance has resumed. However, all these events are recorded in the secure memory of the device and can be sent by wireless communication or retrieved locally by an authorized person.

[0015] [Math.3]

[0016] Where tv corresponds to the last time stamp of vigilance and h the first temporary value indicating the greatest probability of the presence of drowsiness [Tables 1] Vigilance rate: Etet Acb.oa j- O, U g 1 ■ga VI -fs S & ON Details 1 ¢-25¾ Very vigilant x XX Measurements and analysisKR, RR and RDI and signal limîiins'jx veit solid- - 26- 55% Vtgiî&it XXX Measurements and analysis HR, RR and RDI and signal lüiusnê'ax green flashDGtaKt JQ- State of vigilance drxiteüx X x X Measurements and analysis HR. RR and RDI and signal luimneax "a solid 4 31¾ First of vigilance very doubtful XKX Measurements and analysis HR.' RR and RDI and signal with a low Kiêcaciqua aisrcis and cu / tn: flashing alert message appearing on a display of •nSerfaoe hrtDime-oiscmiïe located in the cab of coiinôt; y 82- 58¾ Probable state of ^æniimieil XXXXX Measurements and analysis HR, RR and RDI and signal kiminsux red with a taacsmque d medium anm that an alert message ch gaoteat appearing on a display of Hierfaas hccume-madiine located in the cab of cuntreie. y State of ^æniimieil very probable.XXX Measurements and a:ia :yse HR, RR and RDI and red light signal with a high intensity mechanical alarm and a flashing alert message appearing on a hommï-aiachme display located in the cabme de cantrcye. . The onset of human sleep can be defined by 5 cycles of limited duration. Each cycle passes through two stages of light and slow sleep characterized by a decrease in the frequency rates of physiological signals or BCG, particularly the heartbeat HR and the respiratory cycles RR. In particular, the shape of the cardiac response can be considered as a cosine having an angle varying from 0 to ir with an amplitude of AHRvs / 2=(HRv-HRs) / 2 in the first moments of the first cycle. The primary action in the case of monitoring the vigilance of a driver is to detect the drop in HR level in a few seconds of the light sleep phase before moving to the slow sleep phase. [Tables 2] shows a method that can be used to monitor the evolution of vital signals HR as a function of time. The millimeter radar captures measurements following a time increment dt in a discrete manner by a cursor i (i=0, 1, 2...) such that the instant t=i*dt. The evolution of the heartbeats is specified according to a method of processing the signals received by the millimeter radar by means of a fast Fourier transform (FFT) in order to determine HRj for each beat j (j=l, 2, 3...). The evolution of an average HR comprising 10 successive beats is calculated. Then, an implicit derivation of the first order, or of the second if necessary, allows to determine the sign and the value of the slope while eliminating the aliases of the chain of measurements. [Table 2], [Math 4], [Math 5], and [Math 6], show the implicit steps of measurements of the HRj and the associated slopes. The same method can be used to follow the evolution of RR, by considering 3 consecutive breaths allowing the evaluation of the variations of the respiratory cycles in conjunction with the instantaneous verification of the vigilance of a driver.

[0017] [Math.4] HR k (bpm)

[0018] [Math.5] / dHR \ _ HRk-HRk.t V 9t 4“ Tt-Tk.,

[0019] [Math.6] / 3HR \ _ ^(^=8-9^^^=^10^¾) [Tables 2] l 0 î 2 3 .. i-1 ii~l (y di 2dï .. idî '...............y............'..............y............. <...............• '.................Y.................' W T-I? ........... T, ........... SFI S^(bpmf HR; HR s........................HR; ........... HR^bpm) - ~SjS; HR,: flh\,(hym) - .....~ / ¾.-- yêHR y • 2““modet —-—) f——'•"■*_______—; —-------i V^rl ' ''h' • 34”'mode, - —----~~------?RR77------- Where bpm stands for beats per minute.

[0020] Below is a brief description of the figures:

[0021] [Fig-1] Device (1) for monitoring the driver's state of vigilance without contact.

[0022] [Fig.2a] Driving and control cabin of a means of transport indicating the location, explanatory but not exclusive, of a device (1) for monitoring the vigilance of a driver (10) without contact equipped with an optical sensor (2) and a millimeter radar (3) located in the same place in front of a driver (10) following the fields of vision of the optical sensor (2) and the millimeter radar (3).

[0023] [Fig.2b] Driving and control cabin of a means of transport indicating the location, explanatory but not exclusive, of a device (1) for monitoring vigilance without contact equipped with an optical sensor (2) separate from a millimeter radar (3) in front of a driver (10) following the fields of vision of the optical sensor (2) and the millimeter radar (3).

[0024] [Fig.2c] Reference of the coordinates of the position of the optical sensor (3) and the millimeter radar (2) relative to the center of the steering wheel or the control lever of the means of transport.

[0025] [Fig.3] Location of the regions of interest of references RDI0(13), (14) and the references of the axes of symmetry (16), (17) allowing the location of the core (15) of the conductor.

[0026] [Fig.4] Location of the new regions of interest RDI(20), (21) in relation to the reference regions of interest (13), (14) and the new coordinates of the axes (19), (22) allowing the determination of the new coordinates of the core (18) of the conductor.

[0027] [Fig.5] Estimation of the movements and action of a driver (10) by analysis of the changes in the coordinates of the regions of interest in relation to a normal driving position allowing the detection of the state of vigilance and any other action restricting driving safety such as telephone conversation (402) or swallowing drinks (403), etc.

[0028] [Fig.6] Evolution of the HR heartbeats allowing to detect the passage to several levels of drowsiness (300), (301) and the temporal evolution of the derivative (302).

[0029] [Fig.7] Evolution of the HR heartbeats allowing to detect the passage from a certain level of drowsiness to the resumption of the state of vigilance (303), (304) and the evolution of the derivative (305) and (306).

[0030] [Fig.8] Drowsiness curve indicating the 7 levels of alertness.

[0031] [Fig.9] Flowchart describing the major steps for the driver alertness monitoring process.

[0032] [Fig. 10] Detailed flowchart describing the contactless driver vigilance monitoring algorithm.

[0033] [Fig. 11] Detailed flowchart describing the methodology of physiological or ballistocardiographic data and the reference and instantaneous regions of interest.

[0034] Below is a detailed description of the figures:

[0035] [Fig.l] Device (1) for monitoring the state of vigilance of the driver (10) without contact comprising an optical sensor (2) oriented towards the front face of the driver (10) to characterize the regions of interest (11), for example and to digitize the reference coordinates making it possible to monitor the movements of the driver (10) including those of the position of the heart (12). The device (1) also comprises a millimeter radar (3) making it possible to measure the physiological or BCG data of the driver (10). These data are sent to a microprocessor (9) for digital processing in order to determine the physiological reference values ​​of the driver (10) and his state of vigilance.A piezoelectric sensor (5) for measuring the ambient vibrations of the means of transport and the sustained or occasional vibration of the road, characterized in that the spectral densities and frequencies are analyzed by the microprocessor program (9) to refine the process of filtering the data from the millimeter radar (3). A communication (6), if available, makes it possible to detect the starting speed of the means of transport, otherwise this can be deduced from a piezoelectric sensor (5). An alarm management circuit (8) for adjusting the alert levels to the driver according to a vigilance scale, characterized in that light, mechanical, sound and visual alarms are progressively deployed according to the state of drowsiness of a driver (10) accompanied by recordings of all the events in a . memory (25).

[0036] [Fig.2a] Device (1) for monitoring the state of vigilance of the driver (10) without contact characterized in that the optical sensor (2) and the millimeter radar (3) installed in the same place in front of the driver (10), on the control and driving console of a means of transport, following a field of vision optimized in relation to the chest (12) and the regions of interest, (11) for example, making it possible to measure the physiological or BCG data of the driver (10) and to estimate his movements and actions.

[0037] [Fig.2b] Device (1) for monitoring the state of vigilance of the driver (10) without contact characterized in that the optical sensor (2) and the millimeter radar (3) installed on either side of a steering wheel or a joystick for controlling the direction of a means of transport in front of the driver (10), following a field of vision optimized in relation to the chest (12) and the regions of interest, (11) for example, making it possible to measure the physiological or BCG data of the driver (10) and to estimate his movements and actions.

[0038] [Fig.2c] Reference frame of the coordinates of the position of the optical sensor (3) and the millimeter radar (2) relative to the center of the steering wheel or the control lever of the means of transport. This reference frame (23) designated Rv: (O, i, j, k) with center O, in the middle of the steering wheel or the control lever of a means of transport, and with a direct orthonormal base (i, j, k) characterized in that: the axes (O, i) and (O, j) defining the horizontal plane, and the axes (O, i) and (O, k) defining the plane of symmetry of the steering wheel or the control lever, in the ideal conditions of rectilinear movement of the means of transport.

[0039] [Fig.3] An optical sensor (2) allowing the detection of an anatomical template of a driver (10) by projection of a thermal map which facilitates the identification and memorization of the regions of interest of references RDI0 (13), (14) and of the horizontal (17) and vertical (16) axes relative to the fixed reference (23) located in the middle of a steering wheel or a means of control and driving of a means of transport in order to estimate the position of the heart (15).

[0040] [Fig.4] An optical sensor (2) having the capacity to detect the anatomical template of a driver (10) by projection of a thermal map which makes it possible to identify the instantaneous regions of interest RDI(20), (21) and the inclinations of the axes (22), (19) relative to the fixed reference point (23) located in the middle of a steering wheel or control and driving lever of a means of transport in order to estimate the new position of the heart (18).

[0041] [Fig.5] Thermal mapping of a driver (10) in a driving position considered normal allowing to identify several constellations of specific reference points (401) such as the shoulders, the mouth, etc., which are used to monitor the movements and actions of the driver (10) in order to appreciate certain aspects of the vigilance requiring driving safety. As a result, a driver (10) having a new constellation of points (400) indicating, for example, that the driver (10) is carrying an object and raising his hand while observing it. Similarly, the driver (10) is carrying on a telephone conversation while driving according to the new constellation of points (402) or drinking according to the constellation (403).

[0042] [Fig.6] A long recording of heartbeats before and during sleep containing the transition from the highly alert state to the sleep state (301). The drowsiness curve (301), in enlarged mode, clearly discerns the decrease in the driver's alertness (10) characterized in that the time derivative of (301) having a negative slope exceeding predefined thresholds (302).

[0043] [Fig.7] A long recording of the heartbeats before and during awakening (303) containing the transition from the drowsy state to the very alert state (304) characterized in that the heartbeat HR increases with a positive slope exceeding a threshold (306), then the transition from the very alert state to the drowsy state (305) characterized in that the heartbeat decreases with a slope exceeding a negative threshold (306). To minimize false alarms due to fluctuations in the HR measurement level, the first-order derivative with respect to time is performed on an average value every 10 successive HR values ​​measured as defined by [Table 2], [Math 4], [Math 5] and [Math 6]. [Fig.6] and [Fig.7] show erroneous overshoots (308) and (309) at the predefined thresholds for the cases of direct point-by-point derivatives with respect to the measured HR time, called mode 1 derivative, whereas the derivatives of the average values, called mode 2 or 3 derivatives, lead to more credible detections of the drowsiness phases (302) and (307) when passing from the state of vigilance to the state of sleep, or of the waking phases (306) when passing from a state of sleep to a state of vigilance.

[0044] [Fig.8] Classification of alertness state on seven levels according to the percentage of sleepiness %Sn defined by [Math 3] characterized in that a %Sn varying between 0 -25% indicates a “very alert” state, a %Sn varying between 26 - 65% indicates a “vigilant” state, a %Sn varying between 66 - 73% indicates a “doubtful alertness” state, a %Sn varying between 74-81% indicates a “very doubtful alertness” state, a %Sn varying between 82-88% indicates a “probable sleep” state, a %Sn varying between 89-95% indicates a “very probable sleep” state and a %Sn varying between 96 -100% indicates a “sleep” state. To ensure the effectiveness of the device (1) for monitoring the non-contact vigilance of a driver (10) of a means of transport, an appropriate action is carried out for each level of vigilance, as detailed in [Tablesl].

[0045] [Fig.9] Operating method of the vigilance monitoring device (1) contactless detection of a driver (10) of a means of transport. The first step consists of an initiation (SI) characterized in that the device (1) verifies the existence of the driver (10) and the movement of the means of transport. The second step (S2) consists of measuring, by the optical sensor (2), millimeter radar (3), and the piezoelectric sensor (5) the RDI, HR and RR values ​​followed by an identification and storage in the memory (25) of the reference values ​​RDI0, HR0 and RR0. Then, a step (S3) of measurements of the instantaneous RDI, HR and RR will take place, sending the data to the microprocessor (9) to analyze it in a step (S4). In the case of the presence of a state of vigilance, the method returns to step (S3) by continuing the measurements, otherwise, in the case of the presence of suspicion of non-vigilance, the method moves to a step of triggering alarms and countermeasures (S5) by monitoring the physical and physiological reactions of the driver (10).

[0046] [Fig. 10] Contactless monitoring of the vigilance of a driver (10) of a means of transport characterized by an initiation procedure (100) of step (SI) followed by a detection of the presence of a driver (101) then a measurement of the speed (102) of a means of transport, provided by a computer (24) or a piezoelectric sensor (5). If the speed is greater than zero, step (S2) is triggered to identify and memorize the reference values ​​RDI0, HR0 and RR0, otherwise a return to (100) of step (SI). Following the start (103) of step (S2), a frame from the SAS direction sensor is received (104), if existing then the measurements (105) of the regions of interest RDI and the BCG signals are triggered.Then, by continuing the measurements (105) in a successive manner of the HR, RR, and RDI, an identification and storage (107) in the memory (25) of the reference values ​​HR0, RR0 and RDI0 are carried out as soon as these values ​​remain stable in a predefined interval during a period of Ats (106). The stability of the HR and RR values ​​is necessary to identify the reference values ​​HR0, RR0, in order to avoid the effect of external factors on the vital signals, for example a physical effort before starting driving which can cause an increase in the heartbeat and the respiratory cycle rate of the driver (10). Similarly, the driver (10) can move in a random manner at the start, causing a shift in taking reference values ​​of the regions of interest RDIq compared to an ideal driving position.In an ideal driving condition, the driver's symmetry plane (10), which can be identified by analyzing the correlations between the coordinates of the regions of interest RDI, is parallel, i.e. identical to the symmetry plane of the steering wheel or control and navigation joystick of a means of transport in the case of rectilinear driving. Subsequent to the identification and storage of the reference values ​​HR0, RRo, and RDI0, a measurement phase (108) and (109) of the instantaneous HR, RR and RDI is triggered. The values ​​of vital signals HR and RR and its first-order time derivatives are compared to . limit thresholds (111) in order to detect the level of vigilance of the driver (10). If the values ​​of HR, RR, and its derivatives with respect to time do not exceed predefined thresholds, the driver (10) is considered in a state of vigilance (114). In the opposite case, if the values ​​of HR, RR or its derivatives with respect to time exceed predefined thresholds, in particular slopes having a negative trend, a phase of analysis of the coordinates (112) and (113) of the regions of interest RDI is triggered to evaluate the movement of the driver (10). A habitual movement shows that the driver is in a state of vigilance (114), while the absence of movement indicates an area of ​​doubt of vigilance (115) and a probable or doubtful situation of drowsiness such as the sudden inclination of the head (116), for example laterocolis, re-trocolis, or anticolis, indicates the increased risk of non-vigilance (117) of the driver (10) which requires the activation of alarms (118).Depending on the physical and physiological responses (119) of the driver (10) to the alarms (118), the intensity and periodicity of the latter (118) are progressively increased. In certain embodiments, the sensor for measuring the angle of rotation of the steering wheel of the means of transport, hereinafter called the steering sensor (7) or SAS if available, can also contemplate and validate the state of drowsiness by examining the voluntary or involuntary deviations as well as the speed of the means of transport. In all cases, the proof of an awakening or the resumption of vigilance of the driver (10) is ideally translated by a voluntary pressing of a stop button (8) of the alarms (118). The device (1) can progressively increase the intensity of these alarms and in the event of non-awakening and in certain embodiments, the device (1) can request the computer (24) to stop the vehicle (120).

[0047] [Fig. 11] Method describing the methodology for measuring physiological data also called BCG and regions of interest. Following the activation (200) of the different sensors, optical (2), millimeter radar (3) and piezoelectric sensor (5), a synchronization (201) between the data received by these sensors is carried out. The movement of the driver (10) prompts an estimation (202) of the regions of interest RDI by using a thermal map obtained by the optical sensor (2) then processed and modeled (204) by the microprocessor (9) in order to have the reference parameters making it possible to monitor the states of vigilance and distraction of the driver (10). At the same time, the piezoelectric sensor (5) carries out measurements (203) corresponding to the ambient vibrations of the vehicle and those of the road characterized in that a modeling (205) is carried out and used by the microprocessor (9).The instantaneous measurements (206) made by the millimeter radar (3) corresponding to the mechanical displacements of the chest (12) are processed by a filtration process (207) followed by a spectral separation of the BCG signal (208). A second filtration (210) is carried out by adapting the HH and RR parameters to the model. mathematical (209) to arrive at the final HR and RR measurements (211).

Claims

Claims

1. Device (1) for contactless monitoring of the vigilance of a driver (10) of a means of transport comprising: • a) An optical sensor (2) oriented towards the front face of the driver (10) allowing, by projection of a thermal map, to determine regions of interest, located on the forehead of the face, cheek, ear, shoulder, arm and or chin, of reference also designated RDI0 (13), (14), (15) and instantaneous also designated RDI (18), (20), (21), and to digitize coordinates allowing to monitor the movements of the driver (10) including those of the position of the heart (12). • b) A millimeter radar (3) allowing the measurement of physiological or ballistocardiographic BCG data of the driver (10) also called heartbeat signals HR and respiration rate RR. • c) A piezoelectric sensor (5) for measuring the ambient vibrations of the means of transport and the sustained or occasional vibration of the road in order to refine the process of filtering the millimeter radar data (3). • d) An alarm management circuit (8) allowing the driver alert levels to be adjusted according to a vigilance scale characterized in that light, mechanical, sound and visual alarms are progressively deployed according to the state of drowsiness of a driver (10). • e) A memory (25) for storing reference values of heartbeat HR0, respiration rate RRo and regions of interest RDI0, characterized in that the coordinates of the regions of interest of references RDI0 distinctive of normal driving are recorded in the memory (25) of the device. • f) A microprocessor (9) for processing the data obtained by the millimeter radar (3), the piezoelectric sensor (5) and the optical sensor (3) characterized in that the BCG data and the thermal mapping are digitized and instantly compared to the reference values in order to monitor the state of vigilance of the driver (10).

2. Method operating the device according to claim 1, making it possible to monitor the contactless vigilance of a driver (10) of a means of transport by following five steps: • The first step (S1) consists of an initiation characterized in that the device (1) verifies the existence of the driver (10) and the movement of the means of transport. • The second step (S2) consists of measuring and identifying reference values RDI0, HR0 and RR0 and storing them in the memory (25). • The third step (S3) consists of instantly measuring the values RDI, HR and RR. • The fourth step (S4) consists of analyzing the instantaneous values of RDI, HR and RR and thus detecting the state of drowsiness and the vigilance of a driver (10): in the case of the presence of a state of vigilance, the process returns to step (S3), whereas, in the case of the presence of suspicion of non-vigilance, the process moves on to step (S5).• The fifth step (S5) consists of triggering the alarm and countermeasure by monitoring the physical and physiological reactions of the driver (10).

3. Method according to claim 2, characterized in that the regions of interest RDI, using the thermal mapping of a driver (10), makes it possible to identify several constellations of specific reference points (401) to be used to monitor movements and actions and to estimate the states of vigilance and distraction of the driver (10).

4. Method according to claim 2, characterized in that when the plots of the HR, RR values or its derivatives with respect to time exceed predefined thresholds, the device triggers a phase of analysis of the coordinates (112) and (113) of the RDI regions of interest to evaluate the movement and posture of a driver (10).

5. Method according to claim 2 and claim 4, for validating the state of vigilance of a driver (10) characterized in that a vigilant state is validated by verifying the existence of habitual driving movements and by contrast to this state, is validated by verifying the existence of abnormal movements and posture

6. assumed static state of the driver (10) leading to a probable or doubtful situation of drowsiness such as a sudden tilt (116) of the head of the driver (10), for example laterocolis, retrocolis, or anticolis. Method according to claim 2, making it possible to evaluate the state of drowsiness of a driver (10) on a scale of seven levels of vigilance characterized in that the drowsiness function %Sn varying between 0-25% designating a “very vigilant” state accompanied by a solid green light signal, between 26-65% indicating a “vigilant” state followed by a flashing green light signal, between 66 - 73% designating a “doubtful vigilance” state accompanied by a yellow alarm, between 74-81% indicating a “very doubtful vigilance” state followed by solid orange light alarms as well as a low intensity mechanical signal and a flashing visual message appearing on a display,between 82-88% indicating a state of "probable sleep" followed by red light alarms as well as a medium intensity mechanical signal and a flashing visual message appearing on a display, between 89-95% indicating a state of "very probable sleep" followed by red light alarms as well as a high intensity mechanical signal and a flashing visual message appearing on a display and between 96 - 100% indicating an imminent "sleep" state characterized by red light alarms as well as a high intensity mechanical signal and a flashing visual message appearing on a display and a signal, sound.