Method and device for monitoring vehicle driver behavior
A curvature-based loss function minimization technique in driver behavior classification models enhances confidence calibration, enabling reliable driver behavior classification and actuator activation, addressing the challenges of existing systems and improving road safety.
Patent Information
- Application Number
- FR2023005226
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing driver behavior monitoring systems face challenges in providing calibrated degrees of confidence in driver behavior classifications, particularly on low-power devices, and require manual calibration or additional computational resources, which is impractical for widespread vehicle deployment.
A driver behavior classification model trained using a gradient descent method with a curvature-based loss function minimization technique to improve confidence calibration, allowing reliable classifications even with disturbed input images, and enabling deployment on low-power devices.
The method enhances road safety by accurately classifying driver behaviors and activating vehicle actuators based on calibrated confidence levels, improving reliability and reducing the need for additional computational resources.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: Method and device for monitoring vehicle driver behavior Technical field
[0001] The present invention relates to the detection of vehicle driver behavior. In particular, the present invention relates to a method and a device for monitoring the behavior of a vehicle driver, in particular an autonomous vehicle. Technological background
[0002] Contemporary vehicles incorporate an increasing number of on-board systems that contribute to the control of the vehicle by assisting the driver and / or the comfort of the driver and passengers of the vehicle. Some of these systems are known as AD AS systems (from the English “Advanced Driver-Assistance System” or in French “Advanced Driving Assistance System”).
[0003] The vehicle driver behavior monitoring system also known as vehicle driver attention monitoring system, driver attention monitor or driver fatigue warning system, abbreviated as DMS (from the English "Driver Monitoring System") is one of these AD AS systems.
[0004] A DMS system comprises one or more cameras positioned in the front of the vehicle passenger compartment so as to capture images / videos of the face of the vehicle driver. One (or more) computer receives, analyzes these captured images / videos and determines output values representative of the level of attention (or behavior) of a vehicle driver, i.e. the levels of drowsiness, fatigue and / or distraction of a vehicle driver. Indicators and / or graphic content are displayed on screens according to these output values to indicate to the vehicle driver his level of attention and possibly advise him to take a break.Among these witnesses, we can cite those which indicate the activation behavior of the DMS system which is either activated or deactivated or those which indicate whether the activated DMS system is operational or not, that is to say if the camera(s) is(are) not obstructed, if the DMS system computers provide the expected output values and if the driver's face is visible (the driver's eyes and mouth are visible). Among the graphic contents, we can cite those which are representative of information messages and which are displayed to explain the reasons for a malfunction of the DMS system (faulty camera, driver wearing glasses or a mask, etc.).
[0005] Today's DMS systems also help determine who is responsible in the event of an accident or other problem, and ensure compliance with current regulations requiring a human driver to be in control of the vehicle at all times.
[0006] Certain current DMS systems which use neural networks to classify driver behavior must provide precise indications as well as so-called calibrated degrees of confidence, i.e. degrees of confidence which guarantee that the driver is indeed in the behaviors which are represented by the information provided at the output of these neural networks.
[0007] A confidence level from a neural network refers to a level of certainty that the neural network has in its classification results. When a driver behavior classification is implemented by a neural network in the form of solving a multi-class classification problem, each class identifier at the output of the neural network corresponds to a particular behavior of a driver. A confidence level is also present at the output of the neural network. This confidence level is generally represented in the form of a probability score that the behavior represented by the class identifier is an actual behavior of the driver.
[0008] A calibrated confidence level then ensures that the probabilities corresponding to driver behaviors (classes) correspond well to actual frequencies of driver behavior classifications. This is crucial for real-world applications, where making incorrect classifications with a high confidence level can have serious consequences. By having a calibrated confidence level, a driver behavior classification model implemented by a neural network can be made more reliable and its classifications can be more interpretable.
[0009] Neural networks that could be used for implementing driver behavior classification models are trained with learning algorithms based on gradient descent methods. This type of trained neural networks tend to be overconfident in their classifications. Confidence properties can be assessed by various measures, such as the expected calibration error (Naeini, M.P., G. Cooper and M. Hauskrecht (2015). “Obtaining well calibrated probabilities using Bayesian binning”. In Proceedings of the Conference on Artificial Intelligence, AAAI 2015).
[0010] While there are possibilities to address overconfidence in classifications of such trained neural networks, these possibilities require either manual post-training inference (e.g., with temperature scaling) or additional computational and memory requirements at the neural network deployment stage (e.g., with Bayesian learning / overall).
[0011] Obtaining a driver behavior classification model that provides driver behavior classifications associated with calibrated confidence levels is one of the problems solved by the present invention.
[0012] This includes providing reliable classifications even in the event of disturbance of the input images (change in brightness, fogging of the lens, degradation of the image, etc.).
[0013] At the same time, the driver behavior classification model must be run on low-power devices with limited computing capabilities on a fleet of millions of vehicles, making manual calibration unrealistic.
[0014] Another problem is to use such a driver behavior classification model to determine actions for managing situations that could prove dangerous depending on the behavior of a driver identified by said driver behavior classification model.
[0015] Summary of the present invention
[0016] An object of the present invention is to solve at least one of the problems of the technological background described above.
[0017] Another object of the present invention is to improve road safety.
[0018] According to a first aspect, the present invention relates to a method for monitoring the behavior of a driver of a vehicle implemented by at least one processor, said method comprising the following steps: - obtaining at least one image of a driver's face from an on-board imaging device of the vehicle; - obtaining a first data item and a second data item by presenting said at least one image obtained as input to a driver behavior classification model trained during a learning phase, said first data item being representative of a driver's behavior and said second data item being representative of a degree of confidence that the driver's behavior corresponds to the behavior represented by the first data item; - selection of at least one actuator of the vehicle based on the first and second data obtained and a third data representative of contextual information relating to the vehicle; - activation of said at least one selected actuator.
[0019] The method makes it possible to classify the behavior of a driver of a vehicle and to activate at least one actuator of the vehicle according to the degree of confidence of this classification and this with the aim of managing dangerous road situations which may occur due to the behavior of the vehicle driver. The method therefore improves road safety.
[0020] According to a variant, the training of the driver behavior classification model determines parameters of the driver behavior classification model by minimizing a loss function using a gradient descent method, said loss function depending on a representative coefficient of curvature at the minima level of the loss function.
[0021] This variant is advantageous because taking into account the information representative of curvature at the level of minima of the loss function which is minimized during the training of the driver behavior classification model makes it possible to improve the calibration of the degrees of confidence at the output of said model.
[0022] This variant is also advantageous because it does not require additional computational costs when training said model or when obtaining a classification of a driver's behavior from said trained model, thus allowing possible deployment on current vehicle devices.
[0023] According to a variant, said coefficient is an amplitude of at least one eigenvalue of a Hessian matrix.
[0024] According to a variant, at least one interval of values being defined for each value of the first data item and each interval of values being associated with at least one actuator of the vehicle, the selection of at least one actuator depends on the value of the first data item obtained, on the membership of the value of the second data item obtained in one of said at least one interval of values and on the third data item.
[0025] According to a variant, the third data is obtained by odometry.
[0026] According to one variant, the driver behavior classification model is trained from a set of driver face images and vehicle driver behaviors.
[0027] According to a second aspect, the present invention relates to a device for monitoring the behavior of a driver of a vehicle, the device comprising a memory associated with a processor configured for implementing the steps of the method according to the first aspect of the present invention.
[0028] According to a third aspect, the present invention relates to a vehicle, for example of the automobile type, comprising a device as described above according to the second aspect of the present invention.
[0029] According to a fourth aspect, the present invention relates to a computer program which comprises instructions adapted for executing the steps of the method according to the first aspect of the present invention, in particular when the computer program is executed by at least one processor.
[0030] Such a computer program may use any programming language, and be in the form of source code, object code, or int code. an intermediate between source code and object code, such as in a partially compiled form, or in any other desirable form.
[0031] According to a fifth aspect, the present invention relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the method according to the first aspect of the present invention.
[0032] On the one hand, the recording medium may be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a ROM memory, a CD-ROM or a microelectronic circuit type ROM memory, or a magnetic recording means or a hard disk.
[0033] Furthermore, this recording medium may also be a transmissible medium such as an electrical or optical signal, such a signal being able to be conveyed via an electrical or optical cable, by conventional or hertzian radio or by self-directed laser beam or by other means. The computer program according to the present invention may in particular be downloaded from an Internet-type network.
[0034] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to perform or to be used in performing the method in question. Brief description of the figures
[0035] Other characteristics and advantages of the present invention will emerge from the description of the particular and non-limiting exemplary embodiments of the present invention below, with reference to the appended figures 1 to 5, in which:
[0036] [Fig-1] schematically illustrates a vehicle 10, according to an exemplary embodiment by particular and non-limiting aspect of the present invention.
[0037] [Fig.2] schematically illustrates curves which represent the evolution of a loss function obtained during the learning phase and of a loss function obtained during an inference phase as a function of internal parameter values of the driver behavior classification model according to a particular and non-limiting exemplary embodiment of the present invention;
[0038] [Fig.3] schematically illustrates intervals of second data for two first data according to a particular and non-limiting example of the present invention;
[0039] [Fig.4] schematically illustrates a device configured to monitor the behavior of a vehicle driver, according to a particular and non-limiting exemplary embodiment of the present invention;
[0040] [Fig.5] illustrates a flowchart of the different stages of a monitoring process of the behavior of a driver of a vehicle, according to a particular and non-limiting exemplary embodiment of the present invention.
[0041] Description of the examples of embodiment
[0042] A method and a device for monitoring the behavior of a driver of a vehicle will now be described in the following with joint reference to Figures 1 to 5. The same elements are identified with the same reference signs throughout the description which follows.
[0043] [Fig-1] schematically illustrates a vehicle 10, according to an exemplary embodiment by particular and non-limiting aspect of the present invention.
[0044] [Fig.l] shows a view of the interior of a vehicle 10, in particular of the front part of the passenger compartment of the vehicle 10 including in particular the steering wheel, the dashboard, the driver's seat (front left seat according to the example of [Fig.l]), the front passenger seat (front right seat according to the example of [Fig.l]) and the windshield of the vehicle 10.
[0045] The vehicle 10 corresponds for example to a vehicle with a thermal engine, with electric motor(s) or even a hybrid vehicle with a thermal engine and one or more electric motors. The vehicle 10 thus corresponds for example to a land vehicle, for example an automobile, a truck, a bus.
[0046] The vehicle 10 advantageously carries an imaging device which provides a set of images of the driver's face to a device 2 on board the vehicle 10 which is configured to monitor the behavior of the driver of the vehicle 10.
[0047] The imaging device comprises a camera 10 associated with an optical system for acquiring images and processing means such as a computer and storage means such as non-volatile memories of the EEPROM or flash type for storing the acquired images.
[0048] The lens of the camera 101 is directed towards the interior of the vehicle 10 for the acquisition of the set of images of the face of the driver of the vehicle 10 seated on the driver's seat. This set of images is provided to the calculation means of the imaging device.
[0049] The camera 101 is for example arranged on or integrated into the dashboard of the vehicle 10, for example at the steering wheel. The shooting point of the camera 101 is directed towards the driver's seat to produce images of the face of the driver of the vehicle 10.
[0050] The camera 101 corresponds for example to a monocular optical camera with a CCD sensor (from the English Char-Coupled Device, in French charge transfer device) or with a CMOS sensor (from the English Complementary Metal-Oxide Semi-conductor, in French complementary metal oxide semiconductor).
[0051] The imaging device provides high-resolution images for processing (e.g. example >1 megabits / pixel).
[0052] The passenger compartment of the vehicle 10 further comprises a display screen 102, for example arranged at the central level of the dashboard.
[0053] The screen 102 is for example configured for the display of graphic contents of an HMI under the control of a computer of the infotainment system, called IVI computer (from the English “In-Vehicle Infotainment” or in French “Infodivertissement embarké”) of the vehicle 10.
[0054] The screen 102 is provided with a touch interface allowing a person on board the vehicle to interact with an HMI of the DMS system displayed on this screen 102.
[0055] An HMI such as HMI 2 corresponds for example to a stack of layers (from the English “layer”), each layer corresponding to the display of a set of determined graphic contents, the superposition of the layers making it possible to obtain the rendering of the HMI. The graphic contents to be displayed are advantageously included in a single layer, for example the upper layer of the stack. By way of example, the stack of layers comprises 5 layers numbered from 1 to 5, the layer comprising graphic contents corresponding to the 5th layer, that is to say the layer superimposed above the other layers.
[0056] The driver's behavior (also called driver state) may correspond to normal behavior, drunk or tired for example.
[0057] The present invention is not limited to a particular type of behavior of the driver of the vehicle and the examples provided are provided only as examples.
[0058] According to the present invention, each driver behavior determined by the process described below is associated with an activation of at least one actuator of the vehicle 10.
[0059] For example, the activation of an actuator of the vehicle 10 may be a warning light that comes on or a window that is displayed on the screen 102. This warning light may correspond to a warning pictogram, for example, when the driver's behavior is considered likely to cause dangerous road situations. An alert message may also be displayed in the window displayed on the screen 102.
[0060] According to another example, the activation of an actuator of the vehicle 10 may be the broadcasting of an audible signal inside the passenger compartment of the vehicle 10.
[0061] According to another example, the activation of an actuator of the vehicle 10 may be the activation of a braking system, and / or taking control of the vehicle to park the vehicle on the side of the road on which the vehicle 10 is traveling or even the activation of hazard lights.
[0062] As can be understood from reading these examples, the actuators of the vehicle 10 are numerous and can be activated as a function of the behavior of a driver of the vehicle 10. However, the present invention is not limited to the activation of the actuators cited here only as examples.
[0063] According to variants of the present invention, several actuators can be activated for the same behavior of the driver of the vehicle 10.
[0064] For example, a warning light or message in a window may be displayed on the screen 102 and an audible signal may be emitted inside the vehicle 10 for driver behavior deemed likely to cause dangerous road conditions.
[0065] The present invention is not limited to any combination of actuators activated for driver behavior but extends to any actuator or combination of actuators of the vehicle 10 that can be activated or deactivated.
[0066] The device 2 of [Fig.4] is configured to implement a process for monitoring the behavior of a driver of the vehicle 10. The operations forming such a process are described below.
[0067] Such a process is advantageously implemented by one or more computers of the vehicle 10.
[0068] In a first operation, at least one image of the face of a driver of the vehicle 10 is obtained from the on-board imaging device of the vehicle 10, for example from the non-volatile memory of the imaging device.
[0069] In a second operation, a first datum and a second datum are obtained by presenting said at least one image obtained as input to a driver behavior classification model trained during a learning phase. Said first datum is representative of a driver behavior and said second datum is representative of a degree of confidence that the driver behavior corresponds to the behavior represented by the first datum.
[0070] According to a variant, each image of the face of the driver of the vehicle 10 obtained undergoes pre-processing which can be implemented by the imaging device so that its resolution and / or its dimensions and / or the format in which this image is represented correspond to the constraints imposed on the images at the input of the trained driver behavior classification model.
[0071] In a third operation, at least one actuator of the vehicle 10 is selected as a function of the first and second data obtained and of a third data representative of contextual information relating to the vehicle.
[0072] According to a variant, the contextual information relating to the vehicle is obtained by odometry.
[0073] In a fourth operation, said at least one selected actuator is activated.
[0074] When the driver behavior classification model is fed with input by an image of the face of a driver of the vehicle 10, it provides an identifier of a class and a degree of confidence that the identified class corresponds to the actual behavior of the driver of vehicle 10.
[0075] To ensure a high degree of confidence, the driver behavior classification model is trained during the learning phase on a set of driver face images exhibiting varied behaviors. Thus, the internal parameters of this driver behavior classification model are determined to minimize a loss function (also called a mismatch function) that measures the difference between the class identifiers provided at the model's output for a set of images presented at its input and the identifiers of the classes that correspond to the driver behaviors that must be provided by the model (sometimes referred to as ground truth).
[0076] When the set of images presented as input to the model is sufficiently large and varied, any driver face image that is statistically close to the driver face images used for training the model will provide an expected driver behavior as output from the model.
[0077] According to a variant, the driver behavior classification model is trained from a set of driver face images and vehicle driver behaviors.
[0078] According to one variant, the driver behavior classification model is implemented by a neural network trained during the learning phase.
[0079] Alternatively, the neural network is a convolution-based model with, for example, a ResNet-type architecture (He, K., X. Zhang, S. Ren, and J. Sun (2016). “Deep Residual Learning for Image Recognition.” In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016. pp. 770-778) with skip connections.
[0080] Such a neural network is typically composed of a sequence of blocks consisting of convolutional layers, nonlinearities, and normalization layers, with inputs propagated for each block. The neural network terminates with a classifier, typically in the form of a multilayer perceptron.
[0081] Alternatively, the driver behavior classification model is implemented by visual transformations as described in the article by Do-sovitskiy, A. et al entitled “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” (In International Conference in Learning Representations, ICLR 2021).
[0082] Alternatively, the neural network is a model based on three-dimensional convolution as described, for example, by Tran, D., et al. entitled “A Closer Look at Spatio-temporal Convolutions for Action Recognition” (Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, CVPR 2018)
[0083] In the case where the learning phase is described to train a convolutional neural network (based on convolution) with a ResNet type architecture, the internal parameters of the neural network are weights and biases of convolutional layers, normalization layers and linear layers. These internal parameters are determined to minimize a loss function which measures the difference between the class identifiers provided at the output of the neural network for a set of images presented at its input and the identifiers of the classes which correspond to the behaviors of the driver which must be provided by the neural network (ground reality).
[0084] According to a variant, the minimization of the loss function is based on a gradient descent method in which the internal parameters of the driver behavior classification model (of the neural network) are updated following the direction of a negative gradient of the loss function, calculated for a set of images.
[0085] Alternatively, the method for minimizing the loss function is that described in the article by Robbins, H. et al. entitled “A Stochastic Approximation Method.” (In The Annals of Mathematical Statistics 22.3, pp. 400-407).
[0086] According to a variant, the learning (training) of the driver behavior classification model determines parameters of the driver behavior classification model by minimizing a loss function using a gradient descent method, said loss function depending on a representative coefficient of curvature at the minima level of the loss function.
[0087] [Fig.2] schematically illustrates curves which represent the evolution of a loss function Fl obtained during the learning phase and of a loss function F2 obtained during an inference phase as a function of internal parameter values of the driver behavior classification model according to a particular and non-limiting exemplary embodiment of the present invention.
[0088] The values of the loss functions (ordinates) are defined as a function of the evolution of the parameters of the driver behavior classification model (abscissas).
[0089] The curves have two local minima, one considered to be flat and the other to be pointed.
[0090] Taking into account the representative coefficient of curvature at the level of minima of the loss function makes it possible to favor the “flat” minima to the detriment of the “pointy” minima of the loss functions, which makes it possible to calibrate the degrees of confidence at the output of the driver behavior classification model.
[0091] According to a variant, the coefficient is an amplitude of an eigenvalue of a Hessian matrix.
[0092] Mathematically, the gradients of the gradient descent method can be written as:
[0093] VL(t?) =VL(# + fe)+«0 + )^
[0094] with h a positive fable value typically between 0.5 and 1, has a value of a weight decay factor between, for example, 0.0001 and 0.001, Y the amplitude of the eigenvalue of a Hessian matrix, for example between 0.001 and 1 and
[0095] Vg(<p, = || VL(ç> ) -VL(0) || J
[0096] e VU» 4 MI2I|vl(UII2
[0097] In the case where the training phase is described in the case where the driver behavior classification model is a convolutional neural network (convolution-based) with a ResNet-like architecture, weight decay and Hessian matrix regularization are applied only for the weight matrices in the dense layers and for the kernel tensors in the convolutional layers, but not for the biases and other vectors, such as the location and scale parameters in the normalization layers. Finally, the gradients are scaled by a constant or programmed learning rate, before being used to update the parameters of the driver behavior classification model.
[0098] According to a variant, the CIFAR10 image set is used to train the convolutional neural network (based on convolution) with a ResNet type architecture.
[0099] The inventor carried out training for different values of the weight decay factor a and different values of the eigenvalue Y of the Hessian matrix.
[0100] Although they all have comparable accuracy, the driver behavior classification models obtained with stronger regularization have a lower calibration error, i.e., a higher confidence calibration. Moreover, this is true when the input images are disturbed, for example with the addition of fog at the camera lens 101 or various other noise sources.
[0101] According to a variant, at least one interval of values being defined for each value of the first data item and each interval of values being associated with at least one actuator of the vehicle, the selection of at least one actuator depends on the value of the first data item obtained, on the membership of the value of the second data item obtained in one of said at least one interval of values and on the third data item.
[0102] According to a variant, the actuator selection as a function of the first and the second data obtained and a third data representative of contextual information relating to the vehicle can be done by manually defining a set of rules which can be defined to meet international road safety laws and / or certification requirements (for example, ISO 26262).
[0103] [Fig.3] schematically illustrates intervals of second data for two first data according to a particular and non-limiting example of the present invention.
[0104] Three intervals of second data (degree of confidence) 31, 32 and 34 are defined for a first value of the first data (driver behavior) identified by a class identifier ID1. If the second data is less than 30% (belongs to the interval 31) then an actuator A1 is activated. If the second data is between 30 and 60% (belongs to the interval 32) then an actuator A2 is activated and if the second data is greater than 60% (belongs to the interval 33) then an actuator A4 is activated.
[0105] Three intervals of second data 34, 35 and 36 are defined for a second value of the first data identified by a class identifier ID2. If the second data is less than 45% (belongs to interval 34) then the actuator A1 is activated. If the second data is between 45 and 85% (belongs to interval 35) then an actuator A3 is activated and if the second data is greater than 85% (belongs to interval 36) then an actuator A5 is activated.
[0106] The thresholds (percentages) are given here only as an example. Each actuator A1 to A5 can correspond to a group of actuators of the vehicle 10.
[0107] The number of thresholds for deciding when to activate actuators and their values are configurable, for example, to minimize false alerts and to minimize the number of drivers who deactivate the system. Activation of these different actuators based on these thresholds can only be triggered safely and be more effective if the confidence levels are reliable and well calibrated.
[0108] [Fig.4] schematically illustrates a device 2 configured to monitor the behavior of a vehicle driver, according to a particular and non-limiting exemplary embodiment of the present invention.
[0109] The device 2 corresponds for example to a device on board the vehicle 10, for example a computer.
[0110] The device 2 is for example configured for the implementation of at least one operation described with regard to figures 1, 2 and 3 and / or steps of the method described with regard to [Fig.5]. Examples of such a device 2 include, but are not limited to, on-board electronic equipment such as an on-board computer of a vehicle, an electronic calculator such as an ECU (“Electronic Control Unit electronics"), a smartphone, a tablet, a laptop. The elements of the device 2, individually or in combination, can be integrated into a single integrated circuit, into several integrated circuits, and / or into discrete components. The device 2 can be produced in the form of electronic circuits or software (or computer) modules or even a combination of electronic circuits and software modules.
[0111] The device 2 comprises one (or more) processor(s) 20 configured to execute instructions for carrying out at least one step of the method and / or for executing the instructions of the software(s) embedded in the device 2. The processor 20 may include integrated memory, an input / output interface, and various circuits known to those skilled in the art. The device 2 further comprises at least one memory 21 corresponding for example to a volatile and / or non-volatile memory and / or comprises a memory storage device which may comprise volatile and / or non-volatile memory, such as EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic or optical disk.
[0112] The computer code of the embedded software(s) comprising the instructions to be loaded and executed by the processor is for example stored in the memory 21.
[0113] According to various particular and non-limiting embodiments, the device 2 is coupled in communication with other similar devices or systems and / or with communication devices, for example a TCU (from the English “Telematic Control Unit” or in French “Telematic Control Unit”), for example via a communication bus or through dedicated input / output ports.
[0114] The driver behavior classification model may be trained by a remote device, such as a server or a series of servers, in communication with the device 2 via a communication network. The internal parameters of the driver behavior classification model may then be transmitted by this remote device to the device 2 which receives them. The device 2 is then able to implement the driver behavior classification model.
[0115] According to a particular and non-limiting exemplary embodiment, the device 2 comprises a block 22 of interface elements for communicating with external devices, for example a remote server or the “cloud”, other nodes of the ad hoc network. The interface elements of the block 22 comprise one or more of the following interfaces: - RF radio frequency interface, for example Wi-Fi® type (according to IEEE 802.11), for example in the 2.4 or 5 GHz frequency bands, or Bluetooth® type (according to IEEE 802.15.1), in the 2.4 GHz frequency band, or Sigfox type using UBN radio technology (from the English Ultra Narrow Band, in French ultra narrow band) narrow), or LoRa in the 868 MHz frequency band, LTE (from the English “Long-Term Evolution” or in French “Evolution à long terme”), LTE-Advanced (or in French LTE-advanced); - USB interface (from the English “Universal Serial Bus” or “Universal Serial Bus” in French); - HDMI interface (from the English “High Definition Multimedia Interface” or “High Definition Multimedia Interface” in French); - LIN interface (from the English “Local Interconnect Network”).
[0116] Data, for example internal parameters of the driver behavior classification model, are for example loaded to the device 2 via the interface of the block 22 using a Wi-Fi® network such as according to IEEE 802.11, an ITS G5 network based on IEEE 802.1 Ip or a mobile network such as a 4G (or 5G) network based on the LTE (Long Term Evolution) standard defined by the 3GPP consortium, in particular an LTE-V2X network.
[0117] According to another particular and non-limiting exemplary embodiment, the device 2 comprises a communication interface 23 which makes it possible to establish communication with other devices (such as other computers of the on-board system) via a communication channel 24. The communication interface 23 corresponds for example to a transmitter configured to transmit and receive information and / or data via the communication channel 24. The communication interface 23 corresponds for example to a wired network of the CAN (from the English “Controller Area Network” or in French “Réseau de contrôles”) type, CAN FD (from the English “Controller Area Network Flexible Data-Rate” or in French “Réseau de contrôles à débit de données flexible”), FlexRay (standardized by the ISO 17458 standard) or Ethernet (standardized by the ISO / IEC 802-3 standard).
[0118] Internal parameters of the driver behavior classification model may be received via the communication interface 23.
[0119] According to a particular and non-limiting exemplary embodiment, the device 2 can provide output signals to one or more external devices, such as a display screen 25, touch-sensitive or not, one or more speakers 26 and / or other peripherals 27 (projection system) via output interfaces 28, 29 and 30 respectively. According to a variant, one or other of the external devices is integrated into the device 2.
[0120] [Fig.5] illustrates a flowchart of the different steps of a method for monitoring the behavior of a driver of a vehicle, according to a particular and non-limiting example of the present invention. The method is for example implemented by a device on board the vehicle 10 or by the device 2 of [Fig.4],
[0121] In a first step 51, at least one image of a face of the driver is obtained from an on-board imaging device of the vehicle.
[0122] In a second step 52, a first datum and a second datum are obtained by presenting said at least one image obtained as input to a driver behavior classification model trained during a learning phase, said first datum being representative of a driver behavior and said second datum being representative of a degree of confidence that the driver behavior corresponds to the behavior represented by the first datum.
[0123] In a third step 53, at least one actuator of the vehicle is selected as a function of the first and second data obtained and of a third data representative of contextual information relating to the vehicle.
[0124] In a fourth step 54, said at least one selected actuator is activated.
[0125] According to a variant, the variants and examples of the operations described in relation to [Fig.l] apply to the process steps of [Fig.5].
[0126] Of course, the present invention is not limited to the exemplary embodiments described above but extends to a method for monitoring the behavior of a vehicle driver which would include secondary steps without thereby departing from the scope of the present invention. The same would apply to a device configured for implementing such a method.
[0127] The present invention also relates to a vehicle, for example an automobile or more generally an autonomous land-based motor vehicle, comprising the device 2 of [Fig.4].
Claims
Claims
1. Method for monitoring a behavior of a driver of a vehicle implemented by at least one processor, said method comprising the following steps: - obtaining (51) at least one image of a face of the driver from an on-board imaging device of the vehicle; - obtaining (52) a first data item and a second data item by presenting said at least one image obtained as input to a driver behavior classification model trained during a learning phase, said first data item being representative of a behavior of the driver and said second data item being representative of a degree of confidence that the behavior of the driver corresponds to the behavior represented by the first data item; - selecting (53) at least one actuator of the vehicle as a function of the first and second data items obtained and a third data item representative of contextual information relating to the vehicle;- activation (54) of said at least one selected actuator;
2. The method of claim 1, wherein training the driver behavior classification model determines parameters of the driver behavior classification model by minimizing a loss function using a gradient descent method, said loss function depending on a representative coefficient of curvature at the minima of the loss function.
3. The method of claim 2, wherein said coefficient is an amplitude of at least one eigenvalue of a Hessian matrix.
4. Method according to one of claims 1 to 3, for which at least one interval of values being defined for each value of the first data item and each interval of values being associated with at least one actuator of the vehicle, the selection of at least one actuator depends on the value of the first data item obtained, on the membership of the value of the second data item obtained in one of said at least one interval of values and on the third data item.
5. Method according to claim 4, for which the third data is obtained by odometry.
6. Method according to one of claims 1 to 5, for which the driver behavior classification model is trained from a set of driver face images and driver behaviors. vehicle drivers.
7. Computer program comprising instructions for implementing the method according to any one of the preceding claims, when these instructions are executed by a processor.
8. A computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the method according to one of claims 1 to 6.
9. Device (2) for monitoring the behavior of a driver of a vehicle comprising a memory (21) associated with at least one processor (20) configured for implementing the steps of the method according to any one of claims 1 to 6.
10. Vehicle (10) comprising the device (2) according to claim 9.
Citation Information
Patent Citations
Distracted driving behavior early warning method based on cab near-infrared camera
CN108764034A
Driver behavior monitoring systems and methods for driver behavior monitoring
US20150186714A1
Cited By
Method and device for controlling an on-board system of a vehicle based on a prediction of the facial emotion of a vehicle driver
FR3172573A1
