System and method for contextualized stimulation of a vehicle driver's attention

Abstract

System and method for contextualized stimulation of a vehicle driver's attention. The invention relates to a contextualized stimulation system (1) for a vehicle driver's attention, capable of: - estimating a level (Nv) of the driver's mind-wandering state, - modeling the driving environment and providing its level of complexity (D), - detecting one or more risks (Rj) related to the driving environment, - providing an overall risk level (C) related to a lack of driver vigilance based on the detected risks and the driver's visual field of view, - alerting the driver to their mind-wandering when the estimated level (Nv) of mind-wandering and the level of complexity (D) are greater than predetermined levels, or to alert the driver to a risk related to a lack of vigilance when the overall risk level (C) is greater than a predetermined risk level. (Figure 1)

Description

Title of the invention: System and method for contextualized stimulation of a vehicle driver's attention

[0001] The present invention relates to the field of the automotive industry, and more specifically concerns a system and a method of contextualized stimulation of the attention of a vehicle driver, that is to say that the stimulation exerted on the driver by a human-machine interface is adapted to a driving context.

[0002] A substantial percentage of traffic accidents are due to states of drowsiness, distraction or inattention, to such an extent that the new European safety rules GSR 2 (for the English "General Safety Regulations" 2), or road safety bodies such as EuroNCAP (for the English "European New Car Assessment Program"), require new vehicles to have systems for detecting and warning of such states.

[0003] In fact, there are many types or causes of distraction or inattention, related to listening to music, a passenger, handling objects in the passenger compartment, consulting a screen, this consultation being related to driving or not, ... These types or causes of distraction can be analyzed by sensors present in the vehicle, which can detect in particular the movements of the driver.

[0004] Another type of distraction, much less easily detectable, is mind-wandering, defined in the scientific literature as a shift in the content of thoughts away from the driving task, towards self-generated thoughts and feelings. Mind-wandering is therefore a form of endogenous or internal distraction stemming from the driver's concerns or thoughts. Its occurrence is facilitated by a monotonous driving environment, a familiar route, or a driving task that does not require sustained attention, thus freeing up attentional resources to think about something else. It is also encouraged by negative emotions, particularly sadness or depression.

[0005] Mind wandering is therefore a type of purely cognitive distraction, not linked to a physical, visual and / or auditory distraction, and therefore cannot be detected solely by observing, for example, the direction of the driver's gaze.

[0006] However, when the driver is in such a state, he sees without looking, reduces his distance from other vehicles, drives faster, reacts more slowly, and has difficulty maintaining a straight trajectory, hence a significant number of accidents related to this type of distraction, which can occupy between one third and two thirds of the driver's driving time.

[0007] It is therefore difficult on the one hand to detect a state of mind wandering in a driver, and on the other hand unproductive to warn the driver too frequently of this state when no imminent risk is present, given that such warnings would no longer have any effect on him and could make the driver insensitive to other more priority alerts.

[0008] The driver is indeed subjected to numerous stimuli related to advanced driver assistance systems, such as navigation instructions or alerts about risks associated with the driving environment. Adding to these alerts those designed to warn the driver that they are in a state of mind-wandering would render all these alerts less effective.

[0009] The present invention aims to remedy at least in part the aforementioned drawbacks by providing a method and a contextualized stimulation system for the attention of a vehicle driver, which makes it possible to prevent accidents by making effective the alerts notifying the driver of a state of mind wandering or of a risk related to the driving environment.

[0010] The present invention aims to remedy, at least in part, the aforementioned drawbacks by providing a contextualized driver attention stimulation system, comprising: - at least one human-machine interface, - a module for estimating the driver's level of mind wandering, - a module for modeling the vehicle's driving environment, capable of providing a level of complexity for the modeled driving environment, - a module for detecting risks related to the modeled driving environment, capable of identifying one or more risks related to the driving environment. - analytical means capable of determining a visual exploration field for the driver, and of providing an overall risk level related to the driver's understanding of the driving environment, based on the risk(s) identified by the detection module and the determined visual exploration field, the contextualized stimulation system being characterized in that it is configured to alert the driver that they are in a high state of mind-wandering, using at least one human-machine interface, when the estimated level of mind-wandering of the driver is greater than a predetermined mind-wandering threshold and when the level of complexity of the driving environment is greater than a predetermined level of complexity, and to alert the driver of a risk related to a lack of vigilance, using at least one human-machine interface,when the overall risk level related to the driver's understanding of the driving environment exceeds a predetermined risk level.

[0011] Thanks to the invention, the driver alert informing them that they are in a high state of mind-wandering will only be triggered when the contextualized stimulation system according to the invention detects that the driver has reached a certain level of mind-wandering and when the driving environment requires a minimum level of attention. The level of mind-wandering is, of course, all the greater the more the driver is lost in their thoughts.

[0012] Furthermore, in one embodiment of the invention, the driver alert regarding their heightened state of mind wandering is not triggered when the conditions for triggering an alert informing the driver of a risk related to a lack of vigilance are met. This prevents the driver from being distracted by an alert informing them that they are in a heightened state of mind wandering, to the detriment of the alert informing them of a risk related to their lack of vigilance, and thus increases the driver's acceptability of the contextualized stimulation system according to the invention. The driver alert regarding a heightened state of mind wandering can only be triggered by the contextualized stimulation system according to the invention when the overall risk level related to the driver's understanding of the driving environment is lower than the predetermined risk level.

[0013] In other words, the contextualized attention stimulation system according to the invention includes means for inhibiting the alert related to a high state of mind-wandering of the driver as soon as an alert of a risk related to a lack of vigilance is triggered by the contextualized stimulation system.

[0014] Furthermore, the contextualized attention stimulation system according to the invention does not send the same type of signal to the at least one human-machine interface, depending on the type of driver alert it triggers. Indeed, the at least one human-machine interface alerts the driver to a high state of mind-wandering differently from the way it alerts the driver to a risk related to a lack of vigilance. Optionally, the alert to a high state of mind-wandering uses a first human-machine interface, distinct from a second human-machine interface used to alert the driver to a risk related to a lack of vigilance.

[0015] The driver's alert of a high state of mind-wandering consists, for example, of a flashing of a diode, which can be more or less rapid or intense, or of the display of an avatar on a screen, which addresses the driver audibly through a loudspeaker.

[0016] The driver's warning of a risk related to a lack of vigilance is more noticeable and may include, for example, a more or less intense beep, vibrations on the steering wheel, a display of a flashing triangle, etc. Of course, the type of interaction for either of these alerts is not limited, however, the alert to the driver of a high state of mind wandering can also use haptic feedback on the steering wheel or an audible signal.

[0017] The contextualized attention stimulation system according to the invention is, for example, part of an advanced vehicle driving assistance system, which is further configured to act on actuators, for example to activate an emergency braking device when an imminent risk such as a pedestrian crossing is detected, while the driver is not aware of this risk.

[0018] Furthermore, in one embodiment of the invention, the analysis means capable of providing an overall risk level related to the driver's understanding of the driving environment are configured to take into account the level of mind wandering estimated by the estimation module. The overall risk level related to the driver's understanding of the driving environment is, for example, first evaluated based on a driving action or driver behavior consistent with the risk(s) detected by the detection module, and then adjusted according to the estimated level of the driver's mind wandering.

[0019] In particular, the analysis means are capable of determining a field of visual exploration according to the driver's gaze, and of adjusting the field of visual exploration according to the estimated level of mind wandering in order to calculate the overall level of risk related to the driver's understanding of the driving environment.

[0020] The visual exploration field is, for example, determined in the form of a cone of vision whose apex originates from the driver's eyes and whose central axis is directed along the driver's gaze. This cone has, for example, an elliptical base, and is less extended vertically than laterally, i.e., towards the sides of the road. The longer side of the ellipse forming the base of the cone of vision is then, for example, decreased or increased by the analysis means, depending on whether the driver's level of mind wandering, estimated by the estimation module, increases or decreases, respectively, relative to a previous estimated level of mind wandering.

[0021] Optionally, when the estimation module provides a reliability level for the estimated level of mind-wandering, calculated, for example, based on the accuracy of sensors used for this estimation, and when this reliability level is below a predetermined reliability threshold, then the analysis means provide an overall risk level related to the driver's understanding of the driving environment, independent of the estimated level of mind-wandering. Furthermore, For example, driver alerts for a high state of mind wandering are inhibited when this level of reliability is below the predetermined reliability threshold.

[0022] The invention also relates to a method for contextualized stimulation of a vehicle driver's attention, using at least one human-machine interface, a module for estimating the driver's level of mind wandering, a module for modeling the vehicle's driving environment, capable of providing a level of complexity for the modeled driving environment, a module for detecting risks related to the modeled driving environment, capable of identifying one or more risks related to the driving environment, and analytical means capable of determining the driver's visual exploration field and providing an overall risk level related to the driver's understanding of the driving environment based on the risk(s) identified by the detection module and the determined visual exploration field. The contextualized stimulation method is characterized in that it comprises the following steps: - Modeling of the driving environment by the modeling module, and provision of the level of complexity of the driving environment modeled by the modeling module, - estimation of the level of mind wandering by the estimation module, - identification of one or more risks related to the driving environment modeled by the detection module, - determination by analytical means of the driver's visual scanning field and provision of an overall risk level related to the driver's understanding of the driving environment, based on the risk(s) identified by the detection module and the visual scanning field determined by the analytical means, - alerting the driver to a high state of mind-wandering, using at least one human-machine interface, when the estimated level of mind-wandering exceeds a predetermined wandering threshold and when the level of complexity of the driving environment exceeds a predetermined complexity level, or - alerting the driver of a risk related to a lack of vigilance, using at least one human-machine interface, when the overall risk level related to the driver's understanding of the driving environment is greater than a predetermined risk level.

[0023] The method for contextualized driver attention stimulation according to the invention has advantages similar to those of the contextualized driver attention stimulation system according to the invention. It is implemented in software and / or hardware form in the contextualized driver attention stimulation system according to the invention.

[0024] In one embodiment of the contextualized stimulation method according to the invention, the driver alert step relating to a high state of mind-wandering is triggered in a manner conditioned on a history of previously carried-out alerts.

[0025] For example, when the estimated level of mind-wandering is above the predetermined mind-wandering threshold and when the level of complexity of the driving environment is below the predetermined level of complexity, the driver is only alerted to a state of high mind-wandering when such an alert had not already been given to the driver in a previous iteration of the contextualized attention stimulation process, for example in the last thirty seconds, or in the previous minute.

[0026] Furthermore, the driver alert for a high level of mind-wandering is adapted to the estimated level of mind-wandering, and / or to a history of previous alerts for a high level of mind-wandering, and / or to the level of complexity of the driving environment. For example, the driver alert for a high level of mind-wandering is more intrusive when the estimated level of mind-wandering is high, and / or when previous alerts for a high level of mind-wandering are close together in time, and / or when the level of complexity of the driving environment is high.

[0027] Furthermore, the contextualized attention stimulation method according to the invention takes into account the driver's reaction to the triggered alerts. For example, the driver is alerted to a high state of mind wandering or a risk related to a lack of vigilance, followed by a step of verifying the driver's reaction to the alert, and a step of intensifying the alert if there is no reaction from the driver, or inhibiting the alert when a driver reaction is confirmed. As an alternative or in addition to this reaction verification step, the alert step is followed by a further step of estimating a level of driver mind wandering, triggering a further step of determining a driver visual exploration field and providing an overall risk level related to the driver's understanding of the driving environment, based in particular on the newly determined visual exploration field, so as to inhibit the alert when the newly estimated mind wandering level is below the predetermined mind wandering threshold or when the newly provided overall risk level is below the predetermined risk level.

[0028] Furthermore, the step of estimating a level of mind-wandering by the estimation module includes on the one hand a sub-step of receiving data representative of physiological measurements of the driver, data representative of driver commands, and at least one data representative of a driving environment, and on the other hand a sub-step of quantifying a state of mind-wandering of the driver as a function of the data received.

[0029] The quantification substep includes, for example, a measurement of distances between a multidimensional vector representing the received data and groups of reference multidimensional vectors associated with levels of mind-wandering, the estimated level of mind-wandering being that of the group of reference multidimensional vectors closest to the multidimensional vector representing the received data. The groups of reference multidimensional vectors are formed, for example, using the K-means algorithm on training data. Alternatively, however, the quantification substep uses, for example, a neural network that has previously undergone a supervised learning phase.

[0030] Each group of multidimensional reference vectors is associated with a single level of thought wandering, so that the set of multidimensional reference vector groups is representative of the different levels of a scale of thought wandering levels.

[0031] Other features and advantages of the invention will become apparent from the following description on the one hand, and from several illustrative and non-limiting examples of embodiments given with reference to the accompanying schematic drawings on the other hand, in which:

[0032] [Fig-1] represents a contextualized attention stimulation system for a vehicle driver according to the invention, in one embodiment of the invention,

[0033] [Fig.2] represents steps of a method according to the invention, for contextualized stimulation of the attention of the driver of the vehicle equipped with the contextualized stimulation system of the [Fig.1], in this embodiment of the invention, and

[0034] [Fig.3] is a table representing alerts of a high state of mind-wandering, made to the driver in the embodiment of Figures 1 and 2, depending in particular on an estimated level of mind-wandering of the driver.

[0035] According to an embodiment represented in [Fig.1], a contextualized driver attention stimulation system 1 of a vehicle according to the invention is embedded in this vehicle, and implements in a software and hardware manner a contextualized driver attention stimulation method 100 (referenced [Fig.2]) according to the invention.

[0036] The contextualized stimulation system 1 is itself part of an advanced driver assistance system (also called ADAS). The contextualized stimulation system 1 here includes a modeling module 10 of a driving environment, which can also be used for other advanced driver assistance functions.

[0037] The driving environment modeling module 10 uses data from the vehicle's various cameras, radars, and LiDARs (Light Detection and Ranging) to identify objects in the vehicle's immediate environment, i.e., within a radius of approximately 200 meters around the vehicle. These objects include the road, road markings and signs, obstacles on the road, other vehicles, pedestrians, cyclists, etc. It thus creates what is called a "world model," that is, a representation of the vehicle's environment, used by the various driver assistance systems to make decisions and provide warnings to the driver.

[0038] The modeling module 10 is implemented in software in a computer, and is capable of delivering, in this embodiment, to other software modules implemented in the same computer or in other computers: - a driving environment for example coded in the form of objects {Ei] with i varying from 1 to the number of objects identified by the modeling module 10; - a selection of objects constituting potential obstacles {Oj} for the vehicle, with j varying from 1 to the number of potential obstacles {Oj}; and - a complexity level D of the driving environment, depending, for example, on the configuration of the road on which the vehicle is traveling, such as its curvature, and the number of potential obstacles {Oj} as identified by the modeling module 10. For example, the complexity level D is minimal when the vehicle is traveling in a straight line without obstacles on the road, and is maximal in an urban environment with heavy traffic and pedestrian crossings. In other words, the complexity level D is higher the more congested the road is or the more potential obstacles it presents, meaning it requires sustained attention from the driver. The complexity level D is, for example, lower the further away the first potential obstacle on the road is from the vehicle.

[0039] The contextualized stimulation system 1 further includes an estimation module 12 of a level of thought wandering of the driver, capable of receiving at least one piece of data from the driving environment {Ei} modeled by the modeling module 10, such as for example a type of road on which the vehicle travels.

[0040] The driver's thought-wandering level estimation module 12 is implemented in a computer and is capable of receiving, in addition to driving environment data, data from a vehicle computer bus, such as a CAN bus (Controller Area Network), and data from physiological sensors, such as an eye tracker and a heart rate monitor. The estimation module 12 is capable of providing a level Nv of driver mind wandering, based on data received from the driving environment, the driver's physiological state, and the commands issued by the driver, these commands being identified by a read from the computer bus.

[0041] For this purpose, the computer implementing the estimation module 12 includes in memory data collected during a learning phase, prior to an estimation of the level of mind wandering of the driver by the estimation module 12. This data was collected over different driving periods, representative of different driving contexts.

[0042] During this learning phase, a driver, who may be the driver of the vehicle or another driver of a different vehicle, travels on various types of roads with different characteristics, such as highways, city streets, straight or winding roads, with heavy or light traffic, pedestrians, or other obstacles, and with varying weather conditions. These different characteristics of the journey—the type of road, its congestion, and the weather conditions—form a driving context. Each driving period is associated with a driving context.

[0043] During each driving period, physiological measurements are taken on the driver using an eye tracker and a heart rate monitor, so as to obtain, for each driving period, a heart rate, and ocular and pupillometric data. The ocular data relate to eye movement and include, for example, the duration of gaze fixation and the number of blinks per minute, while the pupillometric data include, for example, the diameter of the driver's pupil.

[0044] In addition, during each driving period, data are collected from the vehicle's computer bus, such as vehicle speed and engine torque demand. Optionally, an inter-vehicle distance is also evaluated, this distance being representative of the distance between the driver's vehicle and a vehicle traveling in front of the driver's vehicle.

[0045] Each driving period is associated with a level of mind-wandering in the driver, assessed by the driver themselves through questioning during that driving period, for example, via a human-machine interface. In one embodiment of the invention, the state of mind-wandering is quantified on a scale of 1 to 4, with level 1 corresponding to the absence of mind-wandering, level 2 to a slight state of mind-wandering, Level 3 corresponds to a well-established state of mind-wandering, and level 4 to an intense state of mind-wandering.

[0046] For each driving period, the data collected during that driving period are then assembled into a multidimensional reference vector. Each multidimensional reference vector therefore comprises different values ​​representing the driver's state of alertness, for example:

[0047] - physiological data measured on the driver, such as his movements ocular characteristics, heart rate in beats per minute, pupil diameter in millimeters, blink rate in number of blinks per minute, gender, and age,

[0048] - variables originating from the environment in which the driver is traveling, such as the position of surrounding obstacles or other road users, the type of infrastructure, weather conditions, the type of road (mind wandering is much more prevalent on highways than in cities), the distance (in meters) maintained by the driver from the vehicle in front (which tends to decrease during a mind-wandering episode), and the state of the traffic.

[0049] - data from the computer bus, such as vehicle speed in meters per second, the force applied to the brake or accelerator pedal, the engine torque demand in Newton meters, or the steering wheel angle in degrees,

[0050] - the history of previous alert triggers on one or more interfaces man-machine 20 (time since the last alert was triggered, type of alert, etc.) Once the data has been collected, the learning phase involves a classification of the multidimensional vectors recorded during data collection, this classification using a grouping algorithm, here the so-called K-means algorithm, but other grouping algorithms can be used such as HCA (for Hierarchical Ascending Classification) or DBSCAN (for the English "Density-Based Spatial Clustering of Applications with Noise").

[0051] The application of the K-means algorithm makes it possible to group the multidimensional reference vectors from the training data collection into K groups.

[0052] Then a level of mind-wandering is assigned to each group formed by the K-means algorithm, based in particular on the self-assessments made by the driving subject(s) during the driving periods associated with the group's multidimensional reference vectors, and the driving contexts of these driving periods.

[0053] The assignment of a level of mind-wandering to each group therefore depends on the self-assessments of the subject(s) conducting the corresponding vectors multidimensional reference vectors of this group, and the values ​​present in these multidimensional reference vectors.

[0054] Once the groups have been formed and each associated with a level of thought wandering, the learning phase is complete and the data associated with the different groups are recorded in a memory of the computer implementing the estimation module 12.

[0055] As will be explained in detail later in relation to the contextualized stimulation method 100 according to the invention, the estimation module 12 is capable of periodically estimating a level Nv of the driver's mind wandering in real time, representative of the driver's state over a typical driving period. Each driving period lasts, for example, thirty seconds.

[0056] To this end, the estimation module 12 calculates a distance between, on the one hand, a multidimensional vector constructed in the same way as the reference multidimensional vectors during the learning phase, this multidimensional vector being representative of the current driving period, and on the other hand, each of the groups formed during the learning phase. The estimated Nv level of the driver's mind wandering is then equal to the mind wandering level associated with the group whose multidimensional vector is closest.

[0057] In this embodiment of the invention, the estimation module 12 further records a history of the estimations it has performed during driving periods preceding the current driving period. The estimation module 12 is therefore capable of providing an estimated level Nv of the driver's mind wandering during a current driving period, as well as a duration d during which the driver is in this state and, optionally, a level of reliability of the estimated Nv level, depending, for example, on the accuracy or proper functioning of the physiological sensors, and more generally on the reliability of the data received by the estimation module 12.

[0058] The contextualized stimulation system 1 comprises, in addition to the modeling module 10 and the estimation module 12, a risk detection module 14, capable of receiving, in particular, the potential obstacles {Oj} identified by the modeling module 12, and of identifying one or more risks Rj related to the driving environment. Each risk Rj is associated with a potential obstacle Oj, which could be, for example, a pedestrian near the vehicle, a hidden pedestrian crossing, or a chicane.

[0059] More specifically, the risk detection module 14 identifies objective risks associated with the various elements of the driving environment by predicting the vehicle's trajectory. The detected risks relate, for example, to hidden areas or a possible collision with a potential obstacle Oj on or near the road. The hidden areas correspond to areas of interest such as Pedestrian crossings are partially or completely obscured by vehicles or infrastructure. The risk detection module 14 then calculates the probability that a road user, such as a pedestrian or cyclist, will emerge from one of these hidden areas. This probability is significant, for example, for a pedestrian crossing obscured by a bus stopped nearby. Each risk Rj can be associated with a probability Rj, which is higher the greater the risk. This could be, for example, a probability between 0 and 1, or a natural number between 0 and 10.

[0060] The contextualized stimulation system 1 further includes analysis means 16 capable of determining a visual scanning field for the driver and providing an overall risk level C related to the driver's understanding of the driving environment, based on the risk(s) Rj identified by the detection module 14 and the determined visual scanning field of the driver. The analysis means 16 include, for example, a camera fixed to the windshield and a software module that can be integrated into a computer implementing, for example, the detection module 14.

[0061] The analysis means 16 are suitable for receiving, in particular: - the potential obstacles Oj, originating from modeling module 10, - the estimated Nv level of mind wandering, - the identified risk(s) Rj, each including, for example, a corresponding risk probability, and - Representative measures of the driver's visual attention are used to provide the overall risk level C related to the driver's understanding of the driving environment. For this purpose, the driver's visual attention is represented by a cone of vision extending from their eyes, with a central axis aligned with the driver's gaze. This cone of vision has an elliptical base. This cone of vision allows us to determine the driver's central and peripheral vision zones and compare them with potential obstacles Oj corresponding to risks Rj that the driver may consider. The driver's eye or head movements can at least partially confirm or refute this consideration.

[0062] In a particular embodiment of the invention, the size of the driver's cone of vision is adjusted according to the estimated Nv level of the driver's mind wandering. For example, when the estimated Nv level is high, for example equal to 3 or 4 on the 1 to 4 scale used, the peripheral vision area corresponding to lateral portions of the cone of vision is reduced compared to a default peripheral vision area, i.e., without taking into account driver mind wandering, and vice versa. The estimated Nv level of driver mind wandering can alternatively be represented by a real value, for example between 0 and 1, or within another range of values, or a natural number coded on a smaller or larger scale than in this embodiment of the invention, for example on a scale of 1 to 10.

[0063] The overall risk level C related to the driver's understanding of the driving environment, provided by the analysis means 16, depends on the presence of the objects Oj associated with the risks Rj within the driver's cone of vision, possibly adjusted according to the estimated level Nv of the driver's mind wandering, and the probabilities associated with these risks Rj. In the absence of risks Rj, the overall risk level C related to the driver's understanding of the driving environment is minimal. The overall risk level C related to the driver's understanding of the driving environment increases further when a risk Rj is at the edge of or outside the driver's cone of vision and when the probability associated with this risk Rj is high, for example, above a predetermined threshold.

[0064] The contextualized stimulation system 1 finally includes, in this embodiment of the invention, an alert management module 18, capable of receiving the level Nv of driver thought wandering, estimated by the estimation module 12, the overall risk level C related to the driver's understanding of the driving environment, provided by the analysis means 16, and the level of driving complexity D, provided by the modeling module 10.

[0065] The alert management module 18 is connected here to one or more human-machine interfaces 20, capable of generating alerts for the driver. The human-machine interfaces include, for example, a display screen, a loudspeaker, and a haptic device capable of vibrating the driver's steering wheel.

[0066] The steps of the stimulation process are now described in relation to [Fig. 2]. contextualized 100 of the driver's attention of the vehicle, implemented by the contextualized attention stimulation system 1, in this embodiment of the invention.

[0067] The contextualized stimulation method 100 comprises a first step 102 implemented by the modeling module 10, continuously as it receives data from the vehicle's various cameras, radars, and LiDARs. This first step 102 is the modeling of the driving environment {Ei}, the provision of at least one piece of data from this driving environment {Ei} to the estimation module 12, the provision of potential obstacles {Oj} in this driving environment to the risk detection module 14 and the analysis means 16, the determination of the complexity level D of the driving environment {Ei}, and the provision of this complexity level D to the alert management module 18. The complexity level D is, for example, a real value between 0 and 1 or between in another interval, or is a natural number on a scale of 0 to 3, or on a smaller or larger scale.

[0068] This first step 102 is followed by a second step 104 of the contextualized stimulation process 100, implemented by the estimation module 12, which is the estimation of the level Nv of the driver's mind wandering for the current driving period, this estimation step being repeated periodically at each new driving period, fixed for example at thirty seconds.

[0069] This second step 104 comprises a first sub-step 1041 of reception: - Physiological data from physiological sensors, for example a heart rate monitor or a smartwatch worn by the driver, and an eye tracker in the form of, for example, a camera fixed to the vehicle's windshield, - Data from the computer bus, such as, for example, the engine torque demand deduced from the driver's pressure on the accelerator pedal, and the steering wheel angle, - data provided by the modeling module 10, allowing the estimation module 12 to determine a driving context, for example motorway with a clear sky.

[0070] During this first reception substep 1041, the estimation module 12 form with the received data, a current multidimensional vector in the same multidimensional space as the reference multidimensional vectors used during the learning phase to form groups of reference multidimensional vectors.

[0071] Following the first reception substep 1041, the second estimation step 104 of the detection process 100 includes a second, optional normalization substep 1042. In this case, the multidimensional reference vectors of each group of multidimensional reference vectors are also normalized, in the same way, only once during the learning phase.

[0072] This normalization uses, for example, the following formula, which allows all coordinates between 0 and 1 to be normalized:

[0073] x,-min(x) with x; a coordinate of the current multidimensional vector or of the max(X>min(x) multidimensional reference vector considered, corresponding to a data type coded on one dimension, min(x) the minimum value of the coordinates of the multidimensional reference vectors relative to this dimension, and max(x) the maximum value of the coordinates of the multidimensional reference vectors relative to this dimension.

[0074] We can of course choose another normalization method, or another range of values.

[0075] In the following, the current and reference multidimensional vectors considered are normalized, that is to say, we work on coordinates between 0 and 1.

[0076] Then, in a third substep 1043 of the second estimation step 104, the driver's mind-wandering state is quantified. To do this, the estimation module 12 calculates the distances between the current multidimensional vector and each of the reference multidimensional vector groups from the learning phase. More precisely, each distance is calculated as the Euclidean distance between the current multidimensional vector and the center of gravity of the reference multidimensional vector group for which the distance between the current multidimensional vector and this group is calculated.

[0077] The smallest distance is calculated between the current multidimensional vector and the group to which the current multidimensional vector is assigned. The estimation module 12 provides an estimated Nv level of driver mind-wandering corresponding to the mind-wandering level associated with the reference multidimensional vector group closest to the current multidimensional vector.

[0078] It should be noted that when one of the physiological sensors is not reliable, the dimension corresponding to this sensor is not taken into account for the calculation of the distances between the current multidimensional vector and each of the groups from the learning phase, which makes it possible to obtain a level of mind wandering of the driver, independently of the feedback from this physiological sensor.

[0079] Advantageously, it is possible to assign weighting factors to the distance measurement, in order to give different weights to the various inputs considered. For example, the scientific literature has shown that monotonous and undemanding driving environments (such as highways) facilitate the emergence of mind-wandering. It is therefore possible to weight the physiological and behavioral variables more heavily in these road contexts. As a result, a high level of mind-wandering can be reached more quickly. The weights of the distance calculation can thus be adapted according to the context.

[0080] Following the second step 104 of estimating the driver's thought-wandering level Nv, the estimation module 12 delivers to the alert management module 18 the estimated level Nv of the driver's thought-wandering, as well as the duration d during which the driver is in a state of thought-wandering, and the level of reliability of the estimation performed.

[0081] In parallel with the second step 104 of estimating the driver's thought-wandering level Nv, the contextualized stimulation process 100 includes a step 106 implemented continuously by the risk detection module 14, as it receives potential obstacles {Oj} from the module Modeling 10. This step 106 is the identification of one or more risks Rj related to the modeled driving environment, based on potential obstacles Oj. The risks Rj thus identified are sent, in this step 106, to the analysis means 16.

[0082] The risk detection step 106 is followed by a step 108, implemented continuously by the analysis means 16, as they receive from the modeling module 10 the potential obstacles Oj, from the risk detection module 14 the identified risk(s) Rj, and from the estimation module 12 the estimated level Nv of the driver's mind wandering. This step, implemented by the analysis means 16, includes determining the driver's visual scanning field and providing an overall risk level C related to the driver's understanding of the driving environment, based on the risk(s) Rj identified by the detection module 14, the corresponding potential obstacles Oj, and the determined visual scanning field.As explained previously, the visual scanning field is represented by a cone of vision whose apex is located at the driver's eyes, and which scans a driving scene through the driver's eye or head movements. The shape of the base of this cone can be adjusted according to the estimated level Nv of the driver's mind wandering. This potentially adjusted visual scanning field allows for the estimation of an overall risk level C related to the driver's understanding of the driving environment, for example, as a probability between 0 and 1.

[0083] Steps 104 of estimating the driver's level of mind wandering, and 108 of determining the driver's visual exploration field and providing the overall risk level C related to the driver's understanding of the driving environment, are followed by a step 110 of determining alerts to be generated, implemented by the alert management module 18.

[0084] During this step 110 of determining which alerts to generate, the alert management module 18 receives, from the estimation module 12, the estimated level Nv of the driver's mind wandering, and the duration d during which the driver is in this state. The alert management module 18 also receives in this step 110, from the analysis means 16, the overall risk level C related to the driver's understanding of the driving environment, and from the modeling module 10, the complexity level D of the driving environment.

[0085] In this step 110, depending on the data received, the alert management module 18 is able to trigger an alert to the driver of a high state of mind wandering, or an alert of a risk linked to a lack of vigilance, using the human-machine interface(s) 20.

[0086] In particular, in this step 110, when the overall risk level C related to the driver's understanding of the driving environment is strictly greater than a predetermined risk level, then the alert management module 18 alerts the driver 112 to a lack of vigilance on their part, and specifically to the fact that they are not aware of a risk related to the driving environment, using one or more human-machine interfaces 20. For example, it displays a message on a vehicle screen, and / or vibrates the steering wheel using a haptic device, and / or emits an audible message using a loudspeaker. The visual, haptic, or audible message is specific to indicating a lack of vigilance on the part of the driver. The predetermined risk level is, for example, equal to 0.5 when the overall risk level C is coded between 0 and 1.

[0087] This alert to the driver of a risk linked to a lack of vigilance on his part is possibly modulated according to the overall risk level C provided by the analysis means 16. For example, its triggering may be anticipated or of higher intensity if the driver has a very low level of awareness of the driving situation.

[0088] Following this alert 112, when the driver does not react to the urgency of the situation, the advanced driver assistance system implementing the contextualized attention stimulation system 1, may act on the vehicle by using an emergency braking device, or a trajectory control device.

[0089] Step 110 also includes the possibility of alerting 114 the driver that he is in a state of heightened mind-wandering. This 114 alert is inhibited when the 112 alert to the driver of a risk related to a lack of vigilance on his part is activated.

[0090] This avoids over-solicitating the driver, which could generate a rejection of the function, and prevents the mind-wandering alert from reducing the attention that the driver could pay to other higher priority alerts, in particular the risk alert.

[0091] The 114 alert specific to the case of mind-wandering is a contextual alert, that is, it depends on the situation in which the driver finds themselves, in particular the complexity level D of the driving situation. This alert is not systematically sent to the driver when they are experiencing mind-wandering; it may depend on: - the complexity level D of the driving environment, - the level of mind wandering, and - the history of triggering previous alerts.

[0092] Indeed, to increase the acceptability of such an alert, it is necessary to control the frequency of its triggering. For example, we can formulate the following rule: if a previous alert has been activated, we can define a duration T corresponding to the minimum delay required before the re-emission of a new alert. Then the system will not send a new alert to the driver, who is again in a state of mind-wandering, before the duration T.

[0093] Optionally, other criteria may be used to decide whether or not to display the alert specific to the case of mind wandering, for example the duration d since the driver has been in the mind-wandering state, or the number of alerts triggered on the driver's journey.

[0094] Step 110 of determining alerts to be generated includes the comparison between the estimated Nv level of mind wandering and a predetermined mind wandering threshold, for example equal to 2, or to 2.5.

[0095] When the estimated Nv level of mind wandering is strictly greater than the predetermined mind wandering threshold and the complexity level D of the driving environment is greater than a predetermined complexity level, then the alert management module 18 alerts 114 the driver to recall him to his driving activity.

[0096] Alert 114 can also be modulated according to the complexity level D of the driving environment and the time t since the last alert was lifted. An example of a decision table that can be used by the alert management module 18 to determine this alert is illustrated [Fig.3].

[0097] In this table, the Nv level of mind wandering is rated out of 4, and the predetermined mind wandering threshold is 2, meaning that there is no alert for a high state of mind wandering when the estimated Nv level of mind wandering is 1 or 2. Furthermore, in this table, the complexity level D of the driving environment is 1 or 2, and the predetermined complexity level is 1. A complexity level D of 1 is considered a low complexity level, and a complexity level D of 2 is considered a high complexity level.

[0098] According to this decision table, when the estimated Nv level of mind-wandering is 3, and the complexity level D of the driving environment is low, the Alert management module 18 only alerts the driver to their high state of mind-wandering if the time t since the last alert is greater than the time T, for example, equal to 2 minutes. Otherwise, that is, if the time t is less than or equal to the time T, and although the estimated mind-wandering level Nv is 3, alert management module 18 inhibits 120 (sign "-" to the last column "114") triggering the alert to the driver of his high state of mind wandering.

[0099] On the other hand, in the case where the level of complexity D of the driving environment is high and where the estimated level Nv of mind wandering is 3 or 4, the alert management module 18 alerts 114 systematically the driver of his high state of mind wandering, regardless of the duration t since the last alert (the signs “+” in the last column “114” are all the more numerous as the intensity of the alert 114 is greater).

[0100] Of course, the table in [Fig. 3] is only an illustrative example of the dependence of driver alert 114 on a high state of mind-wandering, on the estimated level Nv of mind-wandering, on the complexity level D of the driving environment, and on the time t since the last alert was lifted. Many other implementations are possible.

[0101] The alert 114 for high state of mind wandering can be modulated according to the value of the estimated Nv level of mind wandering, the number of alerts already raised on the driver's journey, the duration d spent in a state of mind wandering, the time spent driving, etc., for example by using different colours, sounds and sound levels.

[0102] According to one embodiment of the invention, following the driver's alert step 114 indicating a heightened state of mind-wandering, the alert management module 18 verifies 116 that the driver is responding to the alert 114. This verification step 116 aims to inhibit the alert 114 without waiting for the next iteration of the process, when the driver reacts sufficiently quickly. In this step 116, a driver reaction is detected, for example: - by a change in the dynamics of the activity of the pedals, the steering wheel, or other controls such as the turn signals, and / or - through a change in the dynamics of visual exploration of the driver's gaze, and / or - when the driver responds to an avatar used to talk to the driver, in the event that the 114 alert uses such an avatar.

[0103] When such a reaction is detected during this verification step 116, the alert management module 18 inhibits the driver alert 120. Conversely, when no driver reaction is detected during this verification step 116, the alert management module 18 intensifies the alert 118, for example by increasing the flashing of LEDs, beeps, or by making an avatar speak louder when addressing the driver.

[0104] This embodiment of the invention is easily implemented in the vehicle, allows for a rapid and real-time calculation of the mind-wandering state of the driver, and a personalization of an advanced driver assistance system incorporating the invention, which facilitates the acceptability of the alerts from the contextualized stimulation system according to the invention. It also allows for an evaluation of the effectiveness of the alerts, through verification of the driver's reactions, and therefore an adaptation of the alert management module based on this evaluation.

[0105] It should also be noted that in this embodiment of the invention, the driver warning step regarding a risk related to a lack of vigilance is implemented when the overall risk level is strictly greater than the predetermined risk level. Alternatively, the driver warning step regarding a risk related to a lack of vigilance is implemented when the overall risk level is greater than or equal to a predetermined risk level. In this alternative, when the risk level related to a lack of vigilance is strictly less than the predetermined risk level, the driver warning of a high level of mind-wandering is, for example, systematically triggered when the estimated level of mind-wandering is greater than or equal to a predetermined mind-wandering threshold and when the driving complexity level is greater than or equal to a predetermined complexity level.In other words, strict inequality conditions can be replaced by non-strict inequality conditions without changing the nature of the invention.

[0106] According to a particular embodiment, the estimated Nv level of driver mind wandering is used to modulate (delay the appearance or, conversely, bring about earlier or more markedly) alerts or indications produced by other vehicle functions / systems, for example by delaying the appearance of an alert or indication related to the presence of a distant obstacle when the driver has a low level of mind wandering.

[0107] Of course, the invention is not limited to the examples just described, and many modifications can be made to these examples without departing from the scope of the invention. In particular, the features of the different embodiments can be combined to carry out the invention, provided that these embodiments are not incompatible with each other.

Claims

Demands

1. A contextualized driver attention stimulation system (1), comprising: - at least one human-machine interface (20), - an estimation module (12) for a level (Nv) of driver thought wandering, - a modeling module (10) for a vehicle driving environment, capable of providing a level of complexity (D) for the modeled driving environment, - a risk detection module (14) for the modeled driving environment, capable of identifying one or more risks (Rj) related to the driving environment, - analysis means (16) capable of determining a visual exploration field for the driver, and of providing an overall risk level (C) related to the driver's understanding of the driving environment, based on the risk(s) (Rj) identified by the detection module (14) and the determined visual exploration field,the contextualized stimulation system (1) being characterized in that it is configured to alert the driver that he is in a high state of mind-wandering, using at least one human-machine interface (20), when the estimated level (Nv) of the driver's mind-wandering is greater than a predetermined mind-wandering threshold and when the level of complexity (D) of the driving environment is greater than a predetermined level of complexity, and to alert the driver of a risk related to a lack of vigilance, using at least one human-machine interface (20), when the overall risk level (C) related to the driver's understanding of the driving environment is greater than a predetermined risk level.

2. Contextualized attention stimulation system (1) according to claim 1, comprising means for inhibiting the alert related to a high state of mind-wandering of the driver as soon as an alert of a risk related to a lack of vigilance is triggered by the contextualized stimulation system (1).

3. A contextualized attention stimulation system (1) according to claim 1 or 2, wherein the analysis means (16) are suitable to provide an overall risk level (C) related to the driver's understanding of the driving environment, are configured to take into account the level (Nv) of mind wandering estimated by the estimation module (12).

4. Contextualized attention stimulation system (1) according to claim 3, wherein the analysis means (16) are capable of determining a visual exploration field as a function of the driver's gaze, and of adjusting the visual exploration field as a function of the estimated level (Nv) of mind wandering to calculate the overall risk level (C) related to the driver's understanding of the driving environment.

5. A method for contextualized stimulation (100) of a vehicle driver's attention, using at least one human-machine interface (20), an estimation module (12) of a level (Nv) of the driver's mind wandering, a modeling module (10) of a vehicle driving environment, capable of providing a level of complexity (D) of the modeled driving environment, a risk detection module (14) related to the modeled driving environment, capable of identifying one or more risks (Rj) related to the driving environment, analysis means (16) capable of determining a visual scanning field of the driver and providing an overall risk level (C) related to the driver's understanding of the driving environment from the risk(s) (Rj) identified by the detection module (14) and the determined visual scanning field,the contextualized stimulation process (100) being characterized in that it comprises the following steps: - modeling (102) of the driving environment by the modeling module (10), and provision of the complexity level (D) of the driving environment modeled by the modeling module (10), - estimation (104) of the level (Nv) of mind wandering by the estimation module (12), - identification (106) of one or more risks related to the driving environment modeled by the detection module (14), - determination (108) by the analysis means (16) of the driver's visual scanning field and provision of an overall risk level (C) related to the driver's understanding of, the driving environment, based on the risk(s) (Rj) identified by the detection module (14) and the visual scanning field determined by the analysis means (16), - alert (114) to the driver of a high state of mind-wandering, using at least one human-machine interface (20), when the estimated level (Nv) of mind-wandering is greater than a predetermined mind-wandering threshold and when the level of complexity (D) of the driving environment is greater than a predetermined level of complexity, or - alert (112) to the driver of a risk related to a lack of vigilance, using at least one human-machine interface (20), when the overall risk level (C) related to the driver's understanding of the driving environment is greater than a predetermined risk level.

6. A method for contextualized attention stimulation (100) according to claim 5, wherein the driver alerting step (114), related to a high state of mind-wandering, is triggered in a manner conditioned on a history of previously performed alerts.

7. A contextualized attention stimulation method (100) according to claim 5 or 6, wherein the alert (114) to the driver of a high state of mind-wandering is adapted to the estimated level (Nv) of mind-wandering, and / or to a history of previous alerts of a high state of mind-wandering, and / or to the level of complexity (D) of the driving environment.

8. A method for contextualized stimulation (100) of attention according to any one of claims 5 to 7, wherein the alert (114) to the driver of a high state of mind-wandering or of a risk related to a lack of vigilance, is followed by a step of verification (116) of a driver's reaction to the alert (114), and a step of intensification (118) of the alert (114) in the absence of a driver's reaction, or of inhibition (120) of the alert (114) when a driver's reaction is confirmed.

9. A method for contextualized attention stimulation (100) according to any one of claims 5 to 8, wherein the estimation step (104) of a level of mind-wandering by the estimation module (12) comprises, on the one hand, a substep of receiving (1041) data representative of measurements physiological data of the driver, data representative of driver commands, and at least one data representative of a driving environment, and on the other hand a sub-step of quantification (1043) of a state of mind-wandering of the driver as a function of the data received.

10. A contextualized attention stimulation method (100) according to claim 9, wherein the quantification substep (1043) comprises a measurement of distances between a multidimensional vector representative of the received data and reference multidimensional vector groups associated with levels of mind-wandering, the estimated mind-wandering level being that of the reference multidimensional vector group closest to the multidimensional vector representative of the received data.

Images