System and method for assessing driving risks for a driver of a vehicle

Abstract

The invention relates to a system for assessing driving risks for a driver of a vehicle, the system comprising: − a first set of sensors integrated into the vehicle and configured to collect, in real time, data relating to the external environment of the vehicle, − a second set of sensors integrated into the vehicle and configured to collect data relating to a driver-specific saccadic eye movement pattern, − an on-board processing unit programmed with an artificial intelligence algorithm, the unit being configured to: -- analyse the data collected by the first set of sensors to identify, in real time, a driving context, -- match the identified driving context to at least one reference saccadic eye movement pattern, -- analyse the data collected by the second set of sensors and perform a measurement of the magnitude of the deviations between the driver-specific saccadic eye movement pattern and the reference saccadic eye movement pattern, -- select one instruction from a plurality of instructions each allowing one or more actions to be generated, which selection depends on the magnitude of the measured deviations, -- execute the selected instruction to generate the corresponding one or more actions.

Description

Description Title: System and method for assessing the driving risks of a vehicle driver. [Technical field [1] The subject of the invention is a system and a method for assessing the driving risks of a vehicle driver. The subject of the invention is also a method for producing an artificial intelligence algorithm for implementing this method. [2] The invention relates primarily, but not exclusively, to the field of road safety and, more particularly, to the use of sensors and artificial intelligence cooperating to detect and evaluate potential risky driver behaviors. This evaluation can then be used to provide feedback contributing to safer and more responsible driving. State of the art [3] The majority of road accidents are due to driver error which is the consequence of an altered cognitive state (drop in concentration, reduced coordination, latency in decision-making). This alteration is generally due to a state of control: drowsiness (illness, fatigue, etc.) and / or substances (alcohol, narcotic, drug, etc.) and / or distraction (mobile phone, music, noise, brightness, etc.) and / or relaxation (habit of the road, post-work stress, high stress due to a mission or personal worries outside of work, etc.). [4] Most drivers, however, are unable to determine when their condition will prevent them from driving normally and / or increase the risk of an accident. This is true even if this state of control may be temporary, and / or even if the driver can sometimes overcome the consequences. [5] Depending on the driver's condition and / or the type of vehicle (car, bus, truck), driving risks may also be exacerbated - or mitigated - depending on the driving contexts (driving in an urban area in hours peak, driving in a rural area in winter conditions, driving on the highway at high speed, ...). [6] Many vehicles today are equipped with advanced driver assistance systems (ADAS). These systems assist the driver by automatically influencing the vehicle's controls.Currently, the most common ADAS systems allow, among other things: to automatically adjust the vehicle's speed to maintain a safe distance from another vehicle (Adaptive Cruise Control); to detect a potential collision situation and automatically apply the brakes if the driver does not react in time (Automatic Emergency Braking); to detect lane markings and automatically correct the vehicle's direction to maintain its trajectory in the lanes (Lane Keeping Assist); to recognize road signs and adjust the vehicle's speed according to the detected speed limits (Traffic Sign Recognition); to warn the driver of the presence of vehicles in blind spots and take corrective action if necessary (Blind Spot Monitoring), etc. [7] However, these ADAS systems only allow the detection of driving contexts that depend exclusively on the vehicle's external environment and / or the actual driving scene of said vehicle (safe distance from another vehicle, deviation from lane markings, road signs, vehicles in blind spots, etc.) and not on the driver's condition. Their ability to assess driving risks and reduce accidents is therefore limited. [8] There is therefore a need to assess a driver's driving risks more precisely. [9] Faced with this state of affairs, the invention aims in particular to achieve all or part of the following objectives to improve road safety and reduce accidents: increase the capacity to detect risky behavior of the driver; reliably predict risky behavior of the driver; propose actions in response to risky behavior, these actions in response being earlier than those offered by the aforementioned ADAS systems; proposing adapted, circumstantial and personalized response actions to the driver; providing a system and method for assessing driving risks that is simple, reliable, robust, and with a high degree of precision. Presentation of the invention

[0010] The solution proposed by the invention is a driving risk assessment system for a driver of a vehicle, said system comprising: - a first set of sensors integrated into the vehicle, configured to collect data in real time relating to the vehicle's external environment, - a second set of sensors integrated into the vehicle, configured to collect data relating to a driver-specific eye saccade pattern, - an on-board processing unit programmed with an artificial intelligence algorithm, said unit being configured to: -- analyze the data collected by the first set of sensors to identify a driving context in real time, -- associate at least one reference eye saccade pattern with the identified driving context, -- analyze the data collected by the second set of sensors and perform a measurement of the magnitude of the deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern, -- select one instruction from a plurality of instructions, each of which generates one or more actions, which selection is a function of the extent of the deviations measured, -- executing the selected instruction to generate the corresponding action(s), and wherein the artificial intelligence algorithm is trained using a set of training data collected from drivers in various altered cognitive states, and placed in a variety of driving contexts, such that the processing unit can associate a saccade pattern reference ocular tests called “abnormal” for each altered cognitive state and each driving context identified.

[0011] Thanks to the combined analysis of driving contexts and driver-specific eye saccade patterns, the invention makes it possible to assess potential risky behaviors, with increased accuracy compared to prior art ADAS systems. This simple, reliable, robust assessment, offering a high degree of accuracy, makes it possible to generate proactive, personalized and earlier actions than those proposed by current ADAS systems, thus contributing to reducing driving risks and improving road safety.

[0012] Analyzing the magnitude of the deviations between the driver's own eye saccade patterns and the reference patterns makes it possible to accurately determine the driver's cognitive state in a given driving context, including their concentration, fatigue, level of alertness, degree of attention, and ability to react appropriately to sudden events and / or hazards. The system thus makes it possible to determine a risky deviation from driving in a cognitive state “normal”. This intelligent analysis of eye saccade patterns not only provides a much more accurate, comprehensive, and early assessment of driving risks, but also allows for the adaptation and personalization of relevant response actions, to provide increased safety and assistance in real time.

[0013] The AI ​​algorithm also associates different driving contexts with "abnormal" reference eye saccade patterns, i.e., eye patterns in which drivers acted with reduced attention and vigilance, with slow or even impaired reaction times, in the face of sudden changes, events or dangers, and corresponding to risky driving. These reference eye saccade patterns "abnormal" reflects a state of fatigue and / or abnormal stress (loud noise, strong light, etc.) of a driver and / or driving under the influence of substance(s) (alcohol, narcotic, drug, medication, etc.). The processing unit is thus able to determine whether the driver is under the influence of substance(s) (and with which substance(s)) and the associated risk level, way of selecting other instructions that are specific to the altered cognitive state thus detected.

[0014] Other advantageous features of the invention are listed below. Each of these features can be considered alone or in combination with the remarkable features defined above. Each of these features contributes, where appropriate, to the resolution of specific technical problems defined further in the description and in which the other features defined above do not necessarily contribute. The following characteristics may thus be the subject, where appropriate, of one or more divisional patent applications:

[0015] According to one embodiment, the processing unit is configured to measure the magnitude of the deviations by determining whether said measured deviations are included in reference value ranges.

[0016] According to one embodiment, the processing unit is configured to execute the selected instruction only if the measured deviations are included in reference value ranges, for a predetermined time.

[0017] According to one embodiment, the processing unit is configured to measure the magnitude of deviations on a combination of characteristics of the eye saccade patterns.

[0018] According to one embodiment, the artificial intelligence algorithm is trained using a set of training data collected on drivers being in various impaired cognitive states, and placed in a variety of driving contexts, such that the processing unit can associate a so-called "abnormal" reference eye saccade pattern with each identified impaired cognitive state and driving context.

[0019] According to one embodiment, the processing unit is configured to: - detect a correspondence between the eye saccade pattern captured in real time and at least one of the “abnormal” reference eye saccade patterns associated with an altered cognitive state, - if there is a match, select an instruction that is specific to the detected altered cognitive state, - execute the selected instruction to generate the action(s).

[0020] According to one embodiment, the second set of sensors is configured to collect data on the following characteristics of the eye saccades: their speed, their amplitude, their frequency, their direction and the fixation duration between the saccades.

[0021] According to one embodiment, the instructions are predefined and recorded in a database connected to the processing unit and / or the instructions are dynamically created by said unit according to the identified driving context.

[0022] Another aspect of the invention relates to a method for assessing the driving risks of a vehicle driver, which method comprises the following steps: - collect in real time an initial set of data relating to the vehicle's external environment to identify a driving context in real time, - collect a second set of data relating to a driver-specific eye saccade pattern, - using an artificial intelligence algorithm: -- analyze the first set of data to identify a driving context in real time, -- associate with the identified driving context, at least one reference eye saccade pattern, -- analyze the second data set and perform a measurement of the magnitude of deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern; - select an instruction from a plurality of instructions, each of which generates one or more actions, which selection is a function of the extent of the deviations measured, - executing the selected instruction to generate the corresponding action(s), said method further comprising a step of training the artificial intelligence algorithm using a set of training data collected on drivers being in various altered cognitive states, and placed in a variety of driving contexts, so as to associate a pattern of reference eye saccades said to be “abnormal” in each altered cognitive state and in each driving context identified.

[0023] According to one embodiment, the method comprises the steps of: - sequence each driving context identified into a series of key moments, - associate at least one reference eye saccade pattern with each identified key moment, - associate reference characterization data with one or more identified key moments, known as “critical key moments”.

[0024] According to one embodiment, the method comprises the following steps for each critical key moment: - collect in real time a third set of data characterizing said key moment, - analyze in a way that combines the second data set and the third data set and perform: -- on the one hand a measure of the extent of the deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern associated with said key moment and, -- on the other hand a measure of the extent of the deviations between the real-time characterization data and the reference characterization data, - select an instruction from a plurality of instructions, each of which allows one or more actions to be generated, which selection is a function of the magnitudes of the measured deviations, - execute the selected instruction to generate the corresponding action(s).

[0025] According to one embodiment of the method: - a first key moment is characterized by data relating to elements of the vehicle's external environment to be seen and understood by the driver in said driving context, - a second key moment is characterized by characterization data relating to the direction of the driver's gaze in relation to said elements, - a third key moment is characterized by characterization data relating to unconscious reflex reactions of the driver, - a fourth key moment is characterized by characterization data relating to conscious, reflective reactions of the driver to the vehicle controls, - a fifth key moment is characterized by characterization data relating to vehicle reactions following these reflected reactions.

[0026] According to one embodiment of the method, the second key moment, the third key moment, the fourth key moment and the fifth key moment are critical moments.

[0027] Yet another aspect of the invention relates to a method for realizing an artificial intelligence algorithm for implementing the method or one of the preceding characteristics, comprising a step of training said algorithm using a set of training data collected in a variety of driving contexts, which training data is used for the identification of the driving contexts and for the identification of eye saccade patterns in said contexts.

[0028] According to one embodiment of the method, training data is collected during driving sessions performed both on closed test circuits and on public roads.

[0029] According to one embodiment of the method, training data is collected from drivers in an unaltered cognitive state during driving sessions.

[0030] According to one embodiment of the method, training data is collected from drivers in various altered cognitive states during driving sessions on closed test circuits. Brief description of the figures

[0031] Other advantages and characteristics of the invention will appear more clearly on reading the description of the embodiments which follow, with reference to the appended drawings, produced as indicative and non-limiting examples and in which: [Fig. 1] is an overview of a system according to the invention. [Fig. 2A] illustrates an example of a “normal” eye saccade pattern. [Fig. 2B] illustrates an example of an “abnormal” saccade eye pattern. [Fig. 3] illustrates an example of a table associating different driving contexts with reference eye saccade patterns, deviation variances and instruction differences. Description of the embodiments

[0032] The invention may implement one or more computer programs executed by equipment. For the sake of clarity, it is to be understood within the meaning of the invention that "equipment does something" or "the computer program does something" means "the computer program executed by a processing unit of the equipment does something".

[0033] Again for the sake of clarity, the present invention may refer to one or more “processes”. These correspond to the actions or results obtained by the execution of instructions of one or more computer applications. Also, it should also be understood within the meaning of the invention that "a process is adapted to do something" means "the instructions of a computer application executed by a processing unit do something."

[0034] For the sake of clarity, the following clarifications are made to certain terms used in the description and claims: - “Computer resource” can be understood in a non-limiting way as: component, hardware, software, file, connection to a computer network, quantity of RAM memory, hard disk space, bandwidth, processor speed, number of CPUs, etc. - "Computer server" can be understood in a non-limiting way as: computer device (hardware or software) comprising computing resources to perform the functions of a server and which offers services, computer, plurality of computers, virtual server on the Internet, virtual server on the Cloud, virtual server on a platform, virtual server on a local infrastructure, server networks, cluster, node, server farm, node farm, etc. - “Processing unit” can be understood in a non-limiting way as: processor, microprocessors, CPU (for Central Processing Unit), etc. - “Database” can be understood in a non-limiting manner as a structured and organized set of data recorded on media accessible by computer equipment and able to be queried, read and updated. Data can be inserted, retrieved, modified and / or destroyed. Management and access to the database can be ensured by a set of computer applications that constitute a database management system (DBMS). - “Vehicle” can be understood in a non-limiting way as a car, a truck or a transport bus (thermal and / or electric engine), a construction machine (bulldozer, excavator, forklift, mobile crane, etc.), a motorized two-wheeler (thermal and / or electric engine), a boat, an aircraft (plane, rocket, etc.), etc. - “X and / or Y” means: X alone or Y alone or X+Y. - Unless otherwise stated, the use of the ordinal adjectives "first", "second", etc., to describe an item merely indicates that different occurrences of similar items are being referred to and does not imply that the items so described must be in any given sequence, whether in time, space, ranking, etc. - Generally speaking, it will be appreciated that on the various attached drawings, the objects are arbitrarily drawn to facilitate their reading.

[0035] In Figure 1, a vehicle V is equipped with all the computing resources allowing the invention to be implemented. It is in particular equipped with a first set of sensors Cn, C12, C13 configured to collect data relating to the external environment of the vehicle (hereinafter referred to as environmental sensors) and an on-board processing unit 1.

[0036] The vehicle's external environment consists of a multitude of elements that can influence driving. These elements include, but are not limited to, surrounding vehicles (such as cars, trucks, motorcycles, bicycles, etc.); road infrastructure (such as pedestrian crossings, traffic lights, road signs, road markings, roundabouts, intersections, etc.); weather conditions (rain, snow, fog, strong winds, etc.); pedestrians and animals crossing a traffic lane or on a sidewalk; physical obstacles on a traffic lane (such as debris, potholes, barriers, etc.); emergency vehicles (ambulances, police cars, fire trucks, etc.); environmental elements natural elements such as trees; temporary elements (roadworks, construction zones, demonstrations, etc.); lighting conditions; road conditions (type of surface, dry, wet, icy, etc.); etc. These elements, whether dynamic or static, can have an impact on driver behavior.

[0037] According to one embodiment, the sensors Cn, C12, C13 are further configured to collect qualitative data associated with the movement of the vehicle in a given external environment. For example, for a given external environment (and common to all drivers), the behavior of the vehicle (speed, lane in which the vehicle is traveling, frequencies of lane changes, pitching, yaw, roll, etc.) can influence the analysis and identification of a driving context. Taking into account this qualitative data makes it possible in particular to differentiate the way drivers drive and to have a more detailed and nuanced analysis and production of information.

[0038] Environmental sensors Cn, C12, C13 may include, but are not limited to, wide field cameras, infrared cameras for night vision, radars, lidars, ultrasonic sensors, rain / snow sensors, outdoor thermometers, ambient light sensors, microphones, motion sensors, geolocation sensors (GPS type), a network of sensors integrated into the CAN bus (or any other type of data bus) of the vehicle.Wide field cameras can be used to capture a panoramic view of the vehicle's surroundings, infrared cameras are particularly useful in low light or foggy conditions, radars can be used to detect the distance and speed of surrounding objects and vehicles, lidars can be used for detailed 3D mapping of the vehicle's immediate surroundings, ultrasonic sensors can be used to detect objects near the vehicle, rain / snow sensors can detect weather conditions, thermometers can measure ambient temperature, ambient light sensors can detect lighting conditions, microphones can pick up environmental noises such as ambulance sirens, etc. fire trucks or police vehicles, motion sensors can be used to detect specific movements such as lateral movement or the movement of approaching objects. Geolocation sensors can provide precise location information to identify the type of area the vehicle is in, for example, a highway, an urban area, a country road. The network of sensors integrated into the vehicle's CAN (Controller Area Network) bus allows for the collection of qualitative data associated with the movement of the said vehicle, and includes, for example, a speed sensor, a gyroscope, an accelerometer, etc. These environmental sensors, alone or in combination, provide a detailed representation and understanding of the vehicle's external environment and allow for the precise and nuanced definition of a driving context in real time.

[0039] The data collected by the environmental sensors Cn, C12, C13 make it possible to identify a driving context in real time. By "driving context" we mean a set of conditions (e.g. speed limit, respect for priorities, etc.) and variables (geographical position of the vehicle, position and alternative route on the traffic lanes, rain, snow, person crossing a pedestrian crossing, vehicle not respecting a priority, etc.), preferably combined with qualitative data associated with the movement of the vehicle, which characterize a specific driving context.

[0040] For example, some driving contexts may be of the type: "driving in an urban area during rush hour", "driving in an urban area during rush hour at a constant and moderate speed", "driving in an urban area during rush hour with fast and choppy speed and with numerous lane changes in the same direction", "driving in a rural area in heavy rain", "driving in a rural area in heavy rain with acceptable grip", "driving in a rural area in heavy rain with aquaplaning", "driving on a motorway at high speed", "driving on a motorway at high speed, with travel mainly in the left lane", "driving on a motorway at high speed with alternating travel on the right and center lanes", "driving in an urban area with frequent intersections", “driving in urban areas with frequent intersections while respecting stop and priority sequences”, “driving in urban areas with frequent intersections while not respecting stop and priority sequences”, or “nighttime driving in low-light conditions”. For example, if the environmental sensors detect the presence of the vehicle in an urban area, heavy traffic, frequent pedestrian crossings and traffic lights, unpredictable movements of vehicles and cyclists, processing unit 1 can identify this case as a “driving in an urban area during rush hour” scenario. If the environmental sensors detect the presence of the vehicle in a rural area and heavy rain, processing unit 1 can identify this context as a “driving in a rural area in heavy rain” scenario.A “high-speed highway driving” scenario can be identified when environmental sensors detect the vehicle’s presence on a highway, without traffic congestion. Similarly, a “urban driving with frequent intersections” scenario can be identified if environmental sensors detect urban environments with many intersections. A “nighttime driving in low-light conditions” scenario can still be identified if light sensors detect low illumination levels.

[0041] According to a preferred embodiment, the processing unit 1 is programmed with a first artificial intelligence algorithm (or artificial intelligence (AI) model, the two terms being equivalent within the meaning of the invention). This first AI model, once trained, is used to identify, in a precise, reliable and robust manner, various driving contexts based on the data collected by the first set of sensors Cn, C12, C13. This identification is preferably carried out in real time.

[0042] The first AI model is advantageously trained using a set of training data from environmental sensors of the types described above. This training data can, for example, be retrieved from a database centralizing all the data from a fleet of vehicles and accessible from a computer server. The training data can be collected in a variety of contexts to ensure robust and accurate identification of driving contexts. Data collection sessions can notably be carried out both on closed test circuits and on public roads. On a test circuit, specific conditions can be simulated in a controlled manner, such as night driving, driving in variable weather conditions, or driving at high speeds. This allows specific data to be captured under conditions that would normally be difficult to encounter in a natural environment. On public roads, data collection can be carried out under more random and unpredictable and / or more varied conditions, including different traffic densities, various types of roads (motorways, urban streets, rural roads, etc.), and various interactions with different types of vehicles and pedestrians.

[0043] According to one embodiment, the first AI model is based on a deep neural network (or DNN). This type of model is capable of processing large amounts of multi-dimensional environmental data to identify complex driving contexts in real time. The DNN model also has the advantage of being able to generalize from training data to new situations not previously seen, which is particularly advantageous for the accurate identification of various driving contexts. Other AI models can, however, be used alone or in combination with a DNN model, such as decision tree-based models, support vector machines (SVMs) or DRL (Deep Reinforcement Learning) models. The combination of several models makes it possible in particular to increase the robustness and accuracy of the identification of driving contexts.

[0044] Vehicle V is also equipped with a second set of sensors C21, C22, C23 adapted to collect data relating to the driver's eye saccade patterns. Eye saccades are rapid and sudden eye movements made by the driver's eyes, in particular when they move from one focal point to another. These movements are generally involuntary and / or unconscious and are very brief (a few milliseconds).

[0045] The characteristics of the eye saccades captured by sensors C21, C22, C23 are preferentially: their speed, their amplitude, their frequency, their direction and the fixation duration between saccades. The speed of the saccades corresponds to the speed with which the eye moves from one point to another and provides indications on the state of vigilance, concentration and reactivity of a driver facing a driving event. The amplitude of the saccades is measured to determine the distance traveled by the eye during each saccade, thus providing indications on the amplitude of the driver's attention zone. The frequency of the saccades corresponds to the number of saccades per unit of time and appears to be a good indicator of the level of attention or vigilance of the driver. The fixation duration corresponds to the time during which the eye remains motionless between two saccades.This data makes it possible, in particular, to assess the level of concentration on one or more elements of the external environment. The direction of the saccades makes it possible to determine whether the eye movements are horizontal, vertical or diagonal, and gives indications on one or more elements of the external environment on which the driver is concentrating. The second set of sensors C21, C22, C23 thus makes it possible to collect data on a very wide and complete range of specific characteristics of eye saccades.

[0046] The sensors C21, C22, C23 may include, but are not limited to, optical sensors directed towards the driver's eyes from strategic positions in the passenger compartment of the vehicle V, such as infrared cameras and / or event cameras integrated into the dashboard or steering wheel, LED-based eye tracking sensors installed on the interior rearview mirror to measure gaze orientation and direction, electro-oculographic (EOG) sensors integrated into eyeglass frames to measure variations in electrical potential associated with eye movements, "dual-view" sensors installed on the interior rearview mirror and using a combination of cameras to capture both the eye image and the scene in front of the driver for a more detailed analysis of what the driver is looking at and his eye behavior.All of these sensors can work synergistically to quickly provide a complete, accurate, and multidimensional profile of the driver's eye movements.

[0047] Data collection from sensors C21, C22, C23 is preferably carried out in real time. To save vehicle resources, this collection can however be triggered only when a sudden and / or risky event is identified in the driving context and / or triggered temporarily (for example every 10 minutes).

[0048] Other eye sensors can be used in combination with sensors C21, C22, C23, to capture additional data such as blinks and eyelid openings, pupil diameter, eye focusing distance, etc.

[0049] The applicant found that the analysis of saccade eye patterns made it possible to accurately determine the cognitive state of the driver in a given driving context, including their state of fatigue, their level of alertness, their degree of attention, and their ability to make decisions. For example, a driver must have a high level of alertness when driving at night in low light conditions and in adverse weather conditions (rain or snow). The level of alertness may be lower when driving during the day and in favorable weather conditions (sun).

[0050] Figure 2A shows an example of an eye saccade pattern. In this example, the eye saccades have regular movements, alternating rightward and leftward. The rightward saccades have a velocity v1 , an amplitude a1 and a frequency f1 . The leftward saccades have a velocity v2 , an amplitude a2 and a frequency f2 . The fixation duration between the rightward and leftward eye saccades corresponds to the duration d, this duration also being regular. This type of pattern may reflect a high level of driver vigilance, i.e., the driver is fully attentive to his driving environment, and is able to react quickly to sudden changes or events and / or potential hazards.

[0051] Figure 2B shows another example of an eye saccade pattern, in the same driving context as Figure 2A. In this example, the saccades are irregular. Two saccades are successively oriented to the right, followed by a single one oriented to the left. The saccades directed to the right have respectively velocities v3, v4, amplitudes a3, a4 and frequencies f3, f4 which are not regular. The saccade directed to the left also has amplitudes which are irregular. In addition, the fixation duration between the eye saccades is much longer than in the example of Figure 2A. This type of pattern may reflect a low level of vigilance, the driver being in an altered cognitive state for example by an advanced level of fatigue, close to drowsiness. This driver is inattentive to his driving environment and / or unable to react quickly to sudden changes or events and / or potential dangers, which increases the risk of errors or accidents.

[0052] The data from the sensors C21, C22, C23 are analyzed by the processing unit 1. According to a preferred embodiment, the processing unit 1 is programmed with a second AI model which, once trained, is capable of accurately, reliably, robustly, and in real time identifying various eye saccade patterns. According to one embodiment, the first AI model and the second AI model are a single AI model.

[0053] The second AI model is advantageously trained using a set of training data from sensors of the type described above to capture data relating to the characteristics of eye saccades. This training data can, for example, be retrieved from a database centralizing all the data of a fleet of vehicles and / or drivers and accessible from a server. The training data is collected in a variety of driving contexts to ensure robust and accurate identification of eye saccade patterns. This training data is notably collected on drivers who are in an unaltered, or “normal,” cognitive state, i.e., in good health, rested, without any influence whatsoever. These drivers are advantageously classified by family (age, sex, driving experience, type of vehicle driven, etc.).This training data is collected by putting drivers in a variety of driving contexts, which are advantageously those allowing the data to be collected from the first AI model mentioned above. The second AI model can thus combine different contexts. led to “normal” reference eye saccade patterns, i.e., eye patterns for which drivers acted safely and efficiently in these driving contexts.

[0054] According to one embodiment, the second AI model is advantageously similar to the aforementioned first AI model, i.e. based on a deep neural network (or DNN). This type of model is capable of processing large quantities of multidimensional data, such as the speeds, amplitudes, frequencies, directions and fixation durations of eye saccades, to identify numerous patterns in real time. As indicated above, other AI models can however be used alone or in combination with a DNN model, a combination of several models making it possible in particular to increase the robustness and precision of the detection of eye patterns.

[0055] Training data for the second AI model can also be collected from drivers put into an altered cognitive state, or "abnormal", i.e. put in a state of abnormal fatigue and / or under the influence of substance(s) (alcohol, narcotic, drug, medication, etc.) and / or in a state of abnormal stress (intense noise, strong light, etc.). These tests are preferably carried out on private test circuits or on simulators, under the supervision of duly approved and authorized medical teams. The drivers carrying out these tests are preferably the same as those initially in a normal cognitive state and / or belong to the same families. This training data is collected by putting the drivers in the aforementioned driving contexts.The second AI model can thus associate different driving contexts with "abnormal" reference eye saccade patterns, i.e. eye patterns for which drivers acted with reduced attention and vigilance, with slow or even impaired reaction times, when faced with sudden changes, events or dangers, and corresponding to risky driving.

[0056] Referring to Figure 3, the processing unit 1, by means of the AI ​​model(s) with which it is programmed, associates each context of driving identified Ci in real time, to a reference eye saccade pattern Schem.i.

[0057] After having identified a driving context Ci, the processing unit 1 compares the driver-specific eye saccade pattern (hereinafter “real-time pattern”) and the reference eye saccade pattern associated with said context (hereinafter “reference pattern”).

[0058] More specifically, processing unit 1 measures the magnitude of the deviations Aij (or variance of deviation) between these two patterns. Processing unit 1 therefore does not simply detect a difference between the patterns in a binary manner, but measures the magnitude of this difference. This quantitative measurement allows for a richer and more informative characterization of the observed deviation.

[0059] The measurement can be made on a single characteristic of the patterns, for example only speed (one-dimensional measurement). However, it is preferentially carried out on a combination of characteristics such as speed, amplitude, frequency, direction and fixation duration (multidimensional measurement).

[0060] A multi-dimensional measurement offers many advantages compared to a single-dimensional measurement. The result of the multi-dimensional measurement is indeed more robust and more precise because it is less sensitive to errors and / or aberrations in the measurement of a single characteristic. The multi-dimensional measurement also makes it possible to characterize in a more detailed and precise manner the cognitive state of the driver and / or an alteration in his state, this information which could go unnoticed if only one characteristic were monitored.

[0061] Depending on the magnitude of the deviations measured, the processing unit 1 selects an instruction from a plurality of instructions Aij, each of which makes it possible to generate one or more actions. These actions may be audible and / or visual alarm signals emitted in the vehicle, notifications transmitted to the driver, notifications transmitted to third parties (relatives, police, firefighters, etc.), actions on the vehicle controls (braking, speed adjustment, stopping the vehicle, limiting the vehicle functions, driving assistance, etc.), etc.

[0062] The applicant found that the extent of the deviations between the real-time patterns and the reference patterns made it possible to accurately determine the driver's cognitive state in a given driving context, and in particular their level of concentration, their state of fatigue, their level of vigilance, their degree of attention, their ability to react appropriately to sudden changes or events and / or dangers. The quantitative measurement of the deviations thus provides information on the level of driving risk.

[0063] For example, in a context of "high-speed highway driving," a 10% deviation may indicate that the driver is in a moderate state of fatigue, i.e., driving with an unproven risk-taking, but likely to increase over time. The selected instruction may then consist of transmitting a notification to the driver inviting him to take a break and / or automatically adjusting the vehicle speed. According to another example, in a context of "driving in a rural area in heavy rain," a 70% deviation may indicate that the driver is in a state of drowsiness close to sleep, inducing a proven risk-taking.The selected instruction may then consist of emitting an audible and / or visual alarm signal in the passenger compartment of the vehicle, automatically reducing the speed and transmitting a notification to the driver indicating that the vehicle will be automatically stopped within a specified period of time (the vehicle being automatically stopped at the end of this period if the driver does not do so voluntarily). According to yet another example, in a context of “night driving in low light conditions”, a deviation of 30% may indicate that the driver is in a state of stress, which may lead to proven risk-taking in the event of changes and / or potential dangers. The selected instruction may then consist of displaying a message on the vehicle’s dashboard aimed at calming the driver, and automatically reducing the speed.

[0064] Combining deviation variances Aij with specific instructions Aij has several advantages. First, the system provides a gradual and proportional response to the observed deviations, which allows for adjusting and personalizing actions based on the driver's observed cognitive state in a given driving context. Also, it offers high responsiveness and very good adaptivity in managing responses to different risky driving situations. It is also possible to combine several instructions to provide more elaborate responses that are better adapted to varied and / or complex driving contexts. Thus, the invention allows real-time monitoring and reaction adapted to the detected cognitive state of the driver.

[0065] According to one embodiment, the instructions Aij are predefined and recorded in a database 2 connected to the processing unit 1 at the time of system design or during updates. These instructions can be defined during series of studies, simulations and / or tests aimed at establishing the reference diagrams and making it possible to define appropriate actions for different driving contexts. Predefined instructions have several advantages. In particular, they are particularly reliable in standardized situations, and can be executed quickly, which is particularly advantageous in contexts where a rapid reaction is necessary.

[0066] According to another embodiment, the instructions Aij are created dynamically according to the identified driving context, in particular thanks to the AI ​​model(s) integrated in the processing unit 1. The latter can in particular be programmed to generate instructions adapted to new or particularly complex driving contexts. The dynamic generation of instructions also allows for finer customization of the system's responses, in particular to adapt to the cognitive particularities of each driver. In addition, due to its ability to create new instructions on the fly, the processing unit 1 can react more quickly to unforeseen events, thus increasing safety in real time.

[0067] According to yet another embodiment, some of the Aij instructions are predefined and others are dynamically generated.

[0068] Measuring the magnitude of deviations can consist of measuring a value assigned to all or part of the characteristics of the diagrams. For example, in a given driving context, a measured speed deviation of X% leads to the selection of a first instruction, a measured amplitude deviation of Y% leads upon selection of a second instruction, a measured frequency deviation of Z% leads to the selection of a third instruction, and so on. Similarly, a measured speed deviation of X% combined with a measured amplitude deviation of Y% and / or combined with a measured frequency deviation of Z%, leads to the selection of another instruction. As previously stated, however, a multidimensional measure is preferred.

[0069] According to the preferred embodiment illustrated in Figure 3, the measurement of the magnitude of the deviations consists of determining whether the measured deviations are included in reference value ranges (or windows) Aij. This method offers the advantage of taking into account natural variability and / or volatility of the characteristics of the eye saccades which are specific to the driver, and ensures a more robust and more nuanced detection of the deviations.

[0070] For example, in a given driving context C1, a measured deviation (of one or more characteristics of the diagrams) included in A10 = [W1%; W2%] does not result in any instruction selection; a measured deviation included in A11 = [X1%; X2%] leads to the selection of a first instruction A11; a measured deviation included in A12 = [Y1%; Y2%] leads to the selection of a second instruction A12; and a measured deviation included in A13 = [Y2%; Y2%] leads to the selection of a third instruction A13, and so on.

[0071] For example, for each context Ci, the number of value ranges Aij can be between 1 and 100 depending on the desired analysis precision. As an example again, for each value range Aij, the number of instructions Aij can be between 0 and 10.

[0072] The extent of each Aij value range is, for example, defined by a statistical model. It can also be personalized for each driver. In this case, personalized calibration of the Aij value ranges can be carried out, in particular, during test phases where the driver is placed in a multitude of driving contexts.

[0073] Advantageously, the selected Aij instruction is executed only if the measured deviations are included in the reference value ranges Aij, and for a predetermined time. This reduces the risk of reacting to one-off anomalies that are not representative of the general condition of the driver, so that the execution of unnecessary actions is minimized. In addition, the accuracy, robustness and consistency of the assessment of the driver's cognitive state and / or driving risk are improved, as the system assesses the driver's cognitive state over a period of time rather than at a single point in time.

[0074] In the case where the second AI model is trained with data collected on drivers put in an abnormal cognitive state, in particular under the influence of substance(s) (alcohol, narcotic or drug), the processing unit 1 not only measures the extent of the deviations Aij, but is also capable of recognizing whether the real-time patterns correspond to abnormal reference patterns. The processing unit 1 is thus capable of determining whether the driver is under the influence of substance(s) (and with which substance(s)) and the associated risk level (by the measurements Aij), so as to select other instructions which are specific to the altered cognitive state thus detected.

[0075] For example, the processing unit 1 may determine that a driver is under the influence of narcotics, their condition being likely to lead to risk-taking in the event of sudden changes or events and / or potential dangers. The selected instruction may then consist of emitting an audible and / or visual alarm signal in the passenger compartment of the vehicle, automatically reducing the speed and transmitting a notification to the driver indicating that the vehicle will be automatically stopped within a specified period of time (the vehicle being automatically stopped at the end of this period if the driver does not do so voluntarily). According to another example, the processing unit 1 may determine that a driver is under the influence of alcohol, their condition immediately leading to proven risk-taking. The selected instruction may then consist of automatically reducing the speed until the vehicle comes to a complete stop or automatically blocking the restarting of the vehicle.

[0076] According to one embodiment, each driving context is sequenced into several key moments. The processing unit 1 is notably programmed to carry out such sequencing in real time. Preferably, the sequencing of the key moments is as follows: M1 = what are the elements of the vehicle's external environment to see and understand in the driving context (e.g.: a person passes over a pedestrian crossing, a car cutting off a priority at an intersection, etc.)? M2 = what was the driver's gaze direction in relation to these important elements (e.g.: is his gaze directed towards the person crossing a pedestrian crossing, or towards the car cutting off the priority, etc.)? M3 = what were the driver's reflex (i.e. unconscious) bodily reactions in this driving context (movement of his face, body, hands, etc.)? M4 = What were his / her reflective (i.e. conscious) driving reactions in this driving context, including the driver's reactions to the vehicle controls? M5 = What was the vehicle's reaction to these deliberate driving actions?

[0077] The moment M1 can be determined / characterized by analyzing the data collected by the first set of sensors Cn, C12, C13.

[0078] The moment M2 can be determined / characterized by analyzing the data collected by the second set of sensors C21, C22, C23.

[0079] The M3 moment can be determined / characterized by analyzing data collected by sensors such as face sensors detecting the driver's facial expressions, video sensors tracking the driver's hand movements, gestures and / or body posture, voice sensors detecting the driver's voice, pressure sensors on the steering wheel providing information about the force with which the driver is holding the steering wheel, which can be an indicator of his stress or concentration level, touch or capacitive sensors placed on the auxiliary controls of the vehicle and allowing to detect any interaction with elements such as the indicators, windshield wipers, hazard lights, etc.

[0080] The M4 moment can be determined / characterized by analyzing data collected by steering sensors determining the angle and rotation speed of the steering wheel, gear lever position sensors determining gear changes, pedal sensors measuring pressure on the accelerator and brake and providing, for example, data on how the driver reacts to the driving context.

[0081] The M5 moment can be determined / characterized by analyzing data collected by speed sensors defining the vehicle speed, position and / or navigation sensors tracking the vehicle position, suspension sensors analyzing the movements and reactions of the vehicle suspension, stability control sensors monitoring the lateral and longitudinal movements of the vehicle to detect skidding or loss of control situations.

[0082] The data used to characterize a key moment are called “characterization data.” Each driving context can be sequenced with more or fewer key moments. And each key moment can be characterized by other characterization data.

[0083] According to a preferred embodiment, the processing unit 1 is programmed with an AI model (in particular with the aforementioned first model and / or second model) which, once trained, is used to determine in a precise, reliable, robust, and real-time manner, the aforementioned key moments, for each of the identified driving contexts and to associate reference eye saccade patterns with these key moments. The training data can for example be retrieved from a database centralizing all the data of a fleet of vehicles. The training data can be collected in a variety of driving contexts to ensure robust and precise identification of the key moments. Data collection sessions can be carried out on closed test circuits and on public roads.

[0084] Once trained, the AI ​​model is also used to associate reference characterization data with one or more “critical” key moments. In the aforementioned embodiment, the key moments M2, M3, M4 and M5 are critical moments insofar as they depend directly or indirectly on the cognitive state of the driver.

[0085] According to one embodiment, the reference characterization data associated with these critical moments are as follows: - For M2: data characterizing a “normal” direction of the driver’s gaze, - For M3: data characterizing “normal” reflex reactions of the driver, - For M4: data characterizing “normal” reflective reactions of the driver, - For M5: data characterizing “normal” reactions of the vehicle following the driver’s deliberate reactions.

[0086] A "normal" direction is a direction directed towards the important element of the driving context. This is, for example, the direction of the element ± 3°. In other words, in this normal direction, the driver sees the important element of the context (the person crossing the pedestrian crossing, the car cutting off the right of way). This "normal" direction is hereinafter referred to as the "reference direction".

[0087] By "normal" reactions we mean the reactions that led to risk-free and / or risk-free and / or risk-avoided driving management, in a given driving context. These "normal" reactions are hereinafter referred to as "reference reactions". The reference direction and the reference reactions constitute the reference characterization data.

[0088] The training data for the AI ​​model can notably come from the sensors of the type described previously with reference to the key moments M2, M3, M4 and M5. This training data can for example be retrieved from a database centralizing all the data of a fleet of drivers. The training data can be collected in a variety of driving contexts to ensure robust and accurate identification of reference directions and reference reactions. Data collection sessions can be carried out on closed test circuits and on public roads.

[0089] Each driving context is thus broken down into key moments M1 -M5 and each of these key moments is associated with a reference diagram. The critical key moments M2, M3, M4 and M5 are also associated with the reference directions and the reference reactions mentioned above.

[0090] And as in the example of Figure 3, in real time, after having identified a driving context and having sequenced it into key moments, for each said key moment, the processing unit 1 compares the real-time diagram of the driver with the reference diagram associated with said key moment and measures the extent of the deviations between these two diagrams. For the critical key moments M2, M3, M4 and M5, the processing unit also compares the real-time characterization data of these moments (direction of gaze, the driver's reactions and the vehicle's reactions) with the reference characterization data, and measures the extent of the deviations between these data.

[0091] This sequencing of driving contexts offers several specific advantages in terms of analysis quality and anticipation of responses to driving risks. Processing unit 1 can in fact evaluate more precisely and earlier (at each key moment) the cognitive state of the driver in the face of elements of a driving context likely to impact his decisions and reactions. For example, the analysis of the real-time pattern combined with the analysis of the driver's gaze direction (M2) makes it possible to predict his future actions (M4) and the behavior of the vehicle (M5). Similarly, the analysis of the real-time pattern combined with the analysis of the driver's unconscious reactions (M3) makes it possible to improve the prediction of his conscious reactions (M4) and the behavior of the vehicle (M5).And finally, the analysis of the real-time pattern combined with the analysis of the driver's conscious reactions (M4) further improves the prediction of vehicle behavior (M5). This sequenced analysis ultimately makes it possible to identify a possible exacerbation of identified problems and to predict a contextualized driving risk. This prediction is made possible thanks to the combined measurements: on the one hand, of the magnitude of the deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern associated with said key moment, and on the other hand, of the magnitude of the deviations between the real-time characterization data and the reference characterization data. The sole analysis of the magnitude of the deviations between the real-time characterization data and the reference characterization data does not allow such a prediction to be obtained.

[0092] The system thus offers improved reaction time, as an appropriate instruction can be executed at the end of each key moment in the event of a observed deviation, in order to anticipate any driving risk. In addition, this sequencing allows for better customization of the actions generated, as the instructions can be adjusted according to the variances observed within the different key moments.

[0093] The arrangement of the various elements and / or means and / or steps of the invention, in the embodiments described above, should not be understood as requiring such an arrangement in all implementations. In any event, it will be understood that various modifications may be made to these elements and / or means and / or steps, without departing from the spirit and scope of the invention.

[0094] Further, one or more features disclosed only in one embodiment may be combined with one or more other features disclosed only in another embodiment. Similarly, one or more features disclosed only in one embodiment may be generalized to other embodiments, even if that feature or features are described only in combination with other features.

Claims

Claims

1. [Driving risk assessment system for a driver of a vehicle (V), said system being characterized in that it comprises: - a first set of sensors (Cn, C12, C13) integrated into the vehicle (V), configured to collect data in real time relating to the external environment of the vehicle, - a second set of sensors (C21, C22, C23) integrated into the vehicle (V), configured to collect data relating to a driver-specific eye saccade pattern, - an on-board processing unit (1) programmed with an artificial intelligence algorithm, said unit being configured to: -- analyze the data collected by the first set of sensors to identify a driving context in real time, -- associate at least one reference eye saccade pattern with the identified driving context, -- analyze the data collected by the second set of sensors and perform a measurement of the magnitude of the deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern, -- select one instruction from a plurality of instructions, each of which generates one or more actions, which selection is a function of the extent of the deviations measured, -- executing the selected instruction to generate the corresponding action(s), and wherein the artificial intelligence algorithm is trained using a set of training data collected from drivers in various impaired cognitive states, and placed in a variety of driving contexts, such that the processing unit (1) can associate a so-called "abnormal" reference eye saccade pattern with each identified impaired cognitive state and driving context.

2. System according to claim 1, wherein the processing unit (1) is configured to measure the magnitude of the deviations by determining whether said measured deviations are included in reference value ranges.

3. System according to claim 2, wherein the processing unit (1) is configured to execute the selected instruction only if the measured deviations are included in reference value ranges, for a predetermined time.

4. System according to one of the preceding claims, in which the processing unit (1) is configured to measure the magnitude of the deviations on a combination of characteristics of the eye saccade patterns.

5. System according to one of the preceding claims, in which the artificial intelligence algorithm is trained using a set of training data collected on drivers being in an unaltered cognitive state, and placed in a variety of driving contexts, so that the processing unit (1) can associate a so-called "normal" reference eye saccade pattern with each identified driving context.

6. System according to one of the preceding claims, in which the processing unit (1) is configured to: - detect a correspondence between the eye saccade pattern captured in real time and at least one of the “abnormal” reference eye saccade patterns associated with an altered cognitive state, - if there is a match, select an instruction that is specific to the detected altered cognitive state, - execute the selected instruction to generate the action(s).

7. System according to one of the preceding claims, in which the second set of sensors (C21, C22, C23) is configured to collect data on the following characteristics of the eye saccades: their speed, their amplitude, their frequency, their direction and the duration of fixation between the saccades.

8. System according to one of the preceding claims, in which the instructions are predefined and recorded in a database data (2) connected to the processing unit (1) and / or the instructions are dynamically created by said unit according to the identified driving context.

9. Method for assessing the driving risks of a vehicle driver (V), characterized by the following steps: - collect in real time an initial set of data relating to the vehicle's external environment to identify a driving context in real time, - collect a second set of data relating to a driver-specific eye saccade pattern, - using an artificial intelligence algorithm: -- analyze the first set of data to identify a driving context in real time, -- associate with the identified driving context, at least one reference eye saccade pattern, -- analyze the second data set and perform a measurement of the magnitude of deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern; - select an instruction from a plurality of instructions, each of which allows one or more actions to be generated, which selection is a function of the extent of the deviations measured, - executing the selected instruction to generate the corresponding action(s), said method further comprising a step of training the artificial intelligence algorithm using a set of training data collected on drivers being in various altered cognitive states, and placed in a variety of driving contexts, so as to associate a so-called "abnormal" reference eye saccade pattern with each altered cognitive state and each identified driving context.

10. A method according to claim 9, comprising the steps of: - sequence each driving context identified into a series of key moments, - associate at least one reference eye saccade pattern with each identified key moment, - associate reference characterization data with one or more identified key moments, known as “critical key moments”.

11. A method according to claim 9, comprising the following steps for each critical key moment: - collect in real time a third set of data characterizing said key moment, - analyze in a way that combines the second data set and the third data set and perform: -- on the one hand a measure of the extent of the deviations between the driver-specific eye saccade pattern and the reference eye saccade pattern associated with said key moment and, -- on the other hand a measure of the extent of the deviations between the real-time characterization data and the reference characterization data, - select an instruction from a plurality of instructions, each of which allows one or more actions to be generated, which selection is a function of the magnitudes of the measured deviations, - execute the selected instruction to generate the corresponding action(s).

12. A method according to either of claims 10 or 11, wherein: - a first key moment is characterized by data relating to elements of the vehicle's external environment to be seen and understood by the driver in said driving context, - a second key moment is characterized by characterization data relating to the direction of the driver's gaze in relation to said elements, - a third key moment is characterized by characterization data relating to unconscious reflex reactions of the driver, - a fourth key moment is characterized by characterization data relating to conscious reflective reactions of the driver on the vehicle controls, - a fifth key moment is characterized by characterization data relating to vehicle reactions following these reflected reactions.

13. The method of claim 12, wherein the second key moment, the third key moment, the fourth key moment, and the fifth key moment are critical moments.

14. A method of producing an artificial intelligence algorithm for implementing the method according to claim 9, comprising a step of training said algorithm using a set of training data collected in a variety of driving contexts, which training data is used for the identification of the driving contexts and for the identification of eye saccade patterns in said contexts.

15. The method of claim 14, wherein the training data is collected during driving sessions conducted both on closed test circuits and on public roads.

16. A method according to one of claims 14 or 15, wherein the training data is collected from drivers in an unaltered cognitive state during the driving sessions.

17. A method according to one of claims 14 to 16, wherein the training data is collected from drivers in various altered cognitive states during driving sessions on closed test circuits.

Images