Method for assessing a need to warn a driver of a hazardous situation

Abstract

The invention relates to a method (100) for assisting in the driving of a motor vehicle, the method implementing an assessment of a warning need in order to determine whether or not it is necessary to warn a driver of a hazardous situation, the method comprising the following steps: - determining (101) a line of sight (21) of the driver's gaze; - receiving (102) an image (22) of a scene surrounding the vehicle, and determining (103) the point of intersection (23) with the line of sight; - calculating (104) a two-dimensional Gaussian (24) centered on this point of intersection, the abscissae of the Gaussian corresponding to the coordinates of the points in the image of the scene; - determining (108) a visual attention function (28) by deforming the Gaussian (24), wherein the deformation uses data relating to an overall warning need and intrinsic characteristics of elements of interest in the surrounding scene. - for each element of interest (25i), calculating (109) a degree of attention which depends on a value of the visual attention function (28) at the location of the element of interest, and comparing (110) with a threshold (Th_1) to determine whether or not it is necessary to warn the driver of a hazardous situation.

Description

DESCRIPTION TITLE: Method for assessing the need to alert a driver to a risky situation TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of driver assistance systems implemented in motor vehicles to improve user experience and road safety.

[0002] The invention relates more particularly to a method for assessing the need to alert the driver to a risky situation or not. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] In prior art, we know of various driver assistance functions which detect the presence of a risky situation and then generate an alert intended to warn the driver of a potential danger.

[0004] These alerts are triggered solely by the detection of a risky situation, without considering the driver's state of attention. Consequently, there is a risk to user acceptance, as alerts can be triggered even when the driver has clearly seen the danger. This unnecessary redundancy of information may lead to the deactivation of these alerts.

[0005] The literature presents algorithms that could address this problem, implementing artificial intelligence algorithms trained to discriminate between cases where the driver had already identified the risky situation and cases where the driver had not, thus requiring an alert. One example is the article “Maad: A model and dataset for "attended awareness" in driving”, by Gopinath, et al. (2021).

[0006] One drawback of such algorithms is their enormous computing power requirements, making them incompatible with real-time, on-vehicle operation. Specifically, currently, deploying one of these algorithms in a standard commercial vehicle requires over 10 gigabytes of storage capacity to produce an estimate of the driver's attention with a 300 ms delay. After just 5 minutes of driving, the cumulative delay is already more than 2 minutes compared to the desired time for the estimate. In extreme cases, implementing these algorithms can lead to a system crash requiring a computer reboot.

[0007] One objective of the present invention is therefore to offer an embedded solution that effectively addresses the problem of alert acceptability.

[0008] In particular, one objective of the present invention is to provide an embedded solution enabling real-time operation in a standard commercial motor vehicle, and effectively addressing the problem of alert acceptability. SUMMARY OF THE INVENTION

[0009] This objective is achieved with a method for assisting the driving of a motor vehicle implementing an assessment of the need for an alert, this assessment consisting of determining whether or not it is necessary to alert a driver to a risky situation, the assessment of the need for an alert comprising the following steps: a) Determination of a line of sight of the driver, using at least one image of the driver; b) Reception of an image of a scene surrounding the vehicle; c) Determination of the point of intersection between the image of the scene and the line of sight of the driver; d) Calculation of a two-dimensional Gaussian curve centered on this point of intersection, the abscissas of said Gaussian curve corresponding to the coordinates of the points in the image of the scene; e) Using data from, in particular, external sensors of the vehicle, identification of at least one element of interest in the environment of the vehicle;f) For each element of interest, determination of intrinsic characteristics of the element of interest, including at least its nature and its position in three-dimensional space; g) Using data from at least one sensor in the vehicle, determination of data relating to an overall warning need; h) Determination of a visual attention function resulting, at least, from a deformation of said Gaussian, the deformation using at least the data relating to the overall warning need and the intrinsic characteristics of the element of interest; i) For each element of interest, calculation of a degree of attention, which depends on at least one value of the visual attention function at the abscissa corresponding to the location of the element of interest in the scene image;j) For each element of interest, a comparison between the corresponding level of attention and a predetermined threshold, to determine whether or not it is necessary to alert the driver to a risky situation.

[0010] A point of interest in the vehicle's environment refers to any element in the vehicle's surroundings that could be relevant in determining whether or not a hazardous situation exists. A point of interest may include at least one element such as a pedestrian, cyclist, another vehicle, an animal, an obstacle on the road, a traffic sign, an intersection, a traffic light, road markings, etc.

[0011] The method according to the invention proposes the use of a simple two-dimensional Gaussian function, which is then deformed to adapt it to a given situation at a specific time. This deformation allows for the consideration of the influence of an overall alert need (a function, in particular, of the driver's overall level of attention), as well as the specific characteristics of each element of interest in the scene. It is thus possible to determine, reliably and with minimal computational effort, whether or not it is necessary to alert the driver to a hazardous situation.

[0012] The invention thus addresses the issue of alert acceptability presented in the introduction, while allowing for real-time implementation within at least one computer embedded in a motor vehicle.

[0013] In other words, the invention offers an “intelligent” alert that activates only when the driver has not been paying attention to the danger. The alerts are personalized by taking into account the context-driver interaction, which preserves the acceptability of the proposed driver assistance systems.

[0014] Throughout the text, an alert preferably refers to an HMI alert, that is, an alert implemented by the emission of a visual, audible or even haptic signal, a human-machine interface.

[0015] Preferably, the method according to the invention further includes a step of generating at least one warning signal, when it is determined in step j) that it is necessary to alert the driver to a risky situation.

[0016] The location of the element of interest on the scene image advantageously includes the location of a surface object on the scene image.

[0017] The degree of attention associated with an element of interest can be defined by the average value of the visual attention function on the corresponding surface object.

[0018] Preferably, the data relating to the overall alert requirement should include at least one of the following: - a type of distraction, including a type of visual distraction and / or a type of auditory distraction and / or a type of cognitive distraction; - a degree of concentration of the driver, determined by analysis of at least one image of the driver; - driver's health data; - eye movements of the driver, measured on a series of images of the driver; - the speed of the vehicle.

[0019] The distortion of the Gaussian includes a modification of at least one of its standard deviations, depending on the data relating to the overall need for alerting.

[0020] The deformation of the Gaussian can use at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes at least one value that depends on the intrinsic characteristics of this element of interest; and - extends over only a part of the abscissas of the Gaussian, over an area including the corresponding element of interest.

[0021] The value taken by the local deformation function advantageously depends on at least one parameter among: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight of the driver and an axis passing through the element of interest and through one eye of the driver; - the speed of movement of the element of interest.

[0022] Each local deformation function can take a constant value, and apply to only a part of the Gaussian, associated with abscissas of the element of interest in the scene image.

[0023] Advantageously, the visual attention function is a weighted sum of said distorted Gaussian and at least secondary Gaussian associated with a previous instant.

[0024] Preferably, the assessment of an alert need is implemented, in use, with at least one software application of interest, executed using hardware and software resources shared with at least one additional software application; and the method according to the invention further includes preliminary steps of: determining a current value of available computing capacity, corresponding to the computing capacity available from said hardware and software resources to implement the assessment of an alert need, and comparing it to a predetermined threshold of computing capacity, the assessment of an alert need forming a so-called enhanced assessment and being implemented when said current value of available computing capacity is greater than said predetermined threshold.

[0025] A degree of priority can be assigned to each of the software applications of interest and at least one additional software application, and the current value of available computing capacity depends on whether or not one or more higher priority additional software applications are running.

[0026] When the current value of available computing capacity is less than the predetermined threshold, the process advantageously includes the implementation of a simplified assessment of an alert need, using fewer hardware and software resources than the enhanced assessment.

[0027] The said simplified assessment of a need for an alert may include the following steps: a) Determination of a line of sight for the driver, using at least one image of the driver; P) Using data from external vehicle sensors, identify at least one element of interest in the vehicle's environment and determine its position in three-dimensional space; Y) For each element of interest, calculate an angular deviation between the line of sight of the driver's gaze and an axis passing through the element of interest and through one of the driver's eyes; and 5) Determining whether or not to alert the driver to a risky situation, by comparing said angular deviation with at least a predetermined angle threshold.

[0028] Preferably, the process includes: obtaining risk level information associated with each item of interest; and switching to the enhanced assessment when the risk level associated with at least one item of interest exceeds a predetermined risk level threshold.

[0029] The invention also covers a computer program comprising instructions, executable by a microprocessor or microcontroller, for the implementation of a method according to the invention, when executed by the microprocessor or microcontroller.

[0030] The invention also covers a computer configured to implement the steps of a process according to the invention. BRIEF DESCRIPTION OF THE FIGURES

[0031] The figures are presented for illustrative purposes only and are in no way limiting to the invention.

[0032] [Fig 1] Figure 1 schematically illustrates the steps of an assessment of an alert need in a driving assistance process according to a first embodiment of the invention;

[0033] [Fig 2A] Figure 2A schematically illustrates the two-dimensional Gaussian distribution, centered on the point of intersection between the image of the scene and the driver's line of sight.

[0034] [Fig 2B] Figure 2B schematically illustrates the visual attention function obtained by deformation of the Gaussian illustrated in Figure 2A;

[0035] [Fig 2C] Figure 2C schematically illustrates a superimposition of the visual attention function of Figure 2B, with the elements of interest in the scene image;

[0036] [Fig 3] Figure 3 schematically illustrates a driving assistance method according to a second embodiment of the invention;

[0037] [Fig 4] Figure 4 schematically illustrates the steps of a simplified assessment of an alert need, in the process of Figure 3; and

[0038] [Fig 5] Figure 5 schematically illustrates a system adapted for the implementation of a driving assistance method according to the invention. DETAILED DESCRIPTION

[0039] The invention relates to a method for assisting the driving of a motor vehicle, implementing an alert need assessment which consists of determining whether or not it is necessary to alert a driver to a risky situation, while the driver is driving the motor vehicle.

[0040] Figure 1 schematically illustrates the steps of an assessment of an alert need, implemented in a process 100 according to the invention.

[0041] The process includes the following steps, implemented in real time by at least one on-board computer in the motor vehicle:

[0042] In step 101, the driver's line of sight 21 is determined by analyzing at least one image of the driver, acquired using a first camera preferably located inside the passenger compartment. In other words, the orientation of the line of sight 21 in three-dimensional space is determined by analyzing the image of the driver's eyes.

[0043] In a step 102 that can be implemented in parallel, the computer receives at least one image 22 of the scene surrounding the vehicle. More specifically, this is an image of the scene located in front of the vehicle, acquired by a second camera aimed this time at the side opposite the passenger compartment.

[0044] In step 103, the position of the point of intersection 23 between the image of the scene 22 and the line of sight 21 is determined. This determination is implemented by simple geometric calculations.

[0045] In step 104, a two-dimensional Gaussian function, referenced as 24 in Figure 1, and centered on the intersection point 23, is defined. This Gaussian 24 is illustrated in Figure 2A. The abscissas of the Gaussian 24 correspond to the (x, y) coordinates of the points in the image of scene 22 in a two-dimensional orthonormal coordinate system. The values ​​taken by the Gaussian 24, or ordinates, are defined by a amplitude, a standard deviation o x along the x-axis and a standard deviation o y along the y-axis (preferably with Ox= O X y). The abscissas of the Gaussian 24 are bounded by the dimensions of the image of the scene 22. In other words, the abscissas of the Gaussian 24 do not extend beyond the coordinates of the image of the scene.

[0046] The computer also implements a step 105 of identifying the elements of interest 25i in the vehicle environment, followed by a step 106 of determining, for each of these elements of interest 25i, intrinsic characteristics including at least its nature and its position in three-dimensional space.

[0047] Steps 105 and 106 can be implemented in parallel with one or more of steps 101 to 104 described above.

[0048] Identifying an item of interest 25i in the vehicle environment means simply determining its presence in the vehicle environment.

[0049] Elements of interest 25i refer in particular to obstacles or signs. These may include, for example, a pedestrian, a cyclist, another vehicle, an animal, an obstacle on the road, a traffic sign, an intersection, a traffic light, road markings, etc. This specificity defines the nature of the element of interest 25i.

[0050] The position of the element of interest, in three-dimensional space, subsequently allows it to be positioned relative to the line of sight 21 of the driver's gaze.

[0051] Steps 105 and 106 are implemented using data from external vehicle sensors, such as radar sensors, LiDAR sensors, and possibly visible light sensors (cameras). This external sensor data is acquired in real time by the sensors and analyzed by at least one onboard computer to derive information for detecting an object of interest, and then to determine its nature and position in space. The data from external sensors can be supplemented by map data, attached to map segments and stored in the vehicle and / or stored in the cloud and transferred to the vehicle wirelessly as it travels.

[0052] The calculator also implements a step 107 of determining data relating to an overall alert need.

[0053] Data relating to an overall alert need may include: - a type of distraction, each type of distraction being subsequently associated with a distinct level of danger. A type of distraction includes, for example, visual distraction (e.g., it is detected that the driver's line of sight is not directed towards the road, or that the driver is looking at a phone screen), and / or auditory distraction (e.g., it is detected that the radio is on in the vehicle, or that there is a conversation in the vehicle), and / or cognitive distraction (e.g., it is detected that there is an ongoing discussion within the passenger compartment or (by telephone); - a degree of concentration of the driver, determined by analysis of at least one image of the driver (based on facial expressions, we can determine a degree of concentration, or in other words a degree of fatigue); - driver health data, including the existence or absence of vision problems (this data is associated, for example, with a digital profile of the driver); - eye movements of the driver (preferably a type of movement with an execution time, the type of movement including for example fixation, saccades, and / or blinks), measured on a series of images of the driver; - vehicle speed (the faster the vehicle is moving, the higher the risk of not noticing a risky situation in time).

[0054] Advantageously, data on the type of distraction and the degree of driver concentration are combined together to define the value of a binary indicator of driver concentration.

[0055] In a later step 108, the calculator calculates a visual attention function 28 (also illustrated in figure 2B).

[0056] The visual attention function 28, like the Gaussian 24, has its abscissas which correspond to the coordinates (x; y) of the points in the image of scene 22.

[0057] The visual attention function 28 is derived, at least, from a distortion of the Gaussian 24.

[0058] The distortion of the Gaussian 24 uses, at least, the data relating to the overall need for alerting and the intrinsic characteristics of the element of interest.

[0059] Preferably, the deformation includes a modification of the standard deviation values. x , o y of the Gaussian 24, depending on the data relating to the overall need for alerting.

[0060] To do this, we can define a function relating a value of o x and o ywith the respective value of each of the data points related to the overall alerting need as listed above. This function will be determined using business data, assigning a greater or lesser weight to each data type and each category within a data type. The weights will be defined based on business knowledge regarding the influence of each data type and each category within a data type on the risk that a hazardous situation may not have been identified by the driver. The weights may vary depending on typical driving conditions (for example, parameters are assigned a greater or lesser weight for city or rural driving).

[0061] The modification of the standard deviation values ​​o x , o yThe spread of Gaussian 24 is reflected by a more or less pronounced broadening of the initial Gaussian 24. This corresponds to the main peak 281 of the visual attention function 28 illustrated in Figure 2B (here narrower than the Gaussian 24 illustrated in Figure 2A).

[0062] The deformation may further include the use of at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes one or more values ​​related to the intrinsic characteristics of this element of interest; and - extends over only a part of the abscissas (x, y) of the Gaussian coordinate system 24, over an area including the abscissas of the corresponding element of interest.

[0063] Each local function is, for example, multiplied or added using the Gaussian function 24 after modification of its standard deviations. x , o y .

[0064] Each use of such a local deformation function results in a local deformation of the Gaussian 24. This corresponds to a region 282 on the visual attention function 28 illustrated in Figure 2B.

[0065] For each element of interest considered, the local deformation function extends over only a portion of the x, y coordinates of the Gaussian curve 24, within a region that includes the coordinates of the element of interest's location in the scene image. Everywhere else, the local deformation function is undefined, or takes a value of 0 or 1, which does not modify the function to which it is added or multiplied, respectively. In other words, the local deformation function applies to only a portion of a Gaussian curve (the Gaussian curve 24, possibly deformed by modifying its standard deviations), at x-coordinates that include those of the element of interest in the scene image.

[0066] Each local deformation function is associated with one of the identified elements of interest and takes at least one value that depends on one or more intrinsic characteristics of that element of interest. Specifically, each local deformation function is associated with one of the elements of interest and takes at least one value that depends on at least one of the following intrinsic characteristics: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight 21 of the driver's gaze, and an axis passing through the element of interest and through one of the driver's eyes. (If applicable, we simply consider whether or not it belongs to one or more cones of predetermined size.) - a speed of movement of the element of interest (movement relative to the vehicle receiving the driver, or movement in the terrestrial frame of reference).

[0067] Different weights can be assigned to each of the intrinsic characteristics used to determine a value taken by the local deformation function, depending on business knowledge and possibly current driving conditions.

[0068] Furthermore, the influence of each particular combination of intrinsic characteristics can be taken into account. For example, it is known that central vision allows us to distinguish objects precisely, while in peripheral vision humans are only attentive to moving objects.

[0069] The local deformation function can take a constant value over an entire region with abscissas (x, y) over which it extends. Alternatively, the local deformation function can, for example, take a decreasing value from the center to the edges of said region.

[0070] Advantageously, the local deformation function extends entirely and exclusively to the (x, y) coordinates of a surface object associated with the corresponding element of interest. This surface object can have a predetermined shape (for example, a square or a rectangle). Alternatively, this surface object can be bounded by the outline of the image of the element of interest on the scene image. In any case, the location of this surface object is centered on the image of the element of interest in the scene image.

[0071] In the example illustrated in Figure 2B, region 282 extends over a square surface, represented by a dashed line. It corresponds to an element of interest associated with a square surface object.

[0072] It is understood that not all the elements of interest 25i visible in the scene image are necessarily associated with a respective local deformation function. For each element of interest, we can: - calculate the value of a risk coefficient based on one or more intrinsic characteristics of this element of interest; - compare this value to a predetermined threshold; and - only define a local deformation function if said value is greater than the threshold.

[0073] The distortion can also implement a forgetting factor, representative of the duration of memorization, by the driver, of information that he has previously visualized.

[0074] In this case, the visual attention function is a weighted sum of the distorted Gaussian 24 as described above, with at least one secondary Gaussian associated with a previous instant.

[0075] The secondary Gaussian corresponds to the 24th Gaussian, associated with a previous instant and after deformation by modification of its standard deviation.

[0076] The weighting coefficient, or forgetting factor, depends on the time elapsed between the current moment and the preceding moment. It can also take into account driver behavior, such as eye movements. Indeed, a driver will retain more information during a fixation than during saccadic viewing. The forgetting factor can also depend on whether the element of interest is fixed or dynamic, depending on its nature. For example, an intangible element of interest, such as a road sign, can be viewed quickly, only once, and yet still be remembered because it is not likely to change over time.

[0077] In any case, the calculation of the visual attention function includes a final normalization step.

[0078] For example, but not limited to, the visual attention function (before normalization) is defined by the following equation:

[0079] [Math 1]

[0080] with :

[0081] attention_map is the visual attention function;

[0082] Aj(Wi, w *Hh,i) the Gaussian deformed by modification of standard deviation, associated with the instant of index i;

[0083] N is the number of Gaussians considered (Gaussians associated with previous times and the Gaussian associated with the current time considered). This number N corresponds to a direction number of the line of sight, taken into account in the calculation of the visual attention function.

[0084] Ai has a weighting coefficient, fixed empirically, and representative of the duration of memorization, by the driver, of information that he visualized at time i.

[0085] [Math 2]

[0086] [Math 3]

[0087] p, 5 of the coefficients depending on eye movement, with fixation < saccade (same for 5);

[0088] (x0,y0) the coordinates of the point of intersection 23 between the line of sight 21 and the image of the scene;

[0089] o x =Oy=o the initial standard deviation of the Gaussian 24 (fixed empirically);

[0090] w the desired width of a range of the visual attention function along the x-axis of the image frame of the scene;

[0091] h the desired height of an extent of the visual attention function along the y-axis of the image frame of the scene;

[0092] OjAj is the local deformation function associated with the element of interest of index j;

[0093] K is the number of elements of interest considered;

[0094] Qj is a weighting coefficient based on the intrinsic characteristics of the object (its nature, its speed, its angular deviation from the current position of the line of sight 21);

[0095] Aj is a positioning matrix of the surface object associated with the element of interest. Aj is a binary matrix with the same dimensions as the scene image and the x-axis of the Gaussian distributions. Each component of the matrix Aj takes the value 0 or 1 depending on the presence or absence of the element of interest at the corresponding x-coordinates (and possibly depending on the speed of this element of interest, for example, to consider only moving objects).

[0096] Next, a calculation step 109 is implemented, for each element of interest 25i, of a degree of attention which depends on at least one value of the visual attention function 28 at the abscissas corresponding to the location of the element of interest in the image of the scene.

[0097] The location of the element of interest in the scene image corresponds to the position and dimensions of a surface object as described above.

[0098] Step 109 preferably consists of calculating, for each element of interest 25i, the average value taken by the visual attention function 28 at the coordinates of the corresponding surface object (the abscissas of the visual attention function 28 and the coordinates of the surface object being defined in the same coordinate system (x,y)). Since the visual attention function 28 is a normalized function, a score between 0 and 1 is thus obtained for each element of interest.

[0099] Figure 2C schematically represents, superimposed, the visual attention function 28 and the surface objects 29i, each associated with one of the respective elements of interest.

[0100] Thanks to the deformation steps of the Gaussian 24, the average value taken by the visual attention function 28 at the coordinates of a surface object 29i is a function of: - data relating to the overall need for alertness (type of distraction, degree of concentration, driver health data, eye movements, vehicle speed), which determines the extent to which the driver pays attention only to what is directly in their line of sight, or also to what is further to the periphery; and - intrinsic characteristics of the corresponding element of interest (in particular its nature, its speed of movement and its angular deviation from the line of sight).

[0101] Thus, the degree of attention associated with each element of interest is finely determined, and takes into account a current situation in its entirety by considering a plurality of factors associated with the behavior of the driver and the external environment.

[0102] This is followed by a comparison step (step 110) for each element of interest, between the corresponding level of attention and a predetermined threshold (Th_1), to determine whether or not it is necessary to alert the driver to a risky situation. Thanks to the precise determination of the level of attention, it is determined accurately, for each element of interest, whether it is necessary to alert the driver to an associated risk, or whether such an alert would instead create redundancy of information likely to lead the driver to deactivate warning functions.

[0103] Steps 101 to 110 determine whether or not the driver needs to be alerted to a risky situation. These steps are advantageously followed by step 111, which generates at least one warning signal, when a need to alert the driver is determined in step 110.

[0104] The warning signal can be audible and / or visual (for example, via a light strip in the passenger compartment or the head-up display) and / or haptic (for example, a vibration in the steering wheel and / or accelerator pedal). It is generated using at least one human-machine interface (HMI). It can be a signal oriented in the same direction as the point of interest for which a warning is to be issued. For example, the vehicle has an LED strip under the windshield, and an LED located at the same height as this point of interest is illuminated.

[0105] Next, with reference to figure 3, a driving assistance method is described according to a second embodiment of the invention.

[0106] In this embodiment, steps 101 to 110 of Figure 1 are implemented, in use, with at least one software application of interest, running using hardware and software resources shared with at least one additional software application.

[0107] At least one additional software application includes, for example, a trajectory prediction application and / or a voice analysis application.

[0108] In the advantageous embodiment illustrated in Figure 3, it is proposed that these steps 101 to 110 no longer form a function to be called continuously, but a function to be used in a reasoned manner with respect to the driving context, calling upon only the computers necessary at the required time.

[0109] The method 300 according to the invention then comprises the following steps, implemented by at least one computer embedded in the vehicle, this computer corresponding to the shared hardware and software resources mentioned above.

[0110] In a step 30, a current value of available computing capacity, Rc, is determined, corresponding to the computing capacity available from said hardware and software resources, and at the time considered, to implement the assessment of an alert need as described with reference to Figure 1.

[0111] In practice, a respective priority level can be assigned to each of the software applications of interest and the additional software applications. The Rc value then corresponds to the difference between the total computing capacity of the hardware and software resources and the computing capacity required to execute the additional software applications with higher priority than the software application of interest, which are currently running or about to be executed at the given time.

[0112] Determining the computing power needed to run the higher priority additional software applications can be done using tools known to those skilled in the art.

[0113] In step 31, the value Rc is compared to a predetermined computing capacity threshold, referenced Th_2. The predetermined threshold Th_2 can be zero, or a positive value representing a tolerance or safety threshold.

[0114] When Rc is greater than Th_2, this means that the computing capacity available at the time considered allows the evaluation of the need for alerting to be implemented as described with reference to Figure 1. In this case, a step 32b of the evaluation of the need for alerting is implemented as described with reference to Figure 1, then called the improved evaluation.

[0115] When Rc is less than Th_2, it means that the available computing power at the time in question is insufficient to implement the enhanced assessment. In this case, a simplified assessment of the need for an alert is implemented (step 32a). The simplified assessment uses fewer hardware and software resources than the enhanced assessment. It also determines whether or not it is necessary to alert a driver to a risky situation, but with less precision in assessing the need.

[0116] Optionally, steps 33 and 34 can be implemented, consisting of: - calculate a risk level associated with each element of interest, this calculation being carried out in parallel with the simplified assessment 32a (step 33); and - switch to the improved assessment of step 32b, when the risk level associated with at least one element of interest exceeds a predetermined risk level threshold Th_3.

[0117] It is therefore possible, when a potential danger is identified, to force the execution of the enhanced assessment in order to offer the driver the most relevant alerts possible.

[0118] For example, moderate priority levels can be defined, corresponding to additional software applications whose execution can be temporarily suspended when a switch to the improved evaluation is required.

[0119] The calculation of the risk level associated with each point of interest is advantageously implemented by a dedicated application, allowing for different risk levels related to potential collisions, overspeed, or even hidden points of interest (pedestrian crossings, etc.). The risk related to the points of interest in the scene is predicted over a period of 6 seconds to 2 seconds before the estimated potential collision.

[0120] Figure 4 illustrates an example of a simplified evaluation procedure that can be implemented in step 32a, using at least one calculator.

[0121] In a step 401, the driver's line of sight 21 is determined using at least one image of the driver.

[0122] In a step 402 (which can be implemented in parallel with step 401), the elements of interest 25i in the vehicle environment are identified, and their respective positions in three-dimensional space are determined.

[0123] This step 402 uses data from external sensors (radar and / or LiDAR and / or camera), and possibly map data. Depending on available computing power, a greater or lesser amount of external sensor data and / or map data can be used to implement this step 402.

[0124] In a later step 403, for each element of interest 25i, the angular deviation Ei is calculated between the line of sight 21 of the driver's gaze, and an axis 41 i passing through the element of interest 25i and through an eye of the driver.

[0125] In step 404, each angular deviation Ei is compared to at least one predetermined angular deviation threshold Th_4. Based on the result of the comparison, it is determined whether or not the driver needs to be alerted to a risky situation.

[0126] In other words, we can define a cone of revolution centered on the line of sight 21 and with apex angle Th_4. Depending on whether the element of interest 25i is located inside or outside the cone, it is considered to be seen or not by the driver. When the element of interest is considered to be seen by the driver, it is assumed that there is no need for an alert. When the point of interest is deemed not to be seen by the driver, it is considered necessary to alert the driver. In this case, before any alert is issued, the system can switch to the enhanced assessment as illustrated with reference to Figure 3.

[0127] In a preferred embodiment, two threshold values ​​for angular difference are defined, corresponding to two cones of vision. The narrower cone defines a zone of maximum driver attention. For example, it has an apex angle between 2° and 10°. The wider cone defines, where it does not overlap the narrower cone, A moderate attention zone. For example, it has a vertex angle between 50° and 70°. The rest of the scene falls within a reduced attention zone for the driver. Each zone is associated with a respective attention level. Depending on the zone in which the element of interest is located, it is assigned the corresponding attention level. Based on the attention level associated with the element of interest, it is determined whether or not the driver needs to be alerted.

[0128] In some variations, more than two concentric cones are used, together defining a plurality of zones each associated with a respective value of degree of attention.

[0129] According to other variants, the simplified assessment also uses intrinsic characteristics of the element of interest, in particular its nature and / or speed, to determine whether or not it is necessary to alert the driver.

[0130] According to other variants, the simplified assessment uses a three-dimensional object representing the element of interest, and calculates an average risk degree value on the three-dimensional object in order to determine whether or not it is necessary to alert the driver.

[0131] In any case, the simplified evaluation uses less sensor data than the enhanced evaluation. In particular, it does not use the data stream from the camera aimed at the scene in front of the vehicle. Thus, the computing resources required for its implementation are significantly lower.

[0132] In any case, the choice between a simplified or enhanced evaluation always allows for assessing the need to alert the driver, regardless of available computing power. The issue of limiting redundancy in alerts remains a priority.

[0133] Finally, with reference to figure 5 and in a schematic manner, a system 50 adapted to the implementation of a driving assistance method according to the invention is illustrated.

[0134] The system 50 includes a set of external sensors 51 from the vehicle, including radars, LIDAR and cameras.

[0135] External sensors 51 may include: - an interior camera filming the driver's face. This camera allows the driver's gaze direction to be seen during driving. Alternatively, the camera can be replaced by a wearable eye tracker; - a contextual camera filming the scene at the front of the vehicle; - at least one radar and / or LiDAR, possibly integrated into a sensor fusion system that allows the identification of obstacles present in the driving scene as well as all information concerning them (their speed, their position relative to the vehicle) of the driver, their nature, etc). These objects are usually processed by dedicated algorithms which make it possible to obtain a representation of the world surrounding the vehicle.

[0136] System 50 also includes a management module (or platform) 52, configured to: - determine information about available computing capacity; - to implement steps 30 and 31 of the process according to the invention, to determine whether a simplified or improved assessment of the need for alerting should be implemented; and - manage data streams from external sensors, to select the sensor data useful or not, at each moment, during the implementation of the invention.

[0137] System 50 also includes a set 53 of at least one computer, configured to implement the steps for simplified or improved assessment of the need for alerting.

[0138] The system 50 also includes at least one human-machine interface 54, for the emission of at least one audible and / or visual and / or haptic warning signal, controlled by the computer 53.

[0139] Block 55 symbolizes a level of risk, or need to alert the driver, which acts both on the human-machine interface 54 which will be used or not, and on the data flow management module 52 if the need to switch from a simplified assessment to an improved assessment of the need for alert is not determined.

[0140] The invention is not limited to the examples described above, and covers variants with other external sensors, other intrinsic data of the element of interest, and / or other data relating to an overall need for alerting.

[0141] The invention finds a particularly advantageous application in the field of Advanced Driver Assistance Systems (ADAS), but also in the field of autonomous vehicles, for example to control a transfer of commands to the driver when it is determined that it is necessary to alert the driver to a risky situation.

Claims

DEMANDS [Claim 1] A method (100) for assisting the driving of a motor vehicle implementing an assessment of a need for an alert, this assessment consisting of determining whether or not it is necessary to alert a driver to a risky situation, the method being characterized in that the assessment of a need for an alert comprises the following steps: a) Determination (101) of a line of sight (21) of the driver's gaze, using at least one image of the driver; b) Reception (102) of an image (22) of a scene surrounding the vehicle; c) Determination (103) of the point of intersection (23) between the image of the scene and the line of sight of the driver's gaze; d) Calculation (104) of a two-dimensional Gaussian (24) centered on this point of intersection, the abscissas of said Gaussian corresponding to the coordinates of the points of the image of the scene;e) Using data from, in particular, external sensors (51) of the vehicle, identification (105) of at least one element of interest (25i) in the environment of the vehicle; f) For each element of interest (25i), determination (106) of intrinsic characteristics of the element of interest, including at least its nature and its position in three-dimensional space; g) Using data from at least one sensor in the vehicle, determination (107) of data relating to an overall warning need; h) Determination (108) of a visual attention function (28) resulting, at least, from a deformation of said Gaussian (24), the deformation using at least the data relating to the overall warning need and the intrinsic characteristics of the element of interest;i) For each element of interest (25i), calculation (109) of a degree of attention, which depends on at least one value of the visual attention function (28) at the abscissa corresponding to the location of the element of interest in the scene image; j) For each element of interest (25i), comparison (110) between the corresponding degree of attention and a predetermined threshold (Th_1), to determine whether or not it is necessary to alert the driver to a risky situation. [Claim 2] Method (100) according to claim 1, further comprising a step (111) of generating at least one warning signal, when it is determined in step j) that it is necessary to alert the driver to a hazardous situation. [Claim 3] Method (100) according to claim 1 or 2, wherein the location of the element of interest on the scene image includes the location of a surface object (29i) on the scene image. [Claim 4] Method (100) according to claim 3, wherein the degree of attention associated with an element of interest (25i) is defined by the average value of the visual attention function (28) on the corresponding surface object (29i). [Claim 5] A method (100) according to any one of the preceding claims, wherein the data relating to the overall alert requirement includes at least one of the following: - a type of distraction, including a type of visual distraction and / or a type of auditory distraction and / or a type of cognitive distraction; - a degree of concentration of the driver, determined by analysis of at least one image of the driver; - driver's health data; - eye movements of the driver, measured on a series of images of the driver; - the speed of the vehicle. [Claim 6] Method (100) according to any one of the preceding claims, wherein the deformation (108) of the Gaussian includes a modification of at least one of its standard deviations, depending on the data relating to the overall warning need. [Claim 7] A method (100) according to any one of the preceding claims, wherein the deformation (108) of the Gaussian uses at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes at least one value that depends on the intrinsic characteristics of this element of interest; and - extends over only a part of the abscissas of the Gaussian, over an area including the corresponding element of interest. [Claim 8] A method (100) according to claim 7, wherein at least one value taken by the local deformation function depends on at least one parameter among: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight (21) of the driver's gaze and an axis passing through the element of interest and through one eye of the driver; - the speed of movement of the element of interest. [Claim 9] Method (100) according to claim 7 or 8, wherein each local deformation function takes a constant value, and applies to only a part of the Gaussian, associated with abscissas of the element of interest (25i) in the scene image. [Claim 10] Method (100) according to any one of claims 1 to 9, wherein the visual attention function (28) is a weighted sum of said distorted Gaussian and at least secondary Gaussian associated with a previous instant. [Claim 11] A method (300) according to any one of claims 1 to 10, wherein: the assessment of an alert need is implemented, in use, with the aid of at least one software application of interest, executed using hardware and software resources shared with at least one additional software application; and the method further comprises preliminary steps of: determining (30) a current value (Rc) of available computing capacity, corresponding to the computing capacity available from said hardware and software resources to implement the assessment of an alert need, and comparing (31) to a predetermined threshold (Th_2) of computing capacity, the assessment of an alert need forming an enhanced assessment and being implemented when said current value of available computing capacity is greater than said predetermined threshold. [Claim 12] Method (300) according to claim 11, wherein a degree of priority is assigned to each of the software application of interest and at least one additional software application, and the current value (Rc) of available computing capacity is a function of whether or not one or more higher priority additional software applications are running. [Claim 13] Method (300) according to claim 11 or 12, comprising, when the current value (Rc) of available computing capacity is less than said predetermined threshold, the implementation (32a) of a simplified assessment of an alert need, using fewer hardware and software resources than the enhanced assessment. [Claim 14] Method (300) according to claim 13, wherein said simplified assessment of a need for alerting comprises the following steps: a) Determination (401) of a sighting axis (21) of the driver's gaze, using at least one image of the driver; P) Using data from external vehicle sensors, identification (402) of at least one element of interest (25i) in the vehicle environment and determination of its position in three-dimensional space; Y) For each element of interest (25i), calculation (403) of an angular deviation (Ei) between the line of sight (21) of the driver's gaze, and an axis (41i) passing through the element of interest and through one eye of the driver; and 5) Determination (404) of the need to alert the driver about a risky situation, by comparison of said angular deviation with at least a predetermined angle threshold. [Claim 15] A method (300) according to claim 13 or 14, comprising: obtaining (33) risk level information associated with each element of interest; and switching (34) to the enhanced assessment (32b) when the risk level associated with at least one element of interest exceeds a predetermined risk level threshold (Th_3). [Claim 16] Computer program comprising instructions, executable by a microprocessor or microcontroller, for the implementation of a method (100; 300) according to any one of claims 1 to 15, when executed by the microprocessor or microcontroller. [Claim 17] Calculator configured to implement the steps of a process (100; 300) according to any one of claims 1 to 15.

Images