Method for assessing the need to alert a driver to a risky situation

Abstract

Method for assessing the need to alert a driver to a risky situation. Method (100) for assisting the driving of a motor vehicle implementing an assessment of the need for an alert to determine whether or not it is necessary to alert a driver to a risky situation, comprising the following steps: - Determination (101) of a line of sight (21) of the driver; - Reception (102) of an image (22) of a scene surrounding the vehicle, and determination (103) of the point of intersection (23) with the line of sight; - Calculation (104) of a two-dimensional Gaussian (24) centered on this point of intersection, the abscissas of the Gaussian corresponding to the coordinates of the points in the image of the scene; - Determination (108) of a visual attention function (28) by deformation of said Gaussian (24), using data relating to an overall alerting need, and intrinsic characteristics of elements of interest on the surrounding scene.- For each element of interest (25i), calculation (109) of a degree of attention which depends on a value of the visual attention function (28) at the location of the element of interest, and comparison (110) with a threshold (Th_1) to determine whether or not it is necessary to alert the driver to a risky situation. Figure 1.

Description

Title of the invention: Method for assessing the need to alert a driver to a risky situation. TECHNICAL FIELD OF THE INVENTION

[0001] The technical field of the invention is that of driver assistance methods implemented in motor vehicles in order to improve the user experience and road safety.

[0002] The invention relates more particularly to a method for evaluating the need to alert the driver to a risky situation or not. TECHNOLOGICAL BACKGROUND OF THE INVENTION

[0003] Various driver assistance functions are known in the prior art, which detect the presence of a risky situation and then generate an alert intended to warn the driver of a potential danger.

[0004] These alerts are issued solely upon detection of a risky situation, without considering the driver's state of attention. Consequently, there is a risk to user acceptance, as alerts may be triggered even when the driver has clearly seen the danger. This unnecessary redundancy of information may therefore lead to the deactivation of these alerts.

[0005] Algorithms presented in the literature as being able to address this problem can be found, implementing artificial intelligence algorithms trained to discriminate between cases in which the driver had already identified the risky situation and cases in which the driver had not identified the risky situation, thus requiring the issuance of an alert. For example, see the article "Maad: A model and dataset for "attended awareness" in driving", by Gopinath, et al. (2021).

[0006] One drawback of such algorithms is that they require enormous computing power, making them incompatible with real-time, on-vehicle operation. Specifically, currently, deploying one of these algorithms in a standard commercial vehicle requires more than 10 gigabytes of storage capacity to produce an estimate of the driver's attention with a 300 ms delay. After 5 minutes of driving, the cumulative delay is already more than 2 minutes relative to the time for which an estimate is desired. In extreme cases, implementing these algorithms results in a system crash requiring a computer restart.

[0007] An objective of the present invention is therefore to propose an embedded solution that effectively addresses the problem of the acceptability of alerts.

[0008] In particular, an objective of the present invention is to provide an embedded solution enabling real-time operation in a standard commercial motor vehicle, and effectively addressing the problem of alert acceptability. Summary of the invention

[0009] This objective is achieved with a method for assisting the driving of a motor vehicle implementing an assessment of the need for an alert, this assessment consisting of determining whether or not it is necessary to alert a driver to a risky situation, the assessment of the need for an alert comprising the following steps: a) Determination of a line of sight for the driver, using at least one image of the driver; b) Reception of an image of a scene surrounding the vehicle, c) Determination of the point of intersection between the image of the scene and the line of sight of the driver's gaze; d) Calculation of a two-dimensional Gaussian centered on this point of intersection, the abscissas of said Gaussian corresponding to the coordinates of the points of the image of the scene; e) Using data from external vehicle sensors, identify at least one element of interest in the vehicle's environment; f) For each element of interest, determination of intrinsic characteristics of the element of interest, including at least its nature and its position in three-dimensional space; (g) Using data from at least one sensor in the vehicle, determination of data relating to an overall warning need; (h) Determination of a visual attention function resulting, at least, from a deformation of said Gaussian, the deformation using at least the data relating to the overall need for alerting and the intrinsic characteristics of the element of interest; i) For each element of interest, calculation of a degree of attention, which depends on at least one value of the visual attention function at the abscissas corresponding to the location of the element of interest in the image of the scene; j) For each element of interest, comparison between the corresponding degree of attention and a predetermined threshold, to determine whether or not it is necessary to alert the driver to a risky situation.

[0010] An element of interest in the vehicle's environment means any element in the scene surrounding the vehicle that may be relevant in determining whether or not a hazardous situation exists. The element of interest may include at least one element such as a pedestrian, a cyclist, another vehicle, an animal, an obstacle on the road, a traffic sign, an intersection, a traffic light, road markings, etc.

[0011] The method according to the invention proposes the use of a simple two-dimensional Gaussian function, which is deformed to adapt it to a situation at a given instant. This deformation makes it possible to take into account the influence of an overall need for alerting (a function, in particular, of the driver's overall level of attention), as well as the specific characteristics of each element of interest in the scene. It is thus possible to determine, reliably and with minimal computational effort, whether or not it is necessary to alert the driver to a risky situation.

[0012] The invention thus addresses the issue of the acceptability of alerts presented in the introduction, while allowing for real-time implementation within at least one computer embedded in a motor vehicle.

[0013] In other words, the invention proposes an “intelligent” alert that activates only when the driver has not been attentive to the danger. The alerts are personalized by taking into account the context-driver interaction, which preserves the acceptability of the proposed driver assistance systems.

[0014] Throughout the text, an alert preferably refers to an HMI alert, that is to say, an alert implemented by the emission of a visual, audible or even haptic signal, a human-machine interface.

[0015] Preferably, the method according to the invention further includes a step of generating at least one warning signal, when it is determined in step j) that it is necessary to alert the driver to a risky situation.

[0016] The location of the element of interest on the scene image advantageously includes the location of a surface object on the scene image.

[0017] The degree of attention associated with an element of interest can be defined by the average value of the visual attention function on the corresponding surface object.

[0018] Preferably, the data relating to the overall alert requirement include at least one of the following: - a type of distraction, including a type of visual distraction and / or a type of auditory distraction and / or a type of cognitive distraction; - a degree of concentration of the driver, determined by analysis of at least one image of the driver; - driver's health data; - eye movements of the driver, measured on a series of images of the driver; - the speed of the vehicle.

[0019] The deformation of the Gaussian includes a modification of at least one of its standard deviations, depending on the data relating to the overall need for alerting.

[0020] The deformation of the Gaussian can use at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes at least one value that depends on the intrinsic characteristics of this element of interest; and - extends over only a part of the abscissas of the Gaussian, over an area including the corresponding element of interest.

[0021] The at least one value taken by the local deformation function advantageously depends on at least one parameter among: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight of the driver and an axis passing through the element of interest and through one eye of the driver; - the speed of movement of the element of interest.

[0022] Each local deformation function can take a constant value, and apply to only a part of the Gaussian, associated with abscissas of the element of interest in the image of the scene.

[0023] Advantageously, the visual attention function is a weighted sum of said distorted Gaussian and at least secondary Gaussian associated with a previous instant.

[0024] Preferably, the assessment of an alert need is implemented, in use, with at least one relevant software application, running using hardware and software resources shared with at least one additional software application; and The method according to the invention further comprises preliminary steps of: determining a current value of available computing capacity, corresponding to the computing capacity available from said hardware and software resources to implement the assessment of an alert need, and comparison to a predetermined computing capacity threshold, the assessment of an alert need forming an enhanced assessment and being implemented when said current value of available computing capacity is greater than said predetermined threshold.

[0025] A priority level can be assigned to each of the software applications of interest and at least one additional software application, and the current value The available computing capacity depends on whether or not one or more additional, higher-priority software applications are running.

[0026] When the current value of available computing capacity is less than said predetermined threshold, the method advantageously includes the implementation of a simplified assessment of an alert need, using fewer hardware and software resources than the enhanced assessment.

[0027] Said simplified assessment of an alert need may include the following steps: a) Determination of a line of sight for the driver, using at least one image of the driver; |3) Using data from external vehicle sensors, identification of at least one element of interest in the vehicle's environment and determination of its position in three-dimensional space; (y) For each feature of interest, calculate an angular deviation between the driver's line of sight and an axis passing through the feature of interest and one of the driver's eyes; and o) Determining whether or not to alert the driver to a risky situation, by comparing said angular deviation with at least a predetermined angle threshold.

[0028] Preferably, the process comprises: obtaining information on the level of risk associated with each item of interest; and a switch to improved assessment when the risk level associated with at least one item of interest exceeds a predetermined risk level threshold.

[0029] The invention also covers a computer program comprising instructions, executable by a microprocessor or a microcontroller, for the implementation of a method according to the invention, when executed by the microprocessor or the microcontroller.

[0030] The invention also covers a computer configured to implement the steps of a process according to the invention. BRIEF DESCRIPTION OF THE FIGURES

[0031] The figures are presented for illustrative purposes only and are in no way limiting of the invention.

[0032] [Fig.1] Fig.1 schematically illustrates the steps of an assessment of an alert need in a driving assistance process according to a first embodiment of the invention;

[0033] [Fig. 2A] [Fig. 2A] schematically illustrates the two-dimensional Gaussian distribution, centered on the point of intersection between the image of the scene and the driver's line of sight.

[0034] [Fig.2B] [Fig.2B] schematically illustrates the visual attention function obtained by deformation of the Gaussian illustrated in [Fig.2A];

[0035] [Fig.2C] [Fig.2C] schematically illustrates a superimposition of the visual attention function of [Fig.2B], with the elements of interest in the scene image;

[0036] [Fig.3] Fig.3 schematically illustrates a driving assistance method according to a second embodiment of the invention;

[0037] [Fig.4] [Fig.4] schematically illustrates the steps of a simplified assessment of an alert need, in the process of [Fig.3]; and

[0038] [Fig.5] Fig.5 schematically illustrates a system adapted to the implementation of a driving assistance method according to the invention. DETAILED DESCRIPTION

[0039] The invention relates to a method for assisting the driving of a motor vehicle, implementing an alert need assessment which consists of determining whether or not it is necessary to alert a driver to a risky situation, while the driver is driving the motor vehicle.

[0040] Fig. 1 schematically illustrates the steps of an assessment of an alert need, implemented in a process 100 according to the invention.

[0041] The method comprises the following steps, implemented in real time by at least one computer embedded in the motor vehicle:

[0042] In step 101, the driver's line of sight 21 is determined by analyzing at least one image of the driver acquired using a first camera preferably located inside the passenger compartment. In other words, the orientation of the line of sight 21 in three-dimensional space is determined by analyzing the image of the driver's eyes.

[0043] In a step 102 that can be implemented in parallel, the computer receives at least one image 22 of the scene surrounding the vehicle. More particularly, this is an image of the scene located in front of the vehicle, acquired by a second camera aimed this time at the side opposite the passenger compartment.

[0044] In a step 103, the position of the point of intersection 23 between the image of the scene 22 and the line of sight 21 is determined. This determination is carried out by simple geometric calculations.

[0045] In step 104, a two-dimensional Gaussian function is defined, referenced 24 in [Fig. 1], and centered on the point of intersection 23. This Gaussian 24 is illustrated in [Fig. 2A]. The abscissas of the Gaussian 24 correspond to the coordinates (x; y) of the points in the image of scene 22 in a two-dimensional orthonormal coordinate system. The values ​​taken by the Gaussian 24, or ordinates, are defined by an amplitude, a standard deviation ox along the x-axis and a standard deviation ovsc along the y-axis (with preferably ox = oxy).

[0046] The abscissas of the Gaussian 24 are bounded by the dimensions of the image of the scene 22. In other words, the abscissas of the Gaussian 24 do not extend beyond the coordinates of the image of the scene.

[0047] The computer also implements a step 105 of identifying the elements of interest 25i in the environment of the vehicle, followed by a step 106 of determining, for each of these elements of interest 25i, intrinsic characteristics including at least its nature and its position in three-dimensional space.

[0048] Steps 105 and 106 can be implemented in parallel with one or more of steps 101 to 104 described above.

[0049] The identification of an element of interest 25i in the vehicle environment refers to the simple determination of its presence in the vehicle environment.

[0050] Elements of interest 25i refer in particular to obstacles or signs. These may include, for example, a pedestrian, a cyclist, another vehicle, an animal, an obstacle on the road, a traffic sign, an intersection, a traffic light, road markings, etc. This specification defines the nature of element of interest 25i.

[0051] The position of the element of interest, in three-dimensional space, subsequently allows it to be positioned relative to the line of sight 21 of the driver's gaze.

[0052] Steps 105 and 106 are implemented using data from external vehicle sensors, for example, radar sensors, LIDAR sensors, and possibly visible light sensors (camera). The external sensor data is acquired in real time by said sensors and analyzed by at least one on-board computer in the vehicle to deduce detection information for an element of interest, and then determine its nature and position in space. The data from external sensors can be supplemented by map data, attached to map segments and stored in the vehicle and / or stored in the cloud and transferred to the vehicle as it travels, via a wireless connection.

[0053] The computer also implements a step 107 of determining data relating to an overall alert need.

[0054] Data relating to an overall alert need may include: - a type of distraction, each type of distraction being able to be associated subsequently with a distinct level of danger. A type of distraction includes, for example, a visual distraction (e.g., it is detected that the driver's line of sight is not directed towards the road, or that the driver is looking at a phone screen), and / or an auditory distraction (e.g., it is detected that the radio is on in the vehicle, or a conversation is taking place in the vehicle) and / or a cognitive distraction (e.g., it is detected that a discussion is taking place inside the passenger compartment or on the phone); - a degree of concentration of the driver, determined by analysis of at least one image of the driver (based on facial expressions, we can determine a degree of concentration, or in other words a degree of fatigue); - driver health data, including the presence or absence of vision problems (this data is associated, for example, with a digital driver profile); - driver eye movements (preferably a type of movement with an execution time, the type of movement including, for example, fixation, saccades, and / or blinks), measured on a series of images of the driver; - vehicle speed (the faster the vehicle is moving, the higher the risk of not noticing a risky situation in time).

[0055] Advantageously, data on the type of distraction and on the degree of driver concentration are combined together to define the value of a binary indicator of driver concentration.

[0056] In a later step 108, the calculator calculates a visual attention function 28 (also illustrated in [Fig.2B]).

[0057] The visual attention function 28, like the Gaussian 24, has its abscissas which correspond to the coordinates (x; y) of the points of the image of the scene 22.

[0058] The visual attention function 28 is derived, at least, from a distortion of the Gaussian 24.

[0059] The deformation of the Gaussian 24 uses, at least, the data relating to the overall need for alerting and the intrinsic characteristics of the element of interest.

[0060] Preferably, the deformation includes a modification of the values ​​of the standard deviations ox, oy of the Gaussian 24, according to the data relating to the overall need for alerting.

[0061] To this end, a function can be defined that links a value of ox and oy with the respective value of each of the data points relating to the overall alert requirement as listed above. This function will be determined using business data, assigning a greater or lesser weight to each data type and each category within a data type. The weights will be defined taking into account business knowledge relating to the influence of each data type and each category within a data type on the risk that a risky situation has not been identified by the driver. Weights may vary depending on typical driving conditions (for example, some parameters are assigned a greater or lesser weight for city or country driving).

[0062] Modifying the values ​​of the standard deviations ox, oy of the Gaussian 24 results in a more or less pronounced spreading of the initial Gaussian 24. This corresponds to the main peak 281 of the visual attention function 28 illustrated in [Fig.2B] (here narrower than the Gaussian 24 illustrated in [Fig.2A]).

[0063] The deformation may further include the use of at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes one or more values ​​related to the intrinsic characteristics of this element of interest; and - extends over only a part of the abscissas (x, y) of the Gaussian coordinate system 24, over an area including the abscissas of the corresponding element of interest.

[0064] Each local function is for example multiplied, or added, to the Gaussian 24 after modification of its standard deviations ox, oy.

[0065] Each use of such a local deformation function results in a local deformation of the Gaussian 24. This corresponds to a region 282 on the visual attention function 28 illustrated in [Fig.2B].

[0066] For each element of interest considered, the local deformation function extends over only a portion of the abscissas (x, y) of the Gaussian 24, over an area including the coordinates of the location of the element of interest in the scene image. Everywhere else, the local deformation function is undefined, or takes a value of 0 or 1 that does not modify the function to which it is added, respectively multiplied. In other words, the local deformation function applies to only a portion of a Gaussian (the Gaussian 24, possibly deformed by modifying its standard deviations), at abscissas including those of the element of interest in the scene image.

[0067] Each local deformation function is associated with one of the identified elements of interest and takes at least one value that depends on one or more intrinsic characteristics of that element of interest. In particular, each local deformation function is associated with one of the elements of interest and takes at least one value that depends on at least one intrinsic characteristic among: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight 21 of the driver's gaze, and an axis passing through the element of interest and through one of the driver's eyes. (If applicable, we simply consider whether or not it belongs to one or more cones of predetermined size.) - a speed of movement of the element of interest (movement relative to the vehicle receiving the driver, or movement in the terrestrial frame of reference).

[0068] Different weights can be assigned to each of the intrinsic characteristics used to determine a value taken by the local deformation function, depending on business knowledge and possibly current driving conditions.

[0069] Furthermore, the influence of each particular combination of intrinsic characteristics can be taken into account. For example, it is known that central vision allows objects to be distinguished precisely, while in peripheral vision humans are only attentive to moving objects.

[0070] The local deformation function can take a constant value over an entire region with abscissas (x, y) over which it extends. Alternatively, the local deformation function can, for example, take a decreasing value from the center to the edges of said region.

[0071] Advantageously, the local deformation function extends entirely and exclusively to the (x, y) coordinates of a surface object associated with the corresponding element of interest. This surface object may have a predetermined shape (for example, a square or a rectangle). Alternatively, this surface object may be bounded by the outline of the image of the element of interest on the scene image. In any case, the location of this surface object is centered on the image of the element of interest in the scene image.

[0072] In the example illustrated in [Fig.2B], region 282 extends over a square surface, represented by dashed lines. It corresponds to an element of interest associated with a square surface object.

[0073] It is understood that not all the elements of interest 25i visible in the scene image are necessarily associated with a respective local deformation function. For each element of interest, one can: - calculate the value of a risk coefficient based on one or more intrinsic characteristics of this element of interest; - compare this value to a predetermined threshold; and - only define a local deformation function if said value is greater than the threshold.

[0074] The distortion can also implement a forgetting factor, representative of a duration of memorization, by the driver, of information that he has previously visualized.

[0075] In this case, the visual attention function is a weighted sum of the deformed Gaussian 24 as described above, with at least one secondary Gaussian associated with a previous instant.

[0076] The secondary Gaussian corresponds to the Gaussian 24, associated with a previous instant and after deformation by modification of its standard deviation.

[0077] The weighting coefficient, or forgetting factor, depends on the time elapsed between a current instant and the preceding instant. It can also take into account driver behavior, such as eye movements. Indeed, a driver will memorize more information during fixation than during saccadic viewing. The forgetting factor can also depend on whether the element of interest is fixed or dynamic, depending on its nature. For example, an intangible element of interest, such as a road sign, can be viewed quickly, only once, and yet still memorized since it is not likely to change over time.

[0078] In any event, the calculation of the visual attention function includes a final normalization step.

[0079] For example, but not limited to, the visual attention function (before normalization) is defined by the following equation:

[0080] a ajAj

[0081] with:

[0082] attention_map the visual attention function;

[0083] Ai(Wi>w*Hh>i) the deformed Gaussian by modification of standard deviation, associated with the instant of index i;

[0084] N is the number of Gaussians considered (Gaussians associated with previous times and the Gaussian associated with the current time considered). This number N corresponds to a direction number of the line of sight, taken into account in the calculation of the visual attention function.

[0085] A; a weighting coefficient, fixed empirically, and representative of a duration of memorization, by the driver, of information that he visualized at time i.

[0086] = exp(£ * <5*xO2 )

[0087] [Math.2]

[0088] [3, ô of the coefficients depending on eye movement, with [3 fixation<|3 Saccade (similarly for ô);

[0089] (x0,y0) the coordinates of the point of intersection 23 between the line of sight 21 and the image of the scene;

[0090] ox=oy=o the initial standard deviation of the Gaussian 24 (fixed empirically);

[0091] w the desired width of an extent of the visual attention function along the x-axis of the image frame of the scene;

[0092] h the desired height of an extent of the visual attention function along the y-axis of the image frame of the scene;

[0093] OjAj the local deformation function associated with the element of interest of index j;

[0094] K the number of elements of interest considered;

[0095] a, a weighting coefficient based on the intrinsic characteristics of the object (its nature, its speed, its angular deviation from the current position of the line of sight 21);

[0096] Aj is a positioning matrix of the surface object associated with the element of interest. Aj is a binary matrix with the same dimensions as the scene image and the x-axis of the Gaussian distributions. Each component of the matrix Aj takes the value 0 or 1 depending on the presence or absence of the element of interest at the corresponding x-coordinates (and possibly depending on the speed of this element of interest, for example, to consider only moving objects).

[0097] A calculation step 109 is then implemented, for each element of interest 25i, of a degree of attention which depends on at least one value of the visual attention function 28 at the abscissas corresponding to the location of the element of interest in the image of the scene.

[0098] The location of the element of interest in the scene image corresponds to the position and dimensions of a surface object as described above.

[0099] Step 109 preferably consists of calculating, for each element of interest 25i, the average value taken by the visual attention function 28 at the coordinates of the corresponding surface object (the abscissas of the visual attention function 28 and the coordinates of the surface object being defined in the same coordinate system (x,y)). Since the visual attention function 28 is a normalized function, a score between 0 and 1 is thus obtained for each element of interest.

[0100] In [Fig.2C], the visual attention function 28 is schematically represented superimposed, with surface objects 29i each associated with one of the respective elements of interest.

[0101] Thanks to the deformation steps of the Gaussian 24, the average value taken by the visual attention function 28 at the coordinates of a surface object 29i is a function of: - data relating to the overall need for alertness (type of distraction, degree of concentration, driver health data, eye movements, vehicle speed), which determines the extent to which the driver pays attention only to what is directly in their line of sight, or also to what is further to the periphery; and - intrinsic characteristics of the corresponding element of interest (in particular its nature, its speed of movement and its angular deviation from the line of sight).

[0102] Thus, the degree of attention associated with each element of interest is determined precisely, and takes into account a current situation in its entirety by considering a plurality of factors associated with the behavior of the driver and the external environment.

[0103] This is followed by a comparison step 110, for each element of interest, between the corresponding level of attention and a predetermined threshold Th_l, to determine whether or not it is necessary to alert the driver to a risky situation. Thanks to the precise determination of the level of attention, it is determined accurately, for each element of interest, whether it is necessary to alert the driver to an associated risk, or whether such an alert would instead create redundancy of information likely to lead the driver to deactivate warning functions.

[0104] Steps 101 to 110 determine whether or not the driver needs to be alerted to a risky situation. These steps are advantageously followed by step 111, which generates at least one warning signal, when a need to alert the driver is determined in step 110.

[0105] The warning signal may be audible and / or visual (for example, via a light strip in the passenger compartment or via the head-up display) and / or haptic (for example, a vibration of the steering wheel and / or accelerator pedal). It is generated using at least one human-machine interface (HMI). It may be a signal oriented in the same direction as the element of interest for which a warning is to be issued. For example, the vehicle has an LED strip under the windshield, and an LED located at the same height as this element of interest is illuminated.

[0106] A driving assistance method according to a second embodiment of the invention is then described with reference to [Fig.3].

[0107] In this embodiment, steps 101 to 110 of [Fig.1] are implemented, in use, with at least one software application of interest, run using hardware and software resources shared with at least one additional software application.

[0108] At least one additional software application includes, for example, a trajectory prediction application and / or a speech analysis application.

[0109] In the advantageous embodiment illustrated in [Fig.3], it is proposed that these steps 101 to 110 no longer form a function to be called continuously, but a function to be used in a reasoned manner with respect to the driving context, calling upon only the computers necessary at the required time.

[0110] The method 300 according to the invention then comprises the following steps, implemented by at least one computer embedded in the vehicle, this computer corresponding to the shared hardware and software resources mentioned above.

[0111] In a step 30, a current value of available computing capacity, Rc, is determined, corresponding to the computing capacity available from said hardware and software resources, and at the time considered, to implement the assessment of an alert need as described with reference to [Fig.1].

[0112] In practice, a respective priority level can be assigned to each of the software applications of interest and the additional software applications. The Rc value then corresponds to the difference between the total computing capacity of the hardware and software resources and the computing capacity required to execute the additional software applications with higher priority than the software application of interest, which are currently running or about to be executed at the given time.

[0113] The determination of a computing capacity required to execute the additional higher priority software applications can be implemented by tools known to those skilled in the art.

[0114] In a step 31, the value Rc is compared to a predetermined computing capacity threshold, referenced Th_2. The predetermined threshold Th_2 can be the zero value, or a positive value representing a tolerance or safety threshold.

[0115] When Rc is greater than Th_2, this means that the computing capacity available at the time considered allows for the implementation of the alert need assessment as described with reference to [Fig.1]. In this case, a step 32b of the alert need assessment as described with reference to [Fig.1] is implemented, then called the enhanced assessment.

[0116] When Rc is less than Th_2, this means that the available computing power at the time considered does not allow for the implementation of the enhanced assessment. In this case, a simplified assessment of the need for an alert is implemented in step 32a. The simplified assessment uses fewer hardware and software resources than the enhanced assessment. It also determines whether or not it is necessary to alert a driver to a risky situation, but with less precision in assessing the need.

[0117] Optionally, steps 33 and 34 may be implemented, consisting of: - calculating a risk level associated with each element of interest, this calculation being carried out in parallel with the simplified assessment 32a (step 33); and - switch to the improved assessment of step 32b, when the risk level associated with at least one element of interest exceeds a predetermined risk level threshold Th_3.

[0118] It is therefore possible, when a potential danger is identified, to force the execution of the enhanced assessment in order to offer the driver the most relevant alerts possible.

[0119] For example, moderate priority levels can be defined, corresponding to additional software applications whose execution can be temporarily suspended when a switch to the improved evaluation is necessary.

[0120] The calculation of the risk level associated with each element of interest is advantageously implemented by a dedicated application, making it possible to provide different risk levels related to potential collisions, overspeed, or even hidden elements of interest (pedestrian crossings, etc.). The risk related to the elements of interest in the scene is predicted over a period of 6 seconds to 2 seconds before the estimated potential collision.

[0121] Figure 4 illustrates an example of a simplified evaluation procedure that can be implemented in step 32a, using at least one calculator.

[0122] In a step 401, the line of sight 21 of the driver's gaze is determined using at least one image of the driver.

[0123] In a step 402 (which can be implemented in parallel with step 401), the elements of interest 25i in the vehicle environment are identified, and their respective positions in three-dimensional space are determined.

[0124] This step 402 uses data from external sensors (radar and / or LiDAR and / or camera), and possibly map data. Depending on available computing power, it is possible to use a greater or lesser amount of external sensor data and / or map data to implement this step 402.

[0125] In a later step 403, for each element of interest 25i, the angular deviation Ei is calculated between the line of sight 21 of the driver's gaze and an axis 41i passing through the element of interest 25i and through an eye of the driver.

[0126] In a step 404, each angular deviation Ei is compared to at least one predetermined angular deviation threshold Th_4. Based on the result of the comparison, it is determined whether or not the driver needs to be alerted to a risky situation.

[0127] In other words, a cone of revolution centered on the line of sight 21 and with apex angle Th_4 can be defined. Depending on whether the element of interest 25i is located inside or outside the cone, it is considered to be seen or not by the driver. When the element of interest is considered to be seen by the driver, no alert is deemed necessary. When the element of interest is considered not seen by the driver, it is deemed necessary to alert the driver. In this case, before any alert is issued, the system can switch to the enhanced evaluation as illustrated with reference to [Fig. 3].

[0128] In a preferred embodiment, two threshold values ​​for angular difference are defined, corresponding to two cones of vision. The narrower cone defines a zone of maximum attention for the driver. For example, it has an apex angle between 2° and 10°. The wider cone defines, where it does not overlap the cone A narrower area represents a moderate attention zone. For example, it might have a vertex angle between 50° and 70°. The rest of the scene falls within a reduced attention zone for the driver. Each zone is associated with a specific attention level. Depending on the zone in which the element of interest is located, it is assigned the corresponding attention level. Based on the attention level assigned to the element of interest, it is determined whether or not the driver needs to be alerted.

[0129] In variants, more than two concentric cones are used, together defining a plurality of zones each associated with a respective value of degree of attention.

[0130] According to other variants, the simplified evaluation also uses intrinsic characteristics of the element of interest, in particular its nature and / or speed, to determine whether or not it is necessary to alert the driver.

[0131] According to other variants, the simplified assessment uses a three-dimensional object representing the element of interest, and calculates an average value of degree of risk on the three-dimensional object in order to determine whether or not it is necessary to alert the driver.

[0132] In any event, the simplified evaluation uses less sensor data than the enhanced evaluation. In particular, the data stream from the camera aimed at the scene in front of the vehicle is not used. Thus, the computing resources required for its implementation are much lower.

[0133] In any event, the choice between a simplified or enhanced evaluation always makes it possible to assess the need to alert the driver, regardless of the available computing power. The issue of limiting redundancy in alerts remains addressed.

[0134] Finally, with reference to [Fig.5] and in a schematic manner, a system 50 adapted to the implementation of a driving assistance method according to the invention is illustrated.

[0135] The system 50 includes a set of external sensors 51 of the vehicle, including radars, LIDAR and cameras.

[0136] The external sensors 51 may include: - an interior camera filming the driver's face. This camera allows the driver's gaze direction to be seen during driving. Alternatively, the camera can be replaced by a wearable eye tracker; - a contextual camera filming the scene at the front of the vehicle; - at least one radar and / or LiDAR, possibly integrated into a sensor fusion system that identifies obstacles present in the driving scene and all information concerning them (their speed, their position relative to the driver's vehicle, their nature, etc.). These objects are usually processed by dedicated algorithms that allow for a representation of the world surrounding the vehicle.

[0137] The system 50 also includes a management module (or platform) 52, configured to: - determine information about available computing capacity; - to implement steps 30 and 31 of the process according to the invention, to determine whether to implement a simplified or improved assessment of the need for alerting; and - to manage data streams from external sensors, to select the sensor data useful or not, at each time, during the implementation of the invention.

[0138] The system 50 also includes a set 53 of at least one computer, configured to implement the simplified or improved assessment steps of the need for alerting.

[0139] The system 50 also includes at least one human-machine interface 54, for the emission of at least one audible and / or visual and / or haptic warning signal, controlled by the computer 53.

[0140] Block 55 symbolizes a level of risk, or need to alert the driver, which acts both on the human-machine interface 54 which will be solicited or not, and on the data flow management module 52 if the need to switch from a simplified assessment to an improved assessment of the need for alert is not determined.

[0141] The invention is not limited to the examples described above, and covers variants with other external sensors, other intrinsic data of the element of interest, and / or other data relating to an overall need for alerting.

[0142] The invention finds a particularly advantageous application in the field of Advanced Driver Assistance Systems (ADAS), but also in the field of autonomous vehicles, for example to control a transfer of commands to the driver when it is determined that it is necessary to alert the driver to a risky situation.

Claims

1. Demands Method (100) for assisting the driving of a motor vehicle implementing an assessment of a need for an alert, this assessment consisting of determining whether or not it is necessary to alert a driver to a risky situation, the method being characterized in that the assessment of a need for an alert comprises the following steps: a) Determination (101) of a line of sight (21) of the driver's gaze, using at least one image of the driver; b) Reception (102) of an image (22) of a scene surrounding the vehicle, c) Determination (103) of the point of intersection (23) between the image of the scene and the line of sight of the driver's gaze; d) Calculation (104) of a two-dimensional Gaussian (24) centered on this point of intersection, the abscissas of said Gaussian corresponding to the coordinates of the points in the image of the scene; e) Using data from, in particular, external sensors (51) of the vehicle, identification (105) of at least one element of interest (25i) in the environment of the vehicle; (f) For each element of interest (25i), determination (106) of intrinsic characteristics of the element of interest, including at least its nature and its position in three-dimensional space; (g) Using data from at least one sensor in the vehicle, determination (107) of data relating to an overall warning need; (h) Determination (108) of a visual attention function (28) resulting, at least, from a deformation of said Gaussian (24), the deformation using at least the data relating to the overall warning need and the intrinsic characteristics of the element of interest; i) For each element of interest (25i), calculation (109) of a degree of attention, which depends on at least one value of the visual attention function (28) at the abscissas corresponding to the location of the element of interest in the image of the scene; j) For each element of interest (25i), comparison (110) between the corresponding degree of attention and a predetermined threshold (Th_l), to determine whether or not it is necessary to alert the driver to a risky situation.

2. Method (100) according to claim 1, further comprising a step (111) of generating at least one warning signal, when it is determined in step j) that it is necessary to alert the driver to a hazardous situation.

3. Method (100) according to claim 1 or 2, wherein the location of the element of interest on the scene image includes the location of a surface object (29i) on the scene image.

4. Method (100) according to claim 3, wherein the degree of attention associated with an element of interest (25i) is defined by the average value of the visual attention function (28) on the corresponding surface object (29i).

5. A method (100) according to any one of the preceding claims, wherein the data relating to the overall need for alerting includes at least one of the following: - a type of distraction, including a type of visual distraction and / or a type of auditory distraction and / or a type of cognitive distraction; - a degree of driver concentration, determined by analysis of at least one image of the driver; - driver health data; - driver eye movements, measured on a series of driver images; - vehicle speed.

6. Method (100) according to any one of the preceding claims, wherein the deformation (108) of the Gaussian includes a modification of at least one of its standard deviations, depending on the data relating to the overall warning need.

7. A method (100) according to any one of the preceding claims, wherein the deformation (108) of the Gaussian uses at least one local deformation function, where each local deformation function: - is associated with one of the respective elements of interest; - takes at least one value that depends on the intrinsic characteristics of that element of interest; and - extends over only a part of the abscissas of the Gaussian, over an area including the corresponding element of interest.

8. Method (100) according to claim 7, wherein at least one value taken by the local deformation function depends on at least one parameter among: - a degree of risk associated with the nature of the element of interest; - an angular deviation between the line of sight (21) of the driver's gaze and an axis passing through the element of interest and through an eye of the driver; - a speed of movement of the element of interest.

9. Method (100) according to claim 7 or 8, wherein each local deformation function takes a constant value, and applies to only a part of the Gaussian, associated with abscissas of the element of interest (25i) in the scene image.

10. Method (100) according to any one of claims 1 to 9, wherein the visual attention function (28) is a weighted sum of said distorted Gaussian and at least secondary Gaussian associated with a previous instant.

11. A method (300) according to any one of claims 1 to 10, wherein: the assessment of an alert need is implemented, in use, with the aid of at least one software application of interest, executed using hardware and software resources shared with at least one additional software application; and the method further comprises preliminary steps of: determining (30) a current value (Rc) of available computing capacity, corresponding to the computing capacity available from said hardware and software resources to implement the assessment of an alert need, and comparing (31) to a predetermined threshold (Th_2) of computing capacity, the assessment of an alert need forming an enhanced assessment and being implemented when said current value of available computing capacity is greater than said predetermined threshold.

12. Method (300) according to claim 11, wherein a degree of priority is assigned to each of the software application of interest and at least one additional software application, and the current value (Rc) of available computing capacity is a function of whether or not one or more higher priority additional software applications are running.

13. Method (300) according to claim 11 or 12, comprising, where the current value (Rc) of available computing capacity is less predetermined threshold audit, the implementation (32a) of a simplified assessment of an alert need, using fewer hardware and software resources than the enhanced assessment.

14. A method (300) according to claim 13, wherein said simplified assessment of a need for an alert comprises the following steps: a) Determination (401) of a line of sight (21) of the driver's gaze, using at least one image of the driver; 3) Using data from, in particular, external sensors of the vehicle, identification (402) of at least one element of interest (25i) in the environment of the vehicle and determination of its position in three-dimensional space; y) For each element of interest (25i), calculation (403) of an angular deviation (Ei) between the line of sight (21) of the driver's gaze, and an axis (41i) passing through the element of interest and through an eye of the driver; and o) Determination (404) of the need to alert the driver to a risky situation, by comparing said angular deviation with at least a predetermined angle threshold.

15. A method (300) according to claim 13 or 14, comprising: obtaining (33) risk level information associated with each element of interest; and switching (34) to the enhanced assessment (32b) when the risk level associated with at least one element of interest exceeds a predetermined risk level threshold (Th_3).

16. Computer program comprising instructions, executable by a microprocessor or microcontroller, for the implementation of a method (100; 300) according to any one of claims 1 to 15, when executed by the microprocessor or microcontroller.

17. Calculator configured to implement the steps of a process (100; 300) according to any one of claims 1 to 15.

Images