Method for evaluating the performance of a driving model for a vehicle

Abstract

Method for Evaluating the Performance of a Driving Model for a Vehicle. The present invention relates to a method for evaluating the performance of a driving model for a vehicle, the driving model being a neural network previously trained to determine a driving action of the vehicle, from among a set of predefined actions, as a function of a set of states characterizing a scenario in which the vehicle evolves in an environment. The method comprises the following phases: obtaining an evaluation database, the evaluation database comprising several scenarios characterized by a set of states, and evaluating the performance of the driving model for each scenario in the evaluation database. The evaluation is performed by determining tolerated and critical actions and limit scenarios for each scenario considered. Figure for the abstract: Figure 2

Description

Title of the invention: Method for evaluating the performance of a driving model for a vehicle

[0001] The present invention relates to a method for evaluating the performance of a driving model for a vehicle. The present invention also relates to an associated computer program product.

[0002] In recent years, the safety of deep learning tools has become a major topic of research in the field of artificial intelligence.

[0003] In particular, deep reinforcement learning, which combines reinforcement learning and deep learning techniques, is mainly used for robot manipulation or navigation. However, the models obtained by such techniques are considered black-box algorithms due to the use of deep neural networks to make the predictions.

[0004] For this purpose, several approaches have been developed to certify the decision of models obtained by deep reinforcement learning. In particular, adversarial reinforcement learning methods using two agents, namely an adversary and a protagonist, are known. Methods based on the observation of the worst possible state are also known. Other methods use control functions where the policy learned by the neural network belongs to a set of policies considered safe.

[0005] However, such methods do not allow security needs to be met in terms of industrial requirements and specifications. The lack of a response to these needs constitutes the main limitation to the deployment and industrialization of models obtained by reinforcement learning.

[0006] There is therefore a need for a means allowing a better evaluation of the performance of an artificial intelligence tool, particularly in terms of operational safety and robustness.

[0007] For this purpose, the present description relates to a method for evaluating the performance of a driving model for a vehicle, the driving model being a neural network previously trained to determine a driving action of the vehicle, from a set of predefined actions, as a function of a set of states characterizing a scenario in which the vehicle evolves in an environment, the method being implemented by computer and comprising the following phases: a. obtaining an evaluation database, the evaluation database comprising several scenarios in which the vehicle evolves in an environment, each scenario being characterized by a set of states capable of taking different values ​​over time, each state having an initial value corresponding to the start time of the corresponding scenario, b. the evaluation of the performance of the driving model for each scenario in the evaluation database, the evaluation phase comprising, for each scenario in the evaluation database, the following steps: i. the determination, from among the set of predefined actions, for each instant of the scenario considered, of the actions tolerated for driving the vehicle according to the scenario considered and of the critical actions for driving the vehicle according to the scenario considered, ii. the determination, by the driving model, of a driving action of the vehicle for each instant of the scenario considered, as long as the determined action is different from a critical action, iii. determining, for at least one state of the scenario considered, called the disturbed state, a lower bound and an upper bound relating to a disturbance of said disturbed state, the disturbance extending over an interval of disturbance values, the lower bound and the upper bound being the outputs of a predetermined model whose inputs are the initial value of the state and the interval of disturbances, the initial value of said disturbed state being between the lower bound and the upper bound, iv. the determination of a set of scenarios, called limit scenarios, for the scenario considered, each limit scenario differing from the scenario considered by replacing the initial value of at least one disturbed state by the lower limit or by the upper limit of said disturbed state, each disturbed state having as its value the lower limit of said disturbed state in at least one limit scenario, and having as its value the upper limit of said disturbed state in at least one other limit scenario, and v. the determination, by the driving model, of a driving action of the vehicle for each instant of each determined limit scenario, as long as the determined action is different from a critical action, vi. validation of the driving model for the scenario considered when, for each instant, the action determined for the scenario considered and the actions determined for the limit scenarios correspond to the same action among the actions tolerated for the scenario considered at that time, the driving model being invalidated for the scenario considered otherwise.

[0008] According to particular embodiments, the method comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0009] - when the driving model is invalidated for the scenario considered, it is obtained at least one corresponding invalidation ground which is chosen from the following grounds:

[0010] - a reason for invalidation for non-compliance with security constraints when at least one of the actions determined for the scenario considered and the limit scenarios is a critical action, and

[0011] - a reason for invalidation for lack of robustness when the action determined for at least one instant of the limit scenario is different from the action determined for the same instant of the scenario considered. - the steps of determining limits, determining a set of limit scenarios, determining a driving action for each limit scenario and validating the driving model for the scenario considered are implemented only when the actions determined for the scenario considered are tolerated actions, the driving model otherwise being invalidated for the scenario considered for non-compliance with safety constraints. - the scenarios of the evaluation database are such that for at least one instant of a scenario, several distinct tolerated actions for driving the autonomous vehicle are determined. - the tolerated and critical actions for each scenario considered are determined according to security constraints, the security constraints preferably being in accordance with the ISO / PAS 21448:2019 SOTIF standard. - the states characterizing each scenario include the position and speed of the vehicle, as well as the relative position and, where applicable, the relative speed, of at least one element in the vehicle's environment. - the disturbed states are chosen from the position and speed of the vehicle. - at the end of the evaluation phase, the process includes one of the following phases: • the validation and / or certification of the driving model for integration into a vehicle when data, relating to the possible scenarios for which the driving model has been invalidated, is included in a first predetermined interval, or • the invalidation of the driving model when data, relating to the possible scenarios for which the driving model has been invalidated, is included in a second predetermined interval, the second predetermined interval being different from the first predetermined interval, or • the determination of rules for using the driving model based on at least one scenario for which the driving model has been invalidated. - the driving model was trained using a reinforcement learning technique, preferably deep reinforcement learning.

[0012] The present description also relates to a computer program product comprising a readable information medium, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and adapted to cause the implementation of a method as previously described when the computer program is implemented on the data processing unit.

[0013] The present description also relates to a readable information medium on which a computer program product as previously described is stored.

[0014] Other characteristics and advantages of the invention will appear on reading the following description of embodiments of the invention, given by way of example only and with reference to the drawings which are:

[0015] [Fig. 1], [Fig. 1], a schematic view of an example of a computer allowing the implementation of a method for evaluating the performance of a driving model for a vehicle,

[0016] [Fig.2], [Fig.2], a flowchart of an example of implementation of a method for evaluating the performance of a driving model for a vehicle,

[0017] [Fig.3], [Fig.3], an example of a scene from a first scenario in which the study vehicle moves in an environment,

[0018] [Fig.4], [Fig.4], an example of a scene from a second scenario in which the study vehicle moves in an environment,

[0019] [Fig.5], [Fig.5], an example of a scene from a third scenario in which the study vehicle moves in an environment, and

[0020] [Fig.6], [Fig.6], an example of a scene from a fourth scenario in which the study vehicle moves in an environment.

[0021] A calculator 10 and a computer program product 12 are illustrated in [Fig.l].

[0022] The calculator 10 is preferably a computer.

[0023] More generally, the computer 10 is an electronic computer capable of manipulating and / or transforming data represented as electronic or physical quantities in computer registers 10 and / or memories into other similar data corresponding to physical data in memories, registers or other types of display, transmission or storage devices.

[0024] The computer 10 interacts with the computer program product 12.

[0025] As illustrated by [Fig.l], the computer 10 comprises a processor 14 comprising a data processing unit 16, memories 18 and an information medium reader 20. In the example illustrated by [Fig.l], the computer 10 comprises a keyboard 22 and a display unit 24.

[0026] The computer program product 12 comprises an information medium 26.

[0027] The information medium 26 is a medium readable by the computer 10, usually by the data processing unit 16. The readable information medium 26 is a medium adapted to memorize electronic instructions and capable of being coupled to a bus of a computer system.

[0028] By way of example, the information medium 26 is a floppy disk or flexible disk (from the English name “Floppy disk”), an optical disk, a CD-ROM, a magneto-optical disk, a ROM memory, a RAM memory, an EPROM memory, an EEPROM memory, a magnetic card or an optical card.

[0029] The computer program 12 comprising program instructions is stored on the information medium 26.

[0030] The computer program 12 is loadable onto the data processing unit 16 and is adapted to cause the implementation of a method for evaluating the performance of a driving model for a vehicle, when the computer program 12 is implemented on the processing unit 16 of the computer 10.

[0031] The operation of the computer 10 will now be described with reference to [Fig. 2], which schematically illustrates an example of implementation of a method for evaluating the performance of a driving model for a vehicle, and to FIGS. 3 to 6 which are examples illustrating scenes of scenarios used during certain phases of the method.

[0032] The evaluation method aims to evaluate the performance of a driving model M for a vehicle. The vehicle is preferably an autonomous vehicle, such as an autonomous car, or a semi-autonomous vehicle. In other examples, the vehicle is chosen from all types of land, air or even naval vehicles.

[0033] The term "performance" means the assessment of the safety of the decisions taken by the model, i.e. compliance with safety constraints by the model, as well as the robustness of the model. The robustness of the model refers to the ability to not not be disturbed by a small change in the input data or in the model parameters.

[0034] The driving model M is a neural network previously trained to determine a driving action At of the vehicle, from a set of predefined actions At, as a function of a set of states St characterizing a scenario in which the vehicle evolves in an environment. A scenario (also called an episode) is a succession of scenes in time, the scenes occurring at successive instants. The instants correspond, for example, to the instant at which images of the scenes are taken and therefore depend on the image frequency. The states St characterizing the scenario are therefore the inputs of the driving model M and the actions At are the potential outputs of the driving model M.

[0035] In particular, the states St characterize the vehicle and the environment in which the vehicle is moving for a given scenario. Preferably, the states St characterizing each scenario comprise the position and the speed of the vehicle, as well as the relative position and, where appropriate, the relative speed, of at least one element in the environment of the vehicle. In one embodiment, the states St also comprise an indication of the direction of movement of the vehicle and of any other moving elements. Alternatively, it is assumed that the vehicle and any elements are moving in the same direction.

[0036] The elements are, for example, obstacles to be avoided by the vehicle. The obstacles are, for example, other vehicles, human beings, animals or even installations or objects on the path of the vehicle.

[0037] The At actions are intended to control the vehicle so as to ensure the driving of the vehicle. The At actions are intended either to be directly implemented on the vehicle (autonomous driving), or to serve as assistance to a driver (semi-autonomous driving).

[0038] In one example, the driving actions At are selected from the following driving actions: accelerate, decelerate, turn right, turn left, and do nothing.

[0039] Preferably, the driving model M has been trained using a reinforcement learning technique, or even a deep reinforcement learning technique. This means that the data used to train the driving model M is not labeled data.

[0040] Reinforcement learning implements an agent that learns to perform a task by maximizing cumulative rewards counted. The agent acts by sequentially choosing actions from observations over a sequence of time steps. Deep reinforcement learning uses networks of neurons to learn or approximate certain components (parameters) of the reinforcement learning algorithm, such as the Bellman equation, state transition, and policy.

[0041] For example, the driving model M was trained according to a so-called proximal policy optimization algorithm (PPO) as described in the article Schulman, J., F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017), Proximal policy optimization algorithms. arXi vpr eprin t arXiv:1707.06347. Such a PPO algorithm uses a critic neural network and an actor neural network. The critic network produces an estimate of a value function that is an estimate of the expected future cumulative discounted reward from current states, following a current policy defined by the actor network.

[0042] Alternatively, the driving model M was trained using other reinforcement learning techniques.

[0043] The evaluation method comprises a phase 100 of obtaining an evaluation database. Phase 100 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0044] The evaluation database was designed to evaluate the performance of the driving model M. The evaluation database thus comprises several scenarios in which the vehicle evolves in an environment, each scenario being characterized by a set of states St taking different values ​​over time. Each state St has an initial value Vc corresponding to the start time (initial time) of the corresponding scenario.

[0045] The scenarios have, for example, been generated via a scenario generation tool, such as the tool described in the article Leurent, E. (2018), An Environment for Autonomous Driving Decision-Making. Alternatively, the scenarios are obtained on real cases via measurements made by sensors.

[0046] Figures 3 to 6 illustrate examples of two-dimensional scenario scenes in which an autonomous car evolves in an environment. In this environment, the agent drives a vehicle (car) on an infinite highway with four traffic lanes all going in the same direction. The agent observes three lanes, the one on which it is located VP, the two adjacent lanes, left VG and right VD. The vehicle driven by the agent (the ego-vehicle 40) is inserted into a traffic flow (the exo-vehicles 42). Each exo-vehicle 42 follows a basic driving algorithm. The objective of the agent is to drive as quickly as possible without collision. The episode ends when the ego-vehicle 40 collides with another vehicle 42 or when a predetermined duration of the episode is reached.

[0047] In these scenarios, the states St include the following states: the position of the ego-vehicle 40 across the width of the road, the speed of the ego-vehicle 40, the positions of the nearest exo-vehicles 42, front and rear, right and left, on each of the adjacent lanes relative to the ego-vehicle 40, and their speeds relative to the speed of the ego-vehicle 40.

[0048] The possible actions At for the driving model M for these scenarios are: accelerate, decelerate, turn right, turn left, and do nothing (stay in the same lane with the same speed). An acceleration is, for example, 5 meters per second. Similarly, a deceleration is, for example, 5 meters per second.

[0049] In this example, the reward function of the driving model M is of the following form:

[0050] = max( _ j. (?ego. M ( ) ) / (max (Vego-M ( ) ))

[0051] Where: • Veg° is the speed of the ego vehicle, • M(Vexo) is the average of the speeds of the exos vehicles, and • max ( X ) denotes the maximum of X.

[0052] The evaluation method comprises a phase 110 of evaluating the performance of the driving model M for each scenario of the evaluation database. The evaluation phase 110 aims in particular to evaluate compliance with safety constraints (also called safety requirements) and the robustness of the driving model M for each scenario of the evaluation database. The phase 110 is implemented by the computer 10 in interaction with the computer program product 12, that is to say is implemented by computer.

[0053] In the following, we describe the steps of the evaluation phase 110 which are implemented for each scenario of the evaluation database.

[0054] The evaluation phase comprises a step 110A of determining, from among the set of actions At predefined for the driving model M, for each instant of the scenario considered, the actions At tolerated for driving the vehicle according to the scenario considered and the actions At critical for driving the vehicle according to the scenario considered. An action is said to be tolerated when its implementation complies with safety constraints for the scenario considered. An action is said to be critical otherwise. During step 110A, each action is determined as being tolerated or critical, independently of the action actually carried out in the simulation.

[0055] Preferably, the tolerated and critical actions At for each scenario considered are determined based on predefined security constraints (requirements).

[0056] The safety constraints are advantageously compliant with the ISO / PAS 21448:2019 SOTIF standard. This standard defines acceptable safety levels for a road vehicle in order to avoid unreasonable risks. The basic concept is the introduction of an iterative process of development and design of the function which includes validation and verification, which leads to a function considered safe. This approach assumes that there is an area of ​​known scenarios with safe behavior of the system and an unknown area with potential damage.

[0057] Preferably, the scenarios of the evaluation database are such that for at least one instant of a scenario, several distinct tolerated actions and / or critical actions for driving the autonomous vehicle are determined. The outputs of the driving model M are therefore not binary.

[0058] In the examples of Figures 3 to 6, the zones Zc corresponding to critical actions for the vehicle and the zones ZT corresponding to actions tolerated for the vehicle are illustrated. In particular, for the scene of the scenario of [Fig.3], the only critical action is to turn right. For the scene of the scenario of [Fig.4], the critical actions are to turn right or to turn left. For the scene of the scenario of [Fig.5], the critical actions are to turn right or to accelerate. For the scene of the scenario of [Fig.6], the only critical action is to decelerate.

[0059] The evaluation phase 110 comprises a step 110B of determining, by the driving model M, a driving action At of the vehicle for each instant of the scenario considered. The determined action is either a tolerated action or a critical action. This is carried out as long as the determined action is different from a critical action (and until the end of the scenario).

[0060] In a preferred embodiment, the following steps of the evaluation phase are implemented only when the action determined for each instant of the scenario considered is a tolerated action. Indeed, when the action determined is a critical action, this means that the driving model M is not safe for the scenario considered (non-compliance with the safety constraints). The driving model M is then considered invalid for the scenario considered, from step 110B.

[0061] The evaluation phase 110 comprises a step 110C of determining, for at least one state St of the scenario considered, called disturbed state St*, a lower bound ôinf and an upper bound ôsup relating to a disturbance of said disturbed state St*. The lower bound ôinf and the upper bound ôsup are values ​​of the same nature as the disturbed state St* (if the disturbed state corresponds to a speed (respectively a position), the disturbance is a speed (respectively a position)).

[0062] The disturbance extends over a range of disturbance values. For example, plus or minus 5 km / hour for speed, or plus or minus 3 meters for position.

[0063] The lower bound ôinf and the upper bound ôsup are determined on the basis of a predetermined model, called the abstract interpretation model. The inputs of the abstract interpretation model are the initial value Vc of the perturbed state and the corresponding perturbation interval. The outputs of the abstract interpretation model are the lower bound ôinf and the upper bound ôsup.

[0064] The abstract interpretation model is a set of functions. The abstract interpretation model was designed in accordance with the theory of abstract interpretation. Examples of implementing an abstract interpretation model are described in the following articles: - Article 1 : Patrick Cousot & Radhia Cousot. Abstract interprétation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Conférence Record ofthe Sixth Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 238—252, Los Angeles, California, 1977. ACM Press, New York. - Article 2 : Gehr, T., M. Mirman, D. Drachsler-Cohen, P. Tsankov, S. Chaudhuri et M. Vechev (2018). Ai2 : Safety and robustness certification of neural networks with abstract interprétation. In 2018 IEEE Symposium on Security and Privacy (SP), pp 3-18 IEEE.

[0065] Abstract interpretation as described in Article 1 is a discrete approximation theory of the semantics of computer systems, mainly used for software analysis and static verification. Program semantics describes the set of all possible behaviors of these programs when executed for all possible input data. In other words, the behavior of any program can be formally described for all the relations and transitions that constitute it. Overall, abstract interpretation is a classical method for robust and accurate over-approximation of a potentially unlimited or infinite set of program behaviors. The key ideas behind this concept are (i) abstract transformers" to approximate program behavior.They are defined for the instructions used by the program (e.g., affine arithmetic, ReLU functions, etc.); (ii) the abstract domain which is simply a set of abstract elements (approximations) generally ordered in a lattice (ordered set where any pair of elements has an upper bound and a lower bound).

[0066] Article 2 develops a verification method called AI2 based on abstract interpretations that can reason about the security and robustness of Deep Learning systems. AI2 aims to evaluate a deep neural network work against local or global perturbations. It also guarantees the absence of adversarial attacks in a certified interval. The abstract interpretation model is, for example, consistent with the model described in this article.

[0067] The abstract interpretation model is, for example, built on the basis of the ERAN and ELINA libraries.

[0068] The initial value Vc of said disturbed state St* is between the lower bound ôinf and the upper bound ôsup.

[0069] The perturbed states St* are preferably states relating to the study vehicle, and not states of the environment. The perturbed states St* are advantageously chosen from the position and speed of the vehicle.

[0070] For example, in the case of the scenario scenes illustrated in Figures 3 to 6, a position disturbance for the ego vehicle is considered which extends over an interval of 0 to 8.75 meters, and a speed disturbance for the ego vehicle which extends over an interval of 0 to 1.75 m / s.

[0071] The evaluation phase 110 comprises a step 110D of determining a set of scenarios, called limit scenarios, for the scenario considered. Each limit scenario differs from the scenario considered by replacing the initial value Vc of at least one disturbed state St* by the lower bound ôinf or by the upper bound ôsup of said disturbed state St*.

[0072] Each perturbed state St* has as its value the lower bound ôinf of said perturbed state St* in at least one limit scenario, and has as its value the upper bound ôsup of said perturbed state St* in at least one other limit scenario. In other words, each lower or upper bound of a perturbed state St* corresponds to at least one limit scenario.

[0073] In one example, each limit scenario corresponds to the lower or upper bounds of several perturbed states St* (position and velocity for example). In another example, the limit scenarios cover all combinations of lower and upper bounds for the perturbed states St*.

[0074] The evaluation phase 110 comprises a step 110E of determining, by the driving model M, a driving action At of the vehicle for each instant of each determined limit scenario. The action determined for each instant of each limit scenario is either a tolerated action or a critical action. This is carried out for each limit scenario as long as the determined action is different from a critical action (and until the end of the limit scenario).

[0075] The evaluation phase 110 comprises a step 110F of validating or invalidating the driving model M for the scenario considered. In particular, the driving model M is validated for the scenario considered when, for each instant, the action At determined for the scenario considered and the actions At determined for the limit scenarios correspond to the same action At among the actions At tolerated for the scenario considered (determined for the moment considered). The driving model M is invalidated for the scenario considered otherwise.

[0076] Preferably, when the driving model M is invalidated for the scenario considered, at least one corresponding invalidation reason is obtained which is chosen from the following reasons:

[0077] - a reason for invalidation for non-compliance with security constraints when at least one of the At actions determined for the considered scenario and the limit scenarios is a critical At action, and

[0078] - a reason for invalidation for lack of robustness when the action At determined for at least one instant of the limit scenario is different from the action At determined for the same instant of the scenario considered.

[0079] Optionally, the evaluation method 120 comprises one of the following phases:

[0080] - the validation and / or certification of the driving model M for integration into a vehicle when data relating to the possible scenarios for which the driving model M has been invalidated is included in a first predetermined interval. The data is, for example, the rate (percentage) of invalidated scenarios. The first predetermined interval is, for example, between 0 and 5%.

[0081] - invalidation of the driving model M when data relating to possible scenarios for which the driving model M has been invalidated is included in a second predetermined interval. The second predetermined interval is different from the first predetermined interval. The invalidated model is, for example, used to be subsequently retrained on an enriched training database. The data is, for example, the rate (percentage) of invalidated scenarios. The second predetermined interval is, for example, between 5% and 100%.

[0082] - determining rules for using the driving model M as a function of at least at least one scenario for which the driving model M has been invalidated. The rules stipulate, for example, that a manual driving mode is activated when the input states St of the driving model M correspond to a scenario similar to a scenario for which the driving model M has been invalidated.

[0083] Thus, the present method makes it possible to evaluate the performance of a driving model M by testing the model on different scenarios and by applying a perturbation to each scenario. Such a method thus makes it possible to evaluate the safety and robustness of the driving model M. Depending on the results of the tests, the driving model M may be certified, modified, invalidated or certified by means of predefined usage rules.

[0084] In particular, such a method implements an adaptation of the principle of abstract interpretation for all types of models (including models obtained by reinforcement) in order to evaluate the robustness of the model. In addition, such a method also makes it possible to define critical situations in terms of safety for each scenario without taking into account how the driving model M, learned and in which environment.

[0085] Such a method is particularly suitable for evaluating the performance of models obtained by reinforcement learning, for which several tolerated decisions can be taken by a model.

[0086] Those skilled in the art will understand that the previously described embodiments and variations may be combined to form new embodiments provided that they are technically compatible.

[0087] Furthermore, it should be noted that the order of the different steps of the evaluation phase 110 is given as an example in the description. Thus, this order is likely to be modified as long as the sequence of steps is technically compatible. For example, step 110B can be carried out before step 110A, or step 110B which would be carried out after steps HOC and 110D.

Claims

1. Claims Method for evaluating the performance of a driving model (M) for a vehicle, the driving model (M) being a neural network previously trained to determine a driving action (At) of the vehicle, from a set of predefined actions (At), as a function of a set of states (St) characterizing a scenario in which the vehicle evolves in an environment, the driving model (M) having been trained according to a reinforcement learning technique, the method being implemented by computer and comprising the following phases: a. obtaining an evaluation database, the evaluation database comprising several scenarios in which the vehicle evolves in an environment, each scenario being characterized by a set of states (St) capable of taking different values ​​over time, each state (St) having an initial value (Vc) corresponding to the start time of the corresponding scenario, b. the evaluation of the performance of the driving model (M) for each scenario of the evaluation database, the evaluation phase comprising, for each scenario of the evaluation database, the following steps: i. the determination, from among the set of predefined actions (At), for each instant of the scenario considered, of the actions (At) tolerated for driving the vehicle according to the scenario considered and of the actions (At) critical for driving the vehicle according to the scenario considered, the actions (At) tolerated and critical for each scenario considered being determined according to safety constraints, ii. the determination, by the driving model (M), of a driving action (At) of the vehicle for each instant of the scenario considered, as long as the determined action is different from a critical action, iii. the determination, for at least one state (St) of the scenario considered, called the disturbed state (St*), of a lower bound (ôinf) and an upper bound (ôsup) relating to a disturbance of said disturbed state (St*), the disturbance extending over an interval of disturbance values, the lower bound (ôinf) and the upper bound (ôsup) being the outputs of a predetermined model whose inputs are the initial value (Vc) of the state and the disturbance interval, the initial value (Vc) of said disturbed state (St*) being between the lower bound (ôinf) and the upper bound (ôsup), iv. the determination of a set of scenarios, called limit scenarios, for the scenario considered, each limit scenario differing from the scenario considered by replacing the initial value (Vc) of at least one disturbed state (St*) by the lower bound (ôinf) or by the upper bound (ôsup ) of said disturbed state (St*), each disturbed state (St*) having as its value the lower bound (ôinf) of said disturbed state (St*) in at least one limit scenario, and having as its value the upper bound (ôsup) of said disturbed state (St*) in at least one other limit scenario, and v. the determination, by the driving model (M), of a driving action (At) of the vehicle for each instant of each determined limit scenario, as long as the determined action is different from a critical action, vi. validation of the driving model (M) for the scenario considered when, for each instant, the action (At) determined for the scenario considered and the actions (At) determined for the limit scenarios correspond to the same action (At) among the actions (At) tolerated for the scenario considered at that instant, the driving model (M) being invalidated for the scenario considered otherwise.

2. Method according to claim 1, in which when the driving model (M) is invalidated for the scenario considered, at least one corresponding invalidation reason is obtained which is chosen from the following reasons: a. an invalidation reason for non-compliance with safety constraints when at least one of the actions (At) determined for the scenario considered and the limit scenarios is a critical action (At), and b. an invalidation reason for lack of robustness when the action (At) determined for at least one instant of the limit scenario is different from the action (At) determined for the same instant of the scenario considered.

3. Method according to claim 1 or 2, in which the steps of determining limits, determining a set of limit scenarios, determining a driving action (At) for each limit scenario and validating the driving model (M) for the scenario considered are implemented only when the actions (At) determined for the scenario considered are tolerated actions (At), the driving model (M) otherwise being invalidated for the scenario considered for non-compliance with safety constraints.

4. Method according to any one of claims 1 to 3, in which the scenarios of the evaluation database are such that for at least one instant of a scenario, several distinct tolerated actions are determined for driving the autonomous vehicle.

5. A method according to any one of claims 1 to 4, wherein the security constraints comply with ISO / PAS 21448:2019 SOTIF.

6. Method according to any one of claims 1 to 5, in which the states (St) characterizing each scenario comprise the position and the speed of the vehicle, as well as the relative position and, where appropriate the relative speed, of at least one element in the environment of the vehicle.

7. Method according to claim 6, in which the disturbed states (St*) are chosen from the position and the speed of the vehicle.

8. A method according to any one of claims 1 to 7, wherein at the end of the evaluation phase, the method comprises one of the following phases: a. the validation and / or certification of the driving model (M) for integration into a vehicle when data relating to the possible scenarios for which the driving model (M) has been invalidated is included in a first predetermined interval, or b. invalidation of the driving model (M) when data relating to the possible scenarios for which the driving model (M) has been invalidated is included in a second predetermined interval, the second predetermined interval being different from the first predetermined interval, or c. determining rules for using the driving model (M) based on at least one scenario for which the driving model (M) has been invalidated.

9. A method according to any one of claims 1 to 8, wherein the driving model (M) has been trained using a deep reinforcement learning technique.

10. A computer program product comprising a readable information medium, on which is stored a computer program comprising program instructions, the computer program being loadable onto a data processing unit and causing the implementation of a method according to any one of claims 1 to 9 when the computer program is implemented on the data processing unit.

Images