Aircraft piloting aid method, electronic piloting aid device and associated assistance system

Abstract

Method for assisting the piloting of an aircraft, electronic piloting aid device, and associated assistance system. The present invention relates to a method for assisting the piloting of an aircraft. The method comprises acquiring (110) an aircraft piloting model and a reward function including a piloting constraint, and applying (130) a reinforcement learning algorithm to the piloting model to obtain state variables and piloting commands. The method further comprises training group(s) (140) of data from the state variables and the commands. For each group, the method comprises assigning (150) at least one aircraft state to the state variables and at least one piloting action to the commands, to generate a piloting rule comprising the state(s) and piloting action(s).The method also includes transmitting (170) at least one piloting rule to a display device for display to an aircraft pilot. Figure for abbreviation: [Figure 4].

Description

Title of the invention: Method for assisting the piloting of an aircraft, electronic piloting aid device and associated assistance system

[0001] The present invention relates to a method for assisting in the piloting of an aircraft.

[0002] The present invention also relates to an electronic aid device for aircraft piloting and an aircraft piloting assistance system including such an electronic piloting assistance device.

[0003] The invention relates to the field of assistance to an aircraft pilot.

[0004] A method for managing conflict between aircraft is known from document WO 2020 / 079702 Al.

[0005] For piloting a motor vehicle, it is known to use autopilots configured to determine piloting commands based on state variables measured by vehicle sensors. The commands are then implemented in the vehicle for its piloting.

[0006] To this end, it is known to use a steering model comprising a neural network, to which a reinforcement learning algorithm is applied. Such a reinforcement learning algorithm allows the model to learn by itself to determine the steering commands from the state variables and a reward function quantifying the effect of the determined commands on the steering of the vehicle.

[0007] However, in the aeronautical context, it is generally recommended, even necessary, to obtain certification for the flight control system. Certification ensures that the flight control system will not determine an inconsistent flight command that could jeopardize the integrity of the aircraft and its passengers. However, flight control systems incorporating a model of the aforementioned type are not currently certifiable.

[0008] The invention therefore aims to provide a method for assisting the piloting of an aircraft, an electronic piloting aid device and an associated piloting assistance system, capable of providing certifiable piloting rules.

[0009] To this end, the invention relates to a method for assisting in the piloting of an aircraft, the method being implemented by an electronic piloting assistance device and comprising the following steps:

[0010] - acquisition of an aircraft piloting model and a reward function including a piloting constraint,

[0011] - application, to the control model, of a learning algorithm by ren- necessarily starting from the reward function, the application step includes the following sub-steps:

[0012] + a reception of aircraft state variable(s) at reception times,

[0013] + for each reception instant, a modification of the model from a evaluation of the reward function based on the state variable(s) received at the time of receipt, and

[0014] + for each reception moment, a determination of piloting commands to based on the modified control model and the state variables received at that instant,

[0015] - formation of data group(s) from the received state variables and the determined commands, each data group comprising state variables and commands corresponding to a plurality of successive reception moments,

[0016] - for the data group(s), assignment of at least one aircraft state to state variables and at least one control action to the commands, to generate a control rule comprising at least one state and at least one control action,

[0017] - transmission of at least one control rule to a display device in order to its display intended for an aircraft pilot.

[0018] Each piloting rule generated by the process is certifiable since it complies with a formalism enabling certification, namely associating with at least one state of the aircraft, a piloting action to be implemented.

[0019] According to other advantageous aspects of the invention, the piloting assistance method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0020] - during the application step, for each reception instant, a value of the The steering constraint is calculated during the evaluation of the reward function.

[0021] during the group formation step, each data group further includes the values ​​of the control constraint associated with the state variables and the commands,

[0022] during the assignment step, for the group or each group, at least one effect on the constraint is assigned to the values ​​of the group's control constraint, each rule further including at least one effect on the constraint;

[0023] - during the assignment step a plurality of rules is formed, the process including, furthermore, between the assignment stage and the supply stage, a step of identifying the main rule(s) among the plurality of control rules generated by applying a variable frequency analysis algorithm,

[0024] during the transmission step, only the main rule(s) being transmitted;

[0025] - the identification step further includes comparing the effect(s) on the constraining each main rule to a predetermined threshold to obtain at least one filtered rule, each filtered rule being a respective main rule including at least one effect on the constraint greater than or equal to the threshold,

[0026] during the transmission step, only the filtered rule(s) being preferably provided for display to the aircraft pilot;

[0027] - the assignment step includes for each data group:

[0028] - a calculation of a first difference between the value of the piloting constraint at the last reception instant among the reception instants of the state variables in the data group, and the driving constraint value at the first reception instant among the reception instants of the state variables in the data group,

[0029] - a calculation of a second difference between the value of the piloting constraint at a the reception instant subsequent to the last reception instant among the reception instants of the state variables in the data group, and the driving constraint value at the first reception instant among the reception instants of the state variables in the data group,

[0030] - an assignment, to the values ​​of the piloting constraint, of a short-term effect on the steering constraint, based on the first difference and a table of correspondence between steering constraints, and

[0031] - an assignment, to the values ​​of the control constraint, of a long-term effect on the steering constraint, based on the second difference and the table of correspondence of steering constraints;

[0032] - the training step includes:

[0033] - classification of pilot commands determined from among a plurality of classes predefined and via a command mapping table,

[0034] - for each class, grouping of the piloting controls belonging to said class and determined for the reception moments forming the longest possible sequence of consecutive reception moments, to form at least one set of grouped commands, and

[0035] - for each set of grouped commands, formation of a respective group including the commands of said set and the state variables received at the times of reception for which said commands were determined;

[0036] - during the acquisition stage, a preliminary reward function is acquired,

[0037] the method further comprising, between the acquisition step and the application step, a model training step by applying a preliminary reinforcement learning algorithm to the piloting model based on the preliminary reward function,

[0038] the training step involving a modification of the model based on an evaluation of the preliminary reward function;

[0039] - the state variables are chosen from the group consisting of: a roll angle of the aircraft, an aircraft pitch angle, an aircraft yaw angle, an aircraft speed, an aircraft acceleration, an aircraft wind speed at the aircraft contact, a wind direction relative to the aircraft, and an aircraft position.

[0040] The invention also relates to a computer program product comprising software instructions which, when implemented by a computer, implement a process according to any one of the preceding claims.

[0041] The invention also relates to an electronic device for piloting an aircraft, comprising:

[0042] - an acquisition module configured to acquire an aircraft piloting model and a reward function including a steering constraint,

[0043] - an application module configured to apply to the control model, an al Reinforcement learning algorithm based on the reward function, the application module comprising:

[0044] + a receiving unit configured to receive at least one state variable of the aircraft at times of reception,

[0045] + a modification unit configured to modify, for each instant of reception, the model based on an evaluation of the reward function from the state variable(s), and

[0046] + a unit of determination configured to determine, for each instant of reception of control commands from the modified control model and state variables received at the time of reception,

[0047] - a training module configured to train at least one group of data to based on the received state variables and determined commands, each data group comprising the state variables and determined commands corresponding to a plurality of successive reception moments,

[0048] - an assignment module configured to assign, for the or each group of data, at least one aircraft state with received state variables and at least one piloting action with determined controls, and to generate a piloting rule comprising at least one state and at least one piloting action, and

[0049] - a transmission module configured to transmit, to a display device, at least one piloting rule for display to an aircraft pilot.

[0050] Furthermore, the invention relates to an aircraft piloting assistance system comprising such an electronic piloting assistance device and a display device configured to receive at least one piloting rule from the piloting assistance device and to display said rule to the aircraft pilot.

[0051] These features and advantages of the invention will become clearer at the reading the following description, given solely as a non-limiting example and made with reference to the attached drawings, on which:

[0052] [Fig.1] [Fig.1] is a schematic view of an aircraft piloting assistance system according to the invention, comprising an electronic piloting assistance device according to the invention;

[0053] [Fig.2] [Fig.2] is a representation of a classification of commands of piloting determined by the electronic piloting assistance device of the [Fig.l];

[0054] [Fig. 3] [Fig. 3] is a view of an aircraft trajectory calculated by the flight assistance system of [Fig. 1]; and

[0055] [Fig.4] [Fig.4] is a flowchart of a pilot assistance process according to the invention, implemented by the piloting assistance device of the piloting assistance system of the [Fig.1].

[0056] With reference to [Fig. 1], an aircraft piloting assistance system 10 is described. The assistance system 10 is configured to provide piloting rules to an aircraft pilot 15. The assistance system 10 is connected to a state variable generation device 20 of the aircraft, which will be described below.

[0057] In the remainder of this description, "aircraft" means a flying machine capable of being piloted by the pilot 15, such as an airplane, a helicopter, or a drone. In particular, the pilot 15 of the aircraft is either on board the aircraft, especially in the case of an airplane or a helicopter, or remotely from said aircraft, especially in the case of a drone.

[0058] The assistance system 10 includes an electronic piloting aid device 25 and a display device 30.

[0059] The state variable generation device 20 is, for example, an aircraft simulation environment. The generation device 20 is configured to simulate the aircraft's behavior along a trajectory with waypoints. More specifically, the generation device 20 is configured to simulate the aircraft's behavior between an initial position and a final position. The final position preferably corresponds to the last waypoint of the trajectory. If the aircraft has reached the final position or after the expiration of a predefined maximum time, the generation device 20 restarts the simulation by repositioning the aircraft to the initial position.

[0060] The generation device 20 is then configured to receive flight control commands Ci from the electronic flight control device 25. The generation device 20 is then configured to simulate, for a predefined period, the aircraft's behavior following the implementation of the received flight control commands Ci. The generation device 20 is configured to provide the flight control device 25 with the aircraft's state variables VE following the simulation. Simulation of aircraft behavior over a predefined period. The predefined period is, for example, 200 ms.

[0061] The flight control commands C; are instructions, or commands, intended to be received by aircraft actuators and enabling the piloting of the aircraft.

[0062] In the embodiment in which the aircraft is an airplane, the flight controls Ci are, for example, chosen from the group consisting of:

[0063] - a position of the aircraft's yaw rudder,

[0064] - a position of the aircraft's pitch control surface,

[0065] - a position of the aircraft flaps,

[0066] - a position of the aircraft's ailerons, and

[0067] - a fuel flow in aircraft engines.

[0068] In the embodiment in which the aircraft is a drone, the piloting controls Ci are, for example, the rotation speeds of each rotor of the drone.

[0069] The aircraft state variables VE are the variables used to define the aircraft's state in its environment. The aircraft state variables VE are, for example, chosen from the group consisting of:

[0070] - an aircraft roll angle,

[0071] - an aircraft pitch angle,

[0072] - a yaw angle of the aircraft,

[0073] - an acceleration of the aircraft,

[0074] - an aircraft speed,

[0075] - a position of the aircraft, such as a relative position of the aircraft with respect to the next crossing point or the next two crossing points, and optionally

[0076] - a wind speed at the aircraft's contact point, and

[0077] - a wind direction relative to the aircraft.

[0078] It is then understood that the implementation of piloting commands Q, by the generation device 20, and the simulation of the behavior of the aircraft during the predefined duration lead to a variation of the state variables VE;.

[0079] According to one embodiment, the generation device 20 is included in the aircraft and comprises a set of aircraft sensors and actuators. Thus, the generation device 20 includes actuators for implementing the received flight commands C; and sensors for measuring the aircraft's state variables VE; after the predefined time.

[0080] The electronic piloting aid device 25 is connected to the generation device 20 and the display device 30. The piloting aid device 25 is configured to generate piloting rules from the generation device 20 and to provide them to the display device 30, as will be detailed below.

[0081] The aid device 25 comprises an acquisition module 45, a module application module 50, training module 55, assignment module 60, transmission module 65 and optionally training module 70 and identification module 75.

[0082] In the embodiment shown in [Fig. 1], the acquisition module 45, the application module 50, the training module 55, the assignment module 60, the supply module 65, and optionally the training module 70 and the identification module 75, are each implemented as a software program, or a software component, executable by a processor 77. A memory 76 of the aid device 25 is then capable of storing an acquisition software program, an application software program, a training software program, a supply software program, and optionally a training software program, and an identification software program.

[0083] In an alternative not shown, the acquisition module 45, the application module 50, the training module 55, the assignment module 60, the supply module 65, and optionally the training module 70 and the identification module 75, are each implemented as a programmable logic component, such as an FPGA (Field Programmable Gate Array), or as an integrated circuit, such as an ASIC (Application-Specific Integrated Circuit).

[0084] When the assistance device 25 is implemented in the form of one or more software programs, i.e., in the form of a computer program, it is also capable of being stored on a computer-readable medium (not shown). A computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. For example, a readable medium is an optical disc, a magneto-optical disc, a ROM, a RAM, any type of non-volatile memory (e.g., EPROM, EEPROM, FLASH, NVRAM), a magnetic card, or an optical card. A computer program comprising software instructions is then stored on the readable medium.

[0085] The acquisition module 45 is configured to acquire an aircraft piloting model, a reward function FR including a piloting constraint, and optionally a command mapping table, a state mapping table, and a preliminary reward function FRP not including the piloting constraint and a mapping table of effects on the piloting constraint.

[0086] The flight control model is designed to receive as input the aircraft state variables VE and to provide as output the aircraft flight control commands Ci. The flight control model is, for example, an artificial neural network comprising an input layer with a number of neurons equal to the number of state variables VE. The model further comprises one or more hidden layers, each comprising a plurality of neurons, and an output layer (from the English "output layer") containing a number of neurons equal to the number of control commands C;. The model further includes connections between the neurons of the different successive layers, each connection having an adjustable weight, also called synaptic weight.

[0087] The reward function is a function that takes as input the state variables VE; and provides as output a numerical value. The reward function includes the control constraint representing a control intention imposing compliance with constraint(s).

[0088] According to a first example, the piloting intention is a flight responsible for the environment. In this example, the piloting constraint is an environmental constraint.

[0089] According to a second example, the piloting intention is a shortest flight, the associated constraint then being a flight time constraint.

[0090] The reward function FR increments, at each of a plurality of iterations, by the following magnitude:

[0091] [Math.l] RT + ROA + CP

[0092] where RT is a trajectory reward, for example equal to 0 if the aircraft's position is not on a waypoint, 10 if the aircraft's position is on a waypoint, and -2 if the aircraft missed the waypoint, i.e., if the aircraft's position is in the vicinity of a waypoint without being on the waypoint, and

[0093] ROA is the acceleration orientation reward, for example equal to ] _ ABS^(accelerationX + 1 ) $)' ABS() denotes the absolute value function and accelerationX denotes the magnitude of the aircraft's acceleration taking gravity into account, and

[0094] CP is the steering constraint.

[0095] For example, the quantity accelerationX denotes the magnitude of the acceleration along the axis connecting the wingtips of the two aircraft wings. ROA is therefore a constraint that prevents aircraft passengers from being thrown to the side of the aircraft during a turn. ROA thus allows the aircraft to exhibit more or less realistic behavior.

[0096] According to the first example, the control constraint is, for example, equal to:

[0097] [Math.2] CP = projection, 10'14

[0098] where projection is the dot product between the speed of the aircraft and the wind speed.

[0099] This control constraint therefore makes it possible to increase the value of the re function compensates when the aircraft is carried by the wind and its fuel consumption is reduced.

[0100] According to the second example, the piloting constraint is, for example, equal to:

[0101] [Math.3] CP = y

[0102] where Y is a positive constant.

[0103] According to the second example, at each iteration, the reward function FR is incremented by the value F. Thus, the path that minimizes the reward function is the one that minimizes the number of iterations, i.e. the one that leads to the shortest flight.

[0104] The command correspondence table includes value ranges for the pilot commands C;, and associates each range with a pilot action.

[0105] Similarly, the state correspondence table includes ranges of values ​​for the state variables VE; and associates with each of said ranges, a state of the aircraft.

[0106] Similarly, the table of correspondence of effects on the pilot constraint includes ranges of values, for the values ​​of the pilot constraint CP; and associates with each of said ranges, a short-term effect on the pilot constraint and a long-term effect on the pilot constraint.

[0107] The preliminary reward function FRP is a function that takes the state variables VE as input and provides a numerical value as output. For example, the preliminary reward function FRP increments at each iteration by the following quantity:

[0108] [Math.4] RT + RO A + RA

[0109] where RT is the trajectory reward defined above,

[0110] RO A is the acceleration orientation reward defined above, and [YES] RA is an alignment reward, for example equal to ] + ( alignment + 2 ) where alignment is the dot product between the aircraft speed and the direction between the aircraft and the next waypoint.

[0112] The application module 50 is configured to apply a reinforcement learning algorithm based on the reward function to the piloting model. In particular, the application module 50 is configured to receive the aircraft's state variables VE and determine the piloting commands Ci at a plurality of successive reception times T.

[0113] For this purpose, the application module 50 includes a first receiving unit 81, a first modification unit 82, and a first determination unit 83.

[0114] The first receiving unit 81 is configured to receive, from the generation device 20, the state variables VE; of the aircraft at successive reception times Tj.

[0115] The first modification unit 82 is configured to modify, for each reception instant T;, the model from an evaluation of the reward function from the state variables VE;. To evaluate the reward function, the first modification unit 82 is configured to calculate the value of the pilot constraint from the state variables VE;, for example according to one of the equations (2) or (3).

[0116] The first modification unit 82 is further configured to modify the connection weights of the model based on the evaluation of the reward function. Such a modification is known to those skilled in the art as a classical technique for applying the reinforcement learning algorithm.

[0117] The first determination unit 83 is configured to determine, for each reception instant T;, the control commands Ci from the modified control model and the state variables VE; received at said reception instant T;. To this end, the first determination unit 83 is configured to apply the modified model to the received state variables VE; at each reception instant T;, and to determine the control commands Ci at said reception instant T; as being equal to the output values ​​of the modified model.

[0118] Optionally, the application module 50 is configured to perform a predetermined number of iterations, each iteration comprising the reception of the aircraft state variables VE; by the first receiving unit 81, the modification of the model by the first modification unit 82, and the determination of the flight controls C; by the first determination unit 83.

[0119] Advantageously, the application module 50 is configured not to apply reinforcement learning to the flight model for a predefined number of iterations. Thus, after the predefined number of iterations, the convergence weights generally have not reached convergence. The application module 50 is preferably configured to explore possible aircraft trajectories, taking into account modifications to the model based on the reward function, and in particular the flight constraint included in the reward function.

[0120] The application module 50 is further configured to store, in a database and for each reception instant T;, the received state variables VE;, the determined control commands Ci, and preferably the values ​​of the control constraint CP; calculated during the evaluation of the reward function.

[0121] The training module 55 is configured to train at least one data group from the received state variables VE;, the determined commands C; and optionally the calculated pilot constraint values ​​CP;. Each data group comprises the state variables VE;, the pilot commands Q, and optionally the pilot constraint values ​​CP; corresponding to successive reception times Tj.

[0122] The training module 55 is, for example, configured to train a plurality of data groups. As an example, the application module is configured to apply a classification algorithm to the control commands determined Q by the application module 50, in order to classify said control commands determined C into predefined classes CL; for each reception time Tj. The predefined classes CL; are, for example, included in the command correspondence table.

[0123] For this purpose, the training module 55 is for example configured to compare, for each reception instant T;, the pilot commands Ci to the value intervals of the command correspondence table, in order to determine to which class CL; correspond the pilot commands Ci of said instant T;.

[0124] The training module 55 is configured for example to group the pilot commands Ci belonging to the same respective class CL; and whose reception times T; form a sequence of reception times T; consecutive as long as possible.

[0125] With reference to [Fig.2], several pilot commands Cb C2, C3, C4, C5, C22, C23, C24, C45, C46, ​​C47 belong to a first class CL. These commands Ci, C2, C3, C4, C5, C22, C23, C24, C45, C46, ​​C47 correspond respectively to the following reception times T: Tb T2, T3, T4, T5, T22, T23, T24, T45, T46, T47. The training module 55 is configured to then group together, firstly, the following pilot commands: Ci, C2, C3, C4, C5, secondly, the following pilot commands: C22, C23, C24, and thirdly, the following pilot commands: C45, C46, ​​C47.

[0126] The training module 55 is further configured to train each data group as comprising a plurality of respective grouped commands C, the corresponding state variables VE, and optionally the corresponding control constraint values ​​CP. The term "corresponding state variables VE" refers to the state variables VE received at the reception times T corresponding to the grouped commands C. Similarly, the term "corresponding control constraint values ​​CP" refers to the control constraint values ​​CP calculated for the reception times T corresponding to the grouped commands C.

[0127] Thus, in the previous example of [Fig.2], a first group of data includes the following commands: Cb C2, C3, C4, C5, the corresponding VE state variables: VEi, VE2, VE3, VE4, VE5, and optionally the corresponding control constraint values: CPi, CP2, CP3, CP4, CP5.

[0128] Similarly, a second data group comprises the following commands: C22, C23, C24, the corresponding VE state variables: VE22, VE23, VE24, and optionally the corresponding control constraint values: CP22, CP23, CP24. A third group comprises the following commands: C45, C46, ​​C47, the corresponding VE state variables: VE45, VE46, VE47, and optionally the corresponding control constraint values: CP45, CP46, CP47.

[0129] The assignment module 60 is configured to assign, for the or each data group: at least one aircraft state to the VE state variables;, at least one piloting action to the C piloting controls; and optionally at least one effect on the piloting constraint to the CP piloting constraint values;.

[0130] The assignment module 60 is configured, for example, to assign to the commands Ci of each group at least one action from the command mapping table. The command mapping table includes, for example, for each predefined class CL, at least one action. The assignment module 60 is preferably configured to assign to the commands Ci of each group the action(s) corresponding to the class CL of these commands Ci in the command mapping table.

[0131] Each action is, for example, a text label describing the piloting action performed by the respective piloting controls C;. By way of example, the actions are chosen from the group consisting of:

[0132] - a right turn of the aircraft,

[0133] - a left turn of the aircraft,

[0134] - a straight trajectory of the aircraft,

[0135] - an ascent of the aircraft,

[0136] - a descent of the aircraft,

[0137] - an acceleration of the aircraft,

[0138] - a deceleration of the aircraft, and

[0139] - maintaining aircraft speed.

[0140] It is clear that the assignment module 60 is, for example, configured to assign several actions to the Ci commands of a data group.

[0141] Figure 3 shows an aircraft's route along a trajectory comprising a first PP1, a second PP2, a third PP3, and a fourth PP4 waypoint. In the example in Figure 3, a dashed portion of the route corresponds to a data group.

[0142] In the example of [Fig.3], the assignment module 60 is configured for example to assign to the Q commands of the group corresponding to portion 85, the following actions: a left turn of the aircraft, a descent of the aircraft, and a deceleration of the aircraft.

[0143] The assignment module 60 is configured, for example, to assign to the state variables VE; at least one state from the state mapping table. Each state is, for example, a text label describing the aircraft's state according to the state variables VE; of said aircraft. By way of example, the states are chosen from the group consisting of:

[0144] - the aircraft is very close to the next waypoint,

[0145] - the aircraft is nearing the next waypoint,

[0146] - the aircraft is far from the next waypoint,

[0147] - the aircraft is very far from the next waypoint,

[0148] - the aircraft is far to the left of the next waypoint,

[0149] - the aircraft is slightly to the left of the next waypoint,

[0150] - the aircraft is slightly to the right of the next waypoint,

[0151] - the aircraft is far to the right of the next waypoint,

[0152] - the aircraft is very high above the next waypoint,

[0153] - the aircraft is slightly above the next waypoint,

[0154] - the aircraft is slightly below the next waypoint,

[0155] - the aircraft is well below the next waypoint,

[0156] - the aircraft is very close to the next waypoint,

[0157] - the aircraft is near the next waypoint,

[0158] - the aircraft is far from the next waypoint,

[0159] - the aircraft is very far from the next waypoint,

[0160] - the aircraft is far to the left of the next waypoint,

[0161] - the aircraft is slightly to the left of the next waypoint of passage,

[0162] - the aircraft is slightly to the right of the next waypoint of passage,

[0163] - the aircraft is far to the right of the next waypoint,

[0164] - the aircraft is very high above the next waypoint of passage,

[0165] - the aircraft is slightly above the next waypoint of passage,

[0166] - the aircraft is slightly below the next waypoint of passage,

[0167] - the aircraft is well below the next waypoint of passage,

[0168] - the aircraft is very close to the previous waypoint,

[0169] - the aircraft is close to the previous waypoint,

[0170] - the aircraft is far from the previous waypoint,

[0171] - the aircraft is very far from the previous waypoint,

[0172] - the aircraft is far to the left of the previous waypoint,

[0173] - the aircraft is slightly to the left of the previous waypoint,

[0174] - the aircraft is slightly to the right of the previous waypoint,

[0175] - the aircraft is far to the right of the previous waypoint,

[0176] - the aircraft is well above the previous waypoint,

[0177] - the aircraft is slightly above the previous waypoint,

[0178] - the aircraft is slightly below the previous waypoint,

[0179] - the aircraft is well below the previous waypoint,

[0180] - the roll angle is strongly negative,

[0181] - the roll angle is slightly negative,

[0182] - the roll angle is slightly positive,

[0183] - the roll angle is strongly positive,

[0184] - the pitch angle is strongly negative,

[0185] - the pitch angle is slightly negative,

[0186] - the pitch angle is slightly positive,

[0187] - the pitch angle is strongly positive,

[0188] - the wind at the aircraft's contact point is very strong,

[0189] - the wind at the aircraft's contact point is strong,

[0190] - the wind at the aircraft's contact point is weak,

[0191] - the wind at the aircraft's contact point is very weak,

[0192] - the aircraft has a headwind,

[0193] - the aircraft has a tailwind,

[0194] - the aircraft has a crosswind from the right, and

[0195] - the aircraft has a crosswind from the left.

[0196] The assignment module 60 is, for example, further configured to calculate, for each group, a time average of the value of each state variable VE of the group. The "time average" is understood to be the average calculated between all reception times T of the state variables VE of the group.

[0197] The assignment module 60 is configured to then assign to the VE state variables; of each group, at least one state by comparison of the average values ​​of the VE state variables; to the intervals corresponding to each state in the state matching table.

[0198] It is clear that the assignment module 60 is, for example, configured to assign several states to the state variables VE; of a data group.

[0199] In the example of [Fig.3], the assignment module 60 is configured for example to assign to the state variables VE; of the group corresponding to portion 85, the following states: the aircraft is close to the next waypoint, the aircraft is slightly above the next waypoint, the aircraft is slightly to the right of the next waypoint, the aircraft is far from the previous waypoint, the aircraft is far above the previous waypoint, the aircraft is far to the right of the previous waypoint, the sine of the roll angle is strongly negative, the wind at the contact of the aircraft is weak, and the aircraft has a headwind.

[0200] The assignment module 60 is further configured to generate, for each group, a control rule comprising at least one state and at least one control action, assigned to the group.

[0201] In the example in [Fig.3], the pilot rule generated for portion 85 is, for example:

[0202] - when the aircraft is near the next waypoint, the aircraft is slightly Above the next waypoint, the aircraft is slightly to the right of the next waypoint; the aircraft is far from the previous waypoint; the aircraft is well above the previous waypoint; the aircraft is well to the right of the previous waypoint; the sine of the roll angle is strongly negative; the wind at the aircraft's contact point is weak; and the aircraft has a headwind.

[0203] - perform: a left turn of the aircraft, a descent of the aircraft, and a aircraft deceleration.

[0204] As an optional complement, the assignment module 60 is further configured to calculate, for each group, a first difference between the value of the control constraint CPd at the last reception instant Td among the reception instants T; of the state variables VE; of the data group and the value of the control constraint CPp at the first reception instant Tp among the reception instants T; of the state variables VE; of the data group.

[0205] The first reception instant Tp is the first, i.e., the earliest of the reception instants T of the state variables VE of the group. Similarly, the last reception instant Td is the last, i.e., the most recent of the reception instants T of the state variables VE of the group.

[0206] The assignment module 60 is configured to then assign to the values ​​of the piloting constraint CP;, a short-term effect on the piloting constraint, based on the first difference and the piloting constraint mapping table. Each short-term effect is, for example, a text label describing whether the action(s) implemented, when the aircraft is in the state(s), is positive or negative on the piloting constraint within a short time. As an example, the short-term effects are chosen from a group consisting of:

[0207] - a very negative short-term effect on piloting constraint,

[0208] - a negative short-term effect on piloting constraint,

[0209] - a positive short-term effect on piloting constraint, and

[0210] - a very positive short-term effect on the piloting constraint.

[0211] According to this same optional supplement, the assignment module 60 is further configured to calculate, for each group, a second difference between the value of the piloting constraint CPfinai at a reception time Tfinai subsequent to the last reception time Td among the reception times T; of the state variables VE; of the data group and the value of the piloting constraint CPp at the first reception time Tp among the reception times T; of the state variables VE; of the data group. The reception time Tfinai subsequent to said last reception time Td is, for example, the last reception time T; of the trajectory simulation. In other words, said time Tfinai is the last reception time T; before the aircraft is repositioned at the initial point by the generation module 20. On the [Fig.3], this instant T; is that at which the aircraft reached the fourth waypoint PP4, or final position of the trajectory.

[0212] The assignment module 60 is then configured to assign, to the values ​​of the piloting constraint CP, a long-term effect on the piloting constraint, based on the second difference and the piloting constraint correspondence table. Analogously to the short-term effects, each long-term effect is, for example, a text label describing whether the action(s) implemented, when the aircraft is in the state(s), has a positive or negative effect on the piloting constraint over a long period. For example, the long-term effects are chosen from the group consisting of:

[0213] - a very negative long-term effect on piloting constraint,

[0214] - a negative long-term effect on piloting constraint,

[0215] - a positive long-term effect on piloting constraint, and

[0216] - a very positive long-term effect on piloting constraint.

[0217] According to this optional add-on, assignment module 60 is configured to generate, when generating the control rule, the rule including both the short-term effect on the control constraint and the long-term effect on the control constraint. Thus, in the example in [Fig. 3], the rule generated for portion 85 is, for example:

[0218] - when the aircraft is near the next waypoint, the aircraft is slightly above the next waypoint, the aircraft is slightly to the right of the next waypoint, the aircraft is far from the previous waypoint, the aircraft is well above the previous waypoint, the aircraft is well to the right of the previous waypoint During the passage, the sine of the roll angle is strongly negative, the wind at the aircraft's contact point is weak, and the aircraft has a headwind.

[0219] - perform: a left turn of the aircraft, a descent of the aircraft, and a aircraft deceleration,

[0220] - has an effect on the piloting constraint: very positive in the short term, and negative in the long term term.

[0221] The transmission module 65 is configured to transmit, to the display device 30, the piloting rules for display to the pilot 15 of the aircraft.

[0222] The training module 70 is configured to train the flight control model to follow the trajectory by applying a preliminary reinforcement learning algorithm to the model, based on the preliminary reward function. The training module 70 comprises a second receiving unit 91, a second modifying unit 92, and a second determining unit 93.

[0223] The second receiving unit 91 is analogous to the first receiving unit 81. The state variables received by the second receiving unit 91 are referred to as preliminary state variables in this description. The second receiving unit 91 is then configured to receive the preliminary state variables at preliminary reception times.

[0224] The second modification unit 92 is substantially analogous to the first modification unit 82 except that the second modification unit 92 is configured to modify the model from an evaluation of the preliminary reward function instead of the reward function.

[0225] The second determination unit 93 is analogous to the first determination unit 83. The pilot commands determined by the second determination unit 93 are configured to determine and are referred to as preliminary commands in this description. The second determination unit 93 is then configured to determine said preliminary commands.

[0226] Unlike the application module 50, the training module 70 is configured to receive the preliminary state variables, modify the model and determine the preliminary commands at several successive preliminary times until the preliminary reinforcement learning algorithm converges, i.e. until the model weights are no longer substantially modified by the second modification unit 92. The model is then said to be trained.

[0227] The training module 70 is further configured to transmit the trained model to the application module 50. The application module 50 is then configured to apply the reinforcement learning algorithm to the trained model rather than to the model acquired by the acquisition module 45.

[0228] The identification module 75 is optionally configured to identify at least one main rule among the plurality of generated rules if the rules include at least one effect on the constraint. To this end, the identification module 75 is advantageously configured to apply a variable frequency analysis algorithm, known per se, based on a predefined support threshold and a predefined confidence threshold.

[0229] The support quantifies the frequency of occurrence of a triplet "state(s), action(s), effect(s) on the constraint" among all the rules. The support is between 0 and 1. For example, if the support threshold is equal to 0.5, then each triplet "state(s), action(s), effect(s) on the constraint" of the main rule(s) appears in at least 50% of the control rules.

[0230] Confidence quantifies the frequency of occurrence of a triplet "state(s), action(s), effect(s) on the constraint" among the rules comprising said state(s) and action(s). Confidence is between 0 and 1. For example, if the confidence level is equal to 0.9, then for each main rule "state(s), action(s), effect(s) on the constraint", the effect(s) on the constraint appear in at least 90% of the control rules comprising said state(s) and action(s).

[0231] Thus, the identification module 75 is configured to identify, among the steering rules, the main rule(s) whose support and confidence are above the respective thresholds.

[0232] Optionally, the identification module 75 is further configured to, if the rules include at least one effect on the constraint, compare the effects on the constraint of each main rule to a respective predetermined threshold, in order to obtain at least one filtered rule. The predetermined threshold is, for example, equal to: very positive effect, positive effect, or negative effect. More specifically, the identification module 75 is optionally configured to select only the main rules in which at least one of the effects on the control constraint is greater than or equal to the predetermined threshold. In this description, "very positive" is considered to be greater than or equal to: very positive, positive, negative, and very negative; "positive" is considered to be greater than or equal to: positive, negative, and very negative; and "negative" is considered to be greater than or equal to: negative and very negative.

[0233] If the helper device 25 includes the identification module 75, the transmission module 65 is configured to transmit, to the display device 30, only each main rule or each filtered rule.

[0234] The display device 30 is for example carried in the aircraft if the aircraft is an airplane, or remote from the aircraft if the aircraft is a drone.

[0235] The display device 30 includes, for example, a display screen 95 and optionally a processing unit 97 connected to the display screen 95.

[0236] The display screen 95 is for example suitable for displaying content transmitted by the processing unit 97.

[0237] The processing unit 97 is configured to receive at least one control rule from the assistance device 25. For example, the processing unit 97 is configured to implement a chatbot to interact with the pilot 15. The processing unit 97 is configured to display the received control rule(s) on the display screen 95, for example, following an interaction with the pilot 15.

[0238] The operation of the assistance system 10, and more particularly of the assistance device 25, will now be described with reference to [Fig.4] representing a flowchart of a piloting assistance method according to the invention.

[0239] During an acquisition step 110, the acquisition module 45 acquires the aircraft piloting model, the reward function; and optionally the control mapping table, the state mapping table, the preliminary reward function and the piloting constraint effect mapping table.

[0240] Optionally, during a model training step 120, the training module 70 applies the preliminary reinforcement learning algorithm to the model, in order to train it to follow the trajectory.

[0241] For this purpose, during a first reception sub-step 122, the second reception unit 91 receives, from the generation device 20 and at the preliminary reception times, the preliminary state variables of the aircraft.

[0242] For each moment of preliminary reception, during a first modification substep 124, the second modification unit 92 evaluates the preliminary reward function from the received preliminary state variables, for example using the preceding equation (4). The second modification unit 92 then modifies the model weights, depending on the value of the preliminary reward function, according to a technique known per se.

[0243] For each moment of preliminary reception, during a first sub-step of determination 126, the second unit of determination 93 determines the preliminary commands, for example by applying the modified model to the preliminary state variables received.

[0244] For each preliminary acceptance instant, the training module 70 transmits the determined preliminary commands to the generation device 20. The training module 70 then waits to receive the aircraft's preliminary state variables from the generation device 20 at the next preliminary acceptance instant. Then, the training module 70 repeats the first substeps of acceptance 122, modification 124, and determination 126.

[0245] During the training step 120, the substeps of reception 122, modification 124 and determination 126 are repeated until convergence of the model is reached.

[0246] At the end of training step 120, the model is trained to follow the trajectory.

[0247] Then, during an application step 130, the application module 50 applies the reinforcement learning algorithm to the trained model.

[0248] For this purpose, during a second reception sub-step 132, the first receiving unit 81 receives, from the generation device 20 and at successive reception times T, the state variables VE of the aircraft.

[0249] Then, for each reception instant T, during a second modification substep 134, the first modification unit 82 evaluates the reward function from the received state variables VE, for example according to equation (1). To this end, the first modification unit 82 calculates the value of the driving constraint CP, for example according to one of equations (2) or (3). Next, analogously to the first modification substep 124, the first modification unit 82 modifies the weights of the trained model according to a technique known per se.

[0250] For each reception instant T;, during a second determination substep 136, the first determination unit 83 determines the pilot commands Ci by applying the modified model to the state variables VE;, for example in a manner analogous to what is done during the first determination substep 126.

[0251] During application step 130, for each reception time T, the application module 50 transmits the determined flight control commands Q to the generation device 20. The application module 50 then waits to receive the aircraft's state variables VE from the generation device 20 at the next reception time T. Then, the application module 50 repeats the second substeps of reception 132, modification 134, and determination 136.

[0252] For each reception instant T;, the received state variables VE;, the determined control commands Q, and advantageously the calculated value of the control constraint CP;, are stored in the database.

[0253] During the application step 130, the second substeps of reception 132, modification 134 and determination 136 are repeated until the predefined number of iterations is reached.

[0254] Alternatively, if the process does not include the optional training step 120, the application module 50 applies, during the application step 130, the reinforcement learning algorithm to the model acquired during the acquisition step 110.

[0255] During a training step 140, the training module 55 forms at least one group comprising, for consecutive reception times T, the state variables VE; received, the pilot commands Q determined; and optionally the calculated values ​​of the pilot constraint CP;.

[0256] For this purpose, training module 55 compares, for example, the pilot commands Ci determined for each reception instant T;, to the intervals of values ​​of the command correspondence table, in order to determine to which class CL; correspond the pilot commands Q determined at said instant Tj.

[0257] Then, the training module 55 groups the pilot commands C; belonging to the same respective class CL; and determined for the reception times T; forming the longest possible sequence of consecutive reception times T;

[0258] Then, training module 55 trains each group by including the grouped Q pilot commands, the corresponding VE state variables; and optionally the corresponding CP pilot constraint values;

[0259] During an assignment step 150, the assignment module 60 assigns, for each group, at least one action to the control commands Ci of the group, for example by comparing the control commands Ci to the value ranges of the action correspondence table.

[0260] The assignment module 60 also assigns, for each group, at least one state to the state variables VE; of the group. To do this, the assignment module calculates, for example, the time average of the value of each state variable VE; of the group and compares each time average to the value intervals of the state mapping table.

[0261] If each group also includes the values ​​of the CP pilot constraint, the assignment module 60 calculates, for each group, the first and second differences as described above. The assignment module 60 then assigns to the CP pilot constraint values ​​the short-term effect on the stress and the long-term effect on the stress, comparing the first and second differences to the value ranges in the stress effect correspondence table.

[0262] The assignment module 60 then generates, for each group, a rule including the state(s) of the group, the action(s) of the group; and optionally the short-term effect on the constraint and the long-term effect on the constraint of the group.

[0263] As an optional addition, if the rules include at least one respective effect on the control constraint, the identification module 75 identifies, during an identification step 160, at least one main rule among the generated rules. For example, the identification module 75 applies the variable frequency analysis algorithm, defined above, to the generated rules.

[0264] During identification step 160, the identification module 75 optionally compares the effects on the pilot constraint included in each main rule at the predetermined threshold, to obtain at least one filtered rule. Thus, only the main rules having a very positive, at least positive, or at least negative effect on the constraint are, for example, among the filtered rules.

[0265] During a transmission step 170, the transmission module 65 transmits to the display device 30 the generated rules; or where applicable at least one main rule or at least one filtered rule.

[0266] Then, the display device 30 displays, for the benefit of the pilot 15, the rules which have been transmitted to it by the transmission module 65.

[0267] For example, during the aircraft's flight, the pilot 15 interacts with the dialog implemented by the processing unit 97 to determine the effect(s) on the piloting constraint of implementing one or more respective piloting actions in the state(s) in which the aircraft is located. The processing unit 97 then commands the display on the display screen 95 of all or part of the piloting rule comprising the said action(s) and the said state(s).

[0268] With the assistance method according to the invention, the generated control rules are then certifiable.

[0269] Furthermore, thanks to the reinforcement learning algorithm, the process is capable of generating a large number of flight rules without the need for human intervention. Thus, the risk of the pilot 15 encountering an aircraft state not covered by the flight rules Ci is limited.

[0270] Furthermore, the determination of the piloting commands C; according to the reinforcement learning algorithm makes it possible to determine piloting commands which would not have been taken into consideration by a human operator, leading to the generation of additional rules which a human operator would not have considered.

[0271] Training step 140 allows for the formation of sufficiently small data groups, i.e., reduced, so that each group can correspond to one or more actions performed simultaneously. Furthermore, training step 140 allows for the formation of sufficiently large data groups, i.e., significant, so that the action(s) assigned to the commands C of said group have a quantifiable effect on the control constraint.

[0272] Furthermore, the optional training step 120 allows the flight control model trained in trajectory following to be provided as input to the application step 130. However, trajectory following is the aircraft's primary function. Thus, during the application step 130, the model will continue to follow the trajectory substantially, while introducing slight variations in direction that have a substantially positive effect on the flight control constraints.

[0273] The optional identification step 160 allows extracting from the generated rules only the most representative actions and states, that is, those that have been the most frequently encountered, i.e., the most frequent, and in which the pilot 15 can have confidence. Furthermore, this step 160 makes it possible to limit the number of rules to be transmitted to the display device 30, while retaining the main elements of the generated rules.

Claims

Claims

1. Method for assisting in piloting an aircraft, the method being implemented by an electronic piloting assistance device and comprising the following steps: - acquisition (110) of a piloting model of the aircraft and of a reward function (FR) comprising a piloting constraint (CP), - application (130), to the piloting model, of a reinforcement learning algorithm from the reward function (FR), the application step (130) comprising the following sub-steps: + a reception (122) of state variable(s) (VE;) of the aircraft at reception times (Tj), + for each reception time (Tj), a modification (124) of the model from an evaluation of the reward function (FR) from the state variable(s) (VE;) received at said reception time (T;), and + for each reception time (Tj), a determination (126) of piloting commands (Ci) from the modified piloting model and the state variables (VE;) received at said time (Tj), - formation (140) of group(s) of data from the received state variables (VE;) and the determined commands (Ci), each group of data comprising the state variables (VE;) and the commands (Ci) corresponding to a plurality of successive reception times (Tj), - for the or each group of data, assignment (150) of at least one state of the aircraft to the state variables (VE;) and at least one piloting action to the commands (Ci), to generate a piloting rule comprising the at least one state and the at least one piloting action, - transmission (170) of at least one piloting rule to a display device for display to a pilot (15) of the aircraft.

2. A method according to claim 1, wherein, during the application step (130), for each reception time (Tj), a value of the control constraint (CP;) is calculated during the evaluation of the reward function, during the group formation step(s), each group of data further comprises the values ​​of the control constraint (CP;) associated with the state variables (VE;) and the commands (Ci), during the assignment step, for the or each group, at least one effect on the constraint is assigned to the values ​​of the control constraint (CP;) of the group, each rule further comprising at least one effect on the constraint.

3. Method according to the preceding claim, in which during the assignment step (150) a plurality of rules is formed, the method further comprising between the assignment step (150) and the provision step, a step of identifying (160) the main rule(s) among the plurality of control rules generated by applying a variable frequency analysis algorithm, during the transmission step (170), only the main rule(s) being transmitted.

4. Method according to claims 2 and 3, wherein the identification step (160) further comprises comparing the effect(s) on the constraint of each main rule to a predetermined threshold to obtain at least one filtered rule, each filtered rule being a respective main rule comprising at least one effect on the constraint greater than or equal to the threshold, during the transmission step (170), only the filtered rule(s) being preferably provided for display to the pilot (15) of the aircraft.

5. Method according to any one of claims 2 to 4, in which the assignment step (150) comprises for each group of data: - a calculation of a first difference between the value of the control constraint (CPd) at the last reception time (Td) among the reception times (R) of the state variables (VE;) of the group of data, and the control constraint value (CPp) at the first reception time (T p) among the reception times (Tj) of the state variables (VE;) of the group of data, - a calculation of a second difference between the value of the control constraint (CPfinai) at a reception time (Tfinai) subsequent to the last reception time (Td) among the reception times (Tj) of the state variables (VE;) of the group of data, and the control constraint value (CPp) at the first reception time (Tp) among the reception times (T;) of the state variables (VE;) of the data group, - an assignment, to the values ​​of the control constraint (CR), of a short-term effect on the control constraint, from the first difference and a control constraint correspondence table,; and - an assignment, to the values ​​of the steering constraint (CP;), of a long-term effect on the steering constraint, from the second difference and the steering constraint correspondence table.

6. Method according to any one of the preceding claims, in which the forming step (140) comprises: - classifying the determined pilot commands (Ci) among a plurality of predefined classes (CL;) and via a command correspondence table, - for each class (CL;), grouping the pilot commands (C i) belonging to said class (CL;) and determined for the reception times (Tj) forming a sequence of consecutive reception times (Tj) as long as possible, to form at least one set of grouped commands (Ci), and - for each set of grouped commands (Ci), forming a respective group comprising the commands (Ci) of said set and the state variables (VE;) received at the reception times (T;) for which said commands (Ci) were determined.

7. Method according to any one of the preceding claims, wherein during the acquisition step (110), a preliminary reward function (FRP) is acquired, the method further comprising, between the acquisition step (110) and the application step (130), a step of training (120) the model by applying a preliminary reinforcement learning algorithm to the steering model from the preliminary reward function (FRP), the training step (130) comprising a modification (134) of the model from an evaluation of the preliminary reward function (FRP).

8. A method according to any preceding claim, wherein the state variables (VE;) are selected from the group consisting of: an aircraft roll angle, an aircraft pitch angle, an aircraft yaw angle, an aircraft speed, an aircraft acceleration, a wind speed at contact with the aircraft, a wind orientation relative to the aircraft, and an aircraft position.

9. A computer program product comprising software instructions which, when implemented by a computer, implement a method according to any one of the preceding claims. previous.

10. Electronic device (25) for assisting in piloting an aircraft, comprising: - an acquisition module (45) configured to acquire a piloting model of the aircraft and a reward function comprising a piloting constraint, - an application module (50) configured to apply to the piloting model, a reinforcement learning algorithm from the reward function, the application module comprising: + a reception unit (81) configured to receive at least one state variable (VE;) of the aircraft at reception times (T)), + a modification unit (82) configured to modify, for each reception time (TO, the model from an evaluation of the reward function from the state variable(s) (VE;), and + a determination unit (83) configured to determine, for each reception time (T;), piloting commands (Ci) from the modified piloting model and the state variables (VE;) received at said reception time (T;), - a training module (55) configured to form at least one group of data from the received state variables (VE;) and the determined commands (Ci), each group of data comprising the state variables (V;) and the determined commands (Ci) corresponding to a plurality of successive reception times (T;), - an assignment module (60) configured to assign, for the or each group of data, at least one state of the aircraft to the received state variables (VE;) and at least one piloting action to the determined commands (Ci), and to generate a piloting rule comprising the at least one state and the at least one piloting action, and - a transmission module (65) configured to transmit, to a display device (30), at least one piloting rule for display to a pilot (15) of the aircraft.

11. System (10) for assisting the piloting of an aircraft comprising an electronic piloting assistance device (25) according to the preceding claim and a display device (30) configured to receive at least one piloting rule from the piloting assistance device (25) and to display said rule for the pilot (15) of the aircraft.

Images