SYSTEM FOR MONITORING AN AIRCRAFT PILOT FOR DETECTING PILOT IRREGULARITY; METHOD AND COMPUTER PROGRAM PRODUCT FOR THIS PURPOSE

Description

[0001] The present invention relates to the technical field of systems and methods for monitoring the pilot of an aircraft.

[0002] Some civil aviation players are considering changing the operational context of certain aircraft so that the crew is reduced to a single pilot for at least part of the flight, with the pilot then finding himself alone in the cockpit.

[0003] However, this requires the implementation of pilot monitoring procedures to ensure flight safety.

[0004] A number of monitoring procedures have been proposed using physiological sensors worn by the pilot to determine certain states of the pilot, such as fatigue level, workload, stress level, etc.

[0005] For example, document EP3154038 discloses a method for evaluating the fatigue state of a pilot based on the detection of movements of the piloting organs.

[0006] But these monitoring procedures do not allow for the pilot's incapacitation to be anticipated. Indeed, in the event of incapacitation, the pilot is not only unable to carry out his mission, but also unable to report it. This is the case, for example, in the event of a loss of consciousness following a vasovagal seizure or a heart attack.

[0007] It should be noted that fatigue and incapacitation are not synonymous. Thus, while it is possible to use sensors that detect the movements of the pilot's control components to determine the pilot's fatigue, such sensors cannot determine whether the pilot is incapacitated.

[0008] It was therefore proposed to use another type of monitoring system to detect incapacitation.

[0009] Document FR3098334A1 discloses a pilot consciousness monitoring system based on image acquisition and processing to detect the pilot's eye movements. If no movement is detected, a loss of consciousness signal is emitted.

[0010] Documents FR 3 092 564 A1, DE 20 2007 015835 U1, WO 2022 / 080512 A1, US 2021 / 034053 A1 and US 10 426 393 B2 disclose other systems for monitoring a pilot.

[0011] It should be noted that state of consciousness and incapacitation are synonymous and the use of images of the pilot to determine the state of consciousness could also make it possible to determine incapacitation of the pilot.

[0012] However, such a monitoring system would not allow pilot incapacitation to be detected very quickly, which is essential in certain flight phases (such as the taxiing phase) which must remain under the pilot's manual control. Thus, any pilot incapacitation must be able to be detected almost immediately, typically in less than half a second, to initiate appropriate countermeasures.

[0013] However, detection based on the pilot's eye movements does not allow such rapid detection to be achieved, if only because of the processing time for each image and the need to acquire a temporal succession of images in order to, by comparing them, determine the pilot's state.

[0014] The aim of the present invention is therefore to solve this problem, in particular by proposing a monitoring system making it possible to trigger an alert with very low latency when the pilot is incapacitated.

[0015] For this purpose, the invention relates to a system for monitoring a pilot of an aircraft in accordance with the appended claims.

[0016] The invention and its advantages will be better understood upon reading the following detailed description of a particular embodiment, given solely as a non-limiting example, this description being made with reference to the appended drawings in which: [ Fig 1 ] There figure 1 is a schematic representation of an embodiment of the monitoring system according to the invention; [ Fig 2 ] There figure 2 is a schematic representation of an embodiment of the monitoring method implemented in the system of the figure 1 ; And, [ Fig 3 ] There figure 3 is a block representation of an alarm generation algorithm of the process of the figure 2 .

[0017] The aircraft pilot monitoring system is intended to be mounted on board the cockpit of an aircraft (such as an airplane, a helicopter, or the equivalent) or a drone flight deck, in order to detect the occurrence of pilot incapacitation in real time (less than half a second).

[0018] As illustrated by the embodiment of the figure 1 , the monitoring system 1 comprises a computing unit 2 and a plurality of contact sensors 11 to 19 connected to the input of the computing unit 2.

[0019] The computing unit 2 is a computer comprising computing means, such as a processor 3, storage means, such as a memory 4, a set of connectors 6, allowing each sensor to be connected to the computing unit 2, a network interface 5, allowing bidirectional communications on a network 30 of the aircraft, and an internal communication bus 7, connecting the different components of the computing unit 2.

[0020] The memory 4 stores the instructions of computer programs, in particular those of a monitoring program 8, the execution of which allows the implementation of a pilot monitoring method, which is based on the generation of an alarm when an incapacitation of the pilot is detected from the signals delivered by all or part of the contact sensors.

[0021] The computing unit 2 is interfaced with the aircraft network 30. The format of communications on this network depends on the aircraft manufacturer and the computing unit is therefore adapted to transmit and receive messages in the required format (AFDX, ARINC, etc.), in particular to transmit alarms in the required format.

[0022] To this network 30 are also connected various ancillary systems capable of providing flight context information, Info_Context, to the computing unit 2. For example, an autopilot system 31 indicates the engagement state of the autopilot; a parking brake system 32 indicates the application state of the parking brake; a flight control system 33 indicates the current flight phase of the aircraft; and an instrumentation system 34 indicates one or more flight parameters, such as the current height of the aircraft or its ground speed.

[0023] And also connected to the aircraft network, an alarm management system 35. When the computing unit 2 detects the incapacitation of the pilot, it emits an alarm on the network 30 to the system 35. Depending on this alarm and the operational situation, the system 35 is capable of initiating the necessary countermeasures to guarantee the safety of the aircraft.

[0024] Each contact sensor is surface mounted on an associated control unit.

[0025] A contact sensor is capable of detecting pressure exerted by the pilot on the control member with which it is fitted and of generating a signal intended for the calculation unit 2, the signal corresponding to a measurement of pressure exerted by the pilot on the associated control member.

[0026] A control device equipped with a contact sensor is such that the pilot must exert a certain amount of pressure on the control device if he wants to manipulate it correctly to pilot the aircraft properly. Control devices equipped with contact sensors are therefore control devices that can be actuated by the application of a force by the pilot. Although this force can be applied with the foot or the hand, we will speak of control devices that can be actuated "manually" in what follows.

[0027] For example, in the embodiment illustrated in the figure 1 , a first left contact sensor 11 and a first right contact sensor 12 are arranged on the stick 40 of the aircraft (respectively on the left handle 41 and on the right handle 42 of the stick). Note that the invention is not specific to a type of stick (or steering wheel) and can be implemented whether the stick is of the type “arranged in front of the pilot” (as illustrated in the figure 1 ) or the “side mounted” type (mini joystick with electrical controls).

[0028] For example, a second left contact sensor 15 and a second right contact sensor 16 are arranged on the rudder pedals, respectively left 45 and right 46.

[0029] For example, a third left contact sensor 17 and a third right contact sensor 18 are arranged on the throttle lever of the left engine 47 and that of the right engine 48 respectively.

[0030] For example, a fifth contact sensor 19 advantageously equips the nose wheel lever 49.

[0031] Alternatively, other control components could carry a contact sensor, such as a landing gear adjustment lever, an instrument panel switch or display, an input tool, such as a mouse or trackball, etc.

[0032] Contact sensors can be: mechanical sensors, for example electrical contact (contact of the pilot's hand on a moving part of the sensor equipping the control member generates a mechanical action making it possible to establish or interrupt an electrical contact); pressure sensors, for example piezoelectric (in particular piezoresistive), resistive or capacitive (contact of the pilot's hand on a surface of the sensor equipping the control member generates a deformation of the sensor causing a modification of an electrical property of the sensor); or proximity sensors, for example infrared (contact of the pilot's hand on the sensor equipping the control member makes it possible to reflect a detection signal).

[0033] Different methods for integrating a contact sensor into a control unit are known to those skilled in the art.

[0034] According to the claimed invention, capacitive type contact sensors are implemented, because their integration is simpler. For example, we can cite the possibility of producing a capacitive sensor (and the associated circuit) by 3D printing on the external surface of the control member. This is particularly interesting for equipping ergonomic control members, such as handles or joysticks.

[0035] Capacitive sensors are more suitable for integration into flight controls since they are very thin, invisible to the pilot and require relatively simple control electronics.

[0036] Furthermore, push-button sensors do not cover all possible positions of the pilot's hand on a control device. In particular, the hand can have very variable positions on the throttle lever between the "full throttle" position (throttles forward) and the "reverse thrust" position (throttle lever fully aft). On the other hand, it is possible to broadly cover the surface of a control device, such as the throttle lever, with a capacitive sensor and thus detect the pilot's activity regardless of the position of this control device.

[0037] As shown schematically in the figure 2 , the execution of the program 8 makes it possible to provide the calculation unit 2 with different functionalities, in particular a plurality of modeling modules 51 to 59, an aggregation module 70 and an alarm generation module 72.

[0038] Each contact sensor 11 to 19 is associated with at least one modeling module 51 to 59.

[0039] A contact sensor and the associated modeling module together form a pressure detector.

[0040] The modeling module is capable of acquiring the signal generated by the associated sensor and determining the value of an elementary indicator which indicates whether the pilot is exerting suitable pressure on the corresponding piloting device.

[0041] The modeling module thus implements a contact model which depends on the nature of the corresponding control organ, that is to say on the way in which this organ must be manipulated by the pilot.

[0042] For example, the modeling model compares the pressure measured by the associated contact sensor to a pressure threshold and an indicator is emitted when the pressure exerted is greater than this threshold.

[0043] Alternatively, the contact model takes into account the flight phase to select the contact model to be implemented. For example, the flight phase can be used to define the pressure threshold value to be considered.

[0044] The detection that the pilot is applying the required pressure to the control member is indicative of an intentional action by the pilot and therefore that he is not incapacitated. The invention is therefore based on the assumption that if the pilot loses consciousness, he will no longer be able to apply the pressure normally required to control this equipment to the control member.

[0045] The plurality of pressure detectors comprises at least one contact pressure detector (or contact detector in the following). A contact detector must be able to detect low pressure exerted by the pilot on the corresponding piloting member (for example, the model implemented uses a low pressure threshold).

[0046] The plurality of pressure detectors advantageously comprises at least one force pressure detector (or force detector) in the following. A force detector must make it possible to detect high pressure exerted by the pilot on the corresponding piloting member (for example the model implemented uses a high pressure threshold).

[0047] For example, the first detectors 61, 62 integrating the first sensors 11 and 12 which equip the stick 40 are of the contact detector type: when the pilot correctly grasps the left handle in his left hand, he applies a pressure detectable by the first right detector and when the pilot correctly grasps the right handle in his right hand, he exerts a pressure detectable by the first right detector. These first detectors therefore make it possible to generate the elementary indicators I1, I2 indicating the simple fact that the pilot has his hand correctly placed on the handle.

[0048] For example, the second detectors 65, 66 integrating the second sensors 15 and 16 which equip the rudder pedals are of the contact detector type: to turn the aircraft during taxiing on the ground or to make it make a turn in the air, the pilot places his feet on each pedal. These second detectors therefore make it possible to generate the indicators I5, I6 indicating that the pilot has his feet correctly placed on the pedals.

[0049] For example, the third detectors 67, 68 integrating the third sensors 17 and 18 which equip the throttle levers are of the contact detector type: when the pilot correctly grasps one and / or the other of these levers, without moving them, he applies a detectable pressure. Elementary indicators I7 and I8 are generated.

[0050] For example, the detector 69 integrating the sensor 19 equipping the nose wheel lever is associated with a first modeling module 59 which uses a “straight line” model making it possible to detect that the pilot has his hand placed on this lever. This is still a contact detector. It emits an elementary indicator I9.

[0051] An example of a force sensor is given by the detector 69' integrating the sensor 19 equipping the nose wheel lever and a second modeling module 59' which uses a "turning" model requiring the pilot to exert high pressure to oppose the return force forcing the lever to return to the neutral position. This is a force detector. It emits an elementary indicator I9'.

[0052] A detector therefore makes it possible to determine whether a particular intentional action is carried out by the pilot on the corresponding control device and, if so, to emit the associated elementary indicator.

[0053] It should be noted that a contact sensor, for example capacitive, could allow the detection of low pressure without the need to add a modeling module. However, since the contact sensor is mounted on a curved surface (such as that of the handlebar grip), it may be subject to mechanical constraints which lead to the emission by the contact sensor of a signal corresponding to the application of pressure even though no pressure is actually exerted by the pilot on the sensor. The modeling module then makes it possible to calibrate the contact sensor, for example by adjusting the pressure threshold beyond which detection information is emitted.

[0054] The aggregation module 70 is capable of aggregating all or part of the elementary indicators at the current time to determine a current state of the pilot. For example, the current state of the pilot is a global indicator I taking the value unity when it is detected that the pilot is performing an intentional action or the value zero when no intentional action of the pilot is detected, that is to say that the pilot is incapacitated. In other words, a value zero of the global indicator corresponds to the detection of an incapacitation of the pilot.

[0055] The aggregation module 70 advantageously takes into account the flight context information, Info_Context, to select the elementary indicators to be aggregated to calculate the global indicator. Such flight context information comes from one or other of the ancillary systems 31 to 34.

[0056] For example, if the computing unit 2 receives flight context information that the aircraft is in cruise phase and the autopilot is engaged, this means that the pilot has no reason to hold the stick 40. Consequently, it is appropriate to ignore the elementary indicators derived from the signals coming from the contact sensors arranged on the flight control members, such as the sensors 11 and 12.

[0057] Finally, the alarm generation module 72 is capable of generating an alarm and transmitting it to the alarm management system 35. The module 72 executes an algorithm that takes as input the global indicator I at the current time to decide whether to issue an alarm. Preferably, this algorithm also uses context information, Info_Context.

[0058] For example, as shown in the figure 3 , the algorithm, which is implemented by the module 72, compares (step 110) the flight height H to a predefined threshold, for example 1000 feet.

[0059] When the height is greater than or equal to this threshold, the algorithm checks (step 120) whether the autopilot - AP ("AutoPilot" in English) is engaged ("ON" state).

[0060] If so, the algorithm turns off the existing alarm, if there was one (e.g. by assigning the value zero to the variable ALARM), and loops on step 110.

[0061] If not, the algorithm checks (step 130) the value of the global indicator I to see if the driver is performing an intentional action.

[0062] If so, the algorithm removes the existing alarm, if there was one, and loops to step 110.

[0063] If not, a driver incapacitation is detected and an alarm is issued (step 140). For example, the value unity is assigned to the variable ALARM. The algorithm then loops to step 110 for a new iteration.

[0064] On the other hand, when the height H is lower than the threshold of 1000 feet, the algorithm checks (step 150) if the parking brake (“Brakes”) is applied (“ON” state).

[0065] If so, the algorithm removes the existing alarm, if there was one, and loops to step 110.

[0066] If not, the algorithm checks (step 160) whether the nose wheel steering lever - NWS ("Nose Wheel Steering") is in neutral position ("Neutral" state).

[0067] If not, the algorithm deletes the existing alarm, if there was one, and loops to step 110.

[0068] If so, the algorithm checks (step 170) the value of the global indicator I to see if the pilot is performing an intentional action.

[0069] If so, the algorithm removes the existing alarm, if there was one, and loops to step 110.

[0070] If not, a driver incapacitation is detected and an alarm is issued (step 180). The algorithm loops to step 110 for a new iteration.

[0071] The future operating mode of aircraft when the pilot is alone on board during landing could require having hands on the control column so as to be ready to react when the height of the aircraft is below a certain value, taken here for example at 1000 feet. Consequently, if the contact sensors located on the control column or the throttle do not detect any pressure (the only sensors taken into account in this situation to calculate the overall indicator), an alarm is issued immediately.

[0072] This requirement to have hands on the stick is not strict above 1000 feet and the basic indicators derived from the stick contact sensors are not sufficient to detect incapacitation.

[0073] Upon receiving an alarm, the alert management system 35 may begin by emitting an audible alert in the cockpit and wait for the computing unit 2 to suspend the previously triggered alarm, thereby indicating that the pilot has regained control of the aircraft, for example by immediately placing his hands back on the stick below the ceiling of 1000 feet.

[0074] On the other hand, if during a predetermined time interval, the computing unit 2 continues to emit an alarm, the system 35 switches to countermeasures to take control of the piloting of the aircraft, for example by engaging the autopilot.

[0075] It should be noted that the figure 2 could be read as the different stages of a monitoring process. According to this process, the computing unit 2 carries out: the acquisition of the signals delivered by the various contact sensors, the processing of these signals in order to determine the current state of the pilot: for example, first by calculating an elementary indicator by verifying that the pressure measured by a sensor is adapted to the piloting device monitored (i.e. the measured pressure is consistent with a model), then by aggregating all or part of these elementary indicators into a global indicator as the state of the pilot; and the generation of an alarm when the state of the pilot corresponds to incapacitation.

[0076] Advantageously, the signal processing and alarm generation steps take into account contextual information delivered by other systems on board the aircraft.

[0077] Alternatively, a pressure sensor is a device independent of the computing unit. However, it is preferable that the modeling module associated with a contact sensor is of the software type, in the computing unit, to be able to adjust the model parameters, such as the pressure threshold.

[0078] Optionally, a control device, such as a handle, can be equipped with several contact sensors, arranged at different locations to allow different positions of the pilot's hand on this handle. By multiplying the measurement points or by increasing the surface area of ​​the contact sensor, the occurrence of false alarms is limited.

[0079] Other types of sensors may be connected to the computing unit. These may include cameras analyzing the pilot's behavior, accelerometers integrated into the pilot's helmet to detect head movements, physiological sensors such as those measuring heart rate, a microphone analyzing the sound environment inside the cabin (including sounds emitted by pilots), etc.

[0080] Other indicators may be derived either from signals delivered by these other sensors fitted to the cockpit, or from the actuation of other cockpit equipment. For example, as illustrated in the figure 1, the handles of the stick 40 are equipped with left and right buttons, 43 and 44. The left button 43 allows, for example, the autopilot to be disengaged, while the right button allows the pilot to transmit on the radio. The actuation of one or other of these buttons can, alternatively, serve as an indicator of a voluntary action by the pilot.

Claims

1. A system (1) for monitoring a pilot of an aircraft, in order to detect an incapacitation of the pilot, the aircraft being equipped with at least one piloting device (40) which can be manually actuated by the pilot, characterized in that the system (1) includes: - the piloting device (40), - at least one pressure sensor (62, 65) so as to generate an elementary indicator when a pressure exerted by the pilot on the piloting device is adapted to the use of this piloting device, the pressure sensor integrating: a capacitive pressure sensor (11, 12) mounted on a surface of the piloting device (40), and, a modeling module (51 to 59), the modeling module being adapted to compute the elementary indicator, from the signal delivered by the capacitive pressure sensor, by implementing a contact model that depends on the nature of the piloting device, - a computing unit (2) programmed for processing the elementary indicator delivered by the pressure sensor and for determining a current state of the pilot and, when said current state corresponds to an incapacitation of the pilot, for issuing an alarm (ALARM).

2. The system according to claim 1, wherein the pressure sensor is a contact pressure sensor suitable for detecting the contact of the pilot's hand on the piloting device.

3. The system according to claim 1 or claim 2, wherein the pressure sensor is a force pressure sensor suitable for detecting the application of a force by the pilot, on the piloting device.

4. The system according to any of claims 1 to 3, wherein the computing unit (2) receives at least one item of flight context information (Info_context) from an auxiliary system (31 to 34) equipping the aircraft, and wherein the computing unit (2) processes the elementary indicator delivered by the pressure sensor taking into account said flight context information.

5. The system according to claim 4, wherein the flight context information (Info_context) is a piece of information on a flight phase of the aircraft or a value of at least one flight parameter of the aircraft, the flight phase information being selected among: a taxiing phase, a take-off phase, a cruising phase, and a landing phase, and the flight parameter information being selected among: a height, a speed, an engagement status of the autopilot, a position of a noise wheel steering control, a position of a throttle, a position of a stick, and a position of a rudder-bar.

6. The system according to any of claims 1 to 5, including a plurality of pressure sensors and an aggregation module suitable for aggregating (70) all or part of the plurality of indicators delivered by the plurality of pressure sensors into a global indicator as a current state of the pilot.

7. The system according to claim 6, wherein the aggregation module (70) selects the elementary indicators to be taken into account for calculating the global indicator according to the flight context information.

8. The system according to claim 6 or claim 7, in which the computing unit (2) comprises an alarm generation module (72) capable of applying an algorithm using one or more items of flight context information and the value of the global indicator determined by the aggregation module (70) to determine whether to issue an alarm.