System for monitoring a pilot of an aircraft to detect pilot incapacitation; Associated method and computer program product.

Abstract

System for monitoring a pilot of an aircraft to detect incapacitation of the pilot; Associated method and computer program product. This monitoring system (1) makes it possible to detect incapacitation of the pilot of an aircraft, said aircraft being equipped with a piloting member (40) manually actuated by the pilot. It comprises: at least one pressure detector incorporating a contact sensor (11, 12) mounted on the surface of the piloting member (40), so as to generate an elementary indicator when pressure exerted by the pilot on the piloting member is adapted to a use of this piloting member; and a calculation unit (2) programmed to process the elementary indicator delivered by the pressure detector and determine a current state of the pilot and, when said current state corresponds to incapacitation of the pilot, emit an alarm. Figure for the abstract: Figure 1

Description

Title of the invention: System for monitoring an aircraft pilot to detect pilot incapacitation; Associated method and computer program product.

[0001] The present invention relates to the technical field of systems and methods for monitoring the pilot of an aircraft.

[0002] Some players in civil aeronautics are considering changing the operational context of certain aircraft so that the crew is reduced to a single pilot for at least part of the flight, the pilot then finding himself alone in the cockpit.

[0003] However, this requires the implementation of pilot monitoring procedures in order to guarantee flight safety.

[0004] A number of monitoring procedures have been proposed using physiological sensors worn by the pilot to determine certain states of the pilot, such as his fatigue level, workload, stress level, etc.

[0005] For example, document EP3154038 discloses a method for evaluating the fatigue state of a pilot based on the detection of movements of the piloting organs.

[0006] But these monitoring procedures do not allow for the anticipation of pilot incapacitation. Indeed, in the event of incapacitation, the pilot is not only unable to carry out his mission, but also unable to declare it. This is the case, for example, when he loses consciousness following a vagal malaise or a heart attack.

[0007] It should be emphasized that state of fatigue and incapacitation are not synonymous. Thus, while it is possible to use sensors for detecting the movements of the piloting organs to determine the state of fatigue of the pilot, such sensors do not make it possible to determine incapacitation of the pilot.

[0008] It has therefore been proposed to use another type of monitoring system to detect incapacitation.

[0009] Document FR3098334A1 discloses a system for monitoring the pilot's state of consciousness based on the acquisition of images and their processing in order to detect the pilot's eye movements. If no movement is detected, a loss of consciousness signal is emitted.

[0010] It should be noted that state of consciousness and incapacitation are synonymous and the use of images of the pilot to determine the state of consciousness could also make it possible to determine incapacitation of the pilot.

[0011] However, such a monitoring system would not allow pilot incapacitation to be detected very quickly, which is essential in certain phases of flight. (such as the taxiing phase) which must remain under the manual control of the pilot. Thus, any incapacitation of the pilot must be able to be detected almost immediately, typically in less than half a second, to initiate appropriate countermeasures.

[0012] However, detection based on the pilot's eye movements does not allow such rapid detection to be achieved, if only because of the processing time for each image and the need to acquire a temporal succession of images in order to, by comparing them, determine the state of the pilot.

[0013] The aim of the present invention is therefore to solve this problem, in particular by proposing a monitoring system making it possible to trigger an alert with very low latency when the pilot is incapacitated.

[0014] For this purpose, the invention relates to a system for monitoring a pilot of an aircraft, to detect incapacitation of the pilot, the aircraft being equipped with at least one piloting member that can be manually actuated by the pilot, characterized in that the monitoring system comprises: at least one pressure detector incorporating a contact sensor mounted on the surface of the piloting member, so as to generate an elementary indicator when pressure exerted by the pilot on the piloting member is adapted to a use of this piloting member; a calculation unit programmed to process the elementary indicator delivered by the pressure detector and determine a current state of the pilot and, when said current state corresponds to incapacitation of the pilot, emit an alarm.

[0015] According to particular embodiments, the monitoring system comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

[0016] - the pressure sensor is a contact pressure sensor adapted to detect the contact of the pilot's hand on the steering mechanism.

[0017] - the pressure sensor is a force pressure sensor adapted to detect the application of a force by the pilot on the steering organ.

[0018] - the computing unit receives, from an ancillary system equipping the aircraft, at least one in flight context formation, and in which the calculation unit processes the elementary indicator delivered by the pressure detector taking into account said flight context information.

[0019] - the flight context information is flight phase information of the aircraft or information on a value of at least one flight parameter of the aircraft, the flight phase information being selected from: a taxiing phase, a takeoff phase, a cruise phase, and a landing phase, and the flight parameter information being selected from: a height, a speed, an engagement state of the autopilot, a position of a nose wheel lever, a position of a throttle lever, a position of a stick, and a position of a rudder.

[0020] - a pressure detector comprises, in addition to a contact sensor, a mo module delization, the modeling module being capable, from the signal delivered by the associated contact sensor, of calculating the elementary indicator.

[0021] - the system comprises a plurality of pressure sensors and a module aggregation system capable of aggregating all or part of the plurality of indicators delivered by the plurality of pressure detectors into a global indicator as the current state of the pilot.

[0022] - the aggregation module selects the elementary indicators to be taken into account to calculate the global indicator based on flight context information.

[0023] - the computing unit comprises an alarm generation module capable of applying an algorithm using one or more flight context information and the value of the global indicator determined by the aggregation module to determine whether to issue an alarm.

[0024] - a contact sensor is: a mechanical sensor, for example with electrical contact; a pressure sensor, for example piezoelectric, resistive or capacitive; or a proximity sensor, for example infrared.

[0025] The invention also relates to a method for monitoring the pilot of an aircraft to detect incapacitation of the pilot, said aircraft being equipped with at least one piloting member manually actuated by the pilot, characterized in that the monitoring method comprises the following steps: acquiring, by at least one contact sensor, mounted on the surface of the piloting member, a signal corresponding to a measurement of a pressure exerted by the pilot on the piloting member; processing the acquired signal in order to determine a current state of the pilot; and, when said current state corresponds to incapacitation, emitting an alarm.

[0026] The invention finally relates to a computer program product comprising software instructions which, when executed by a calculation unit of the preceding system, implement the preceding method.

[0027] The invention and its advantages will be better understood on reading the detailed description which follows of a particular embodiment, given solely as a non-limiting example, this description being made with reference to the appended drawings in which:

[0028] [Fig-1] [Fig.l] is a schematic representation of an embodiment of the monitoring system according to the invention;

[0029] [Fig.2] [Fig.2] is a schematic representation of an embodiment of the monitoring method implemented in the system of [Fig.l]; and,

[0030] [Fig.3] [Fig.3] is a block representation of a management algorithm alarm generation of the process of [Fig.2].

[0031] The aircraft pilot monitoring system is intended to be mounted on board the cockpit of an aircraft (such as an airplane, a helicopter, or the equivalent) or a drone cockpit, in order to detect the occurrence of pilot incapacitation in real time (less than half a second).

[0032] As illustrated by the embodiment of [Fig.l], the monitoring system 1 comprises a computing unit 2 and a plurality of contact sensors 11 to 19 connected to the input of the computing unit 2.

[0033] The computing unit 2 is a computer comprising computing means, such as a processor 3, storage means, such as a memory 4, a set of connectors 6, making it possible to connect each sensor to the computing unit 2, a network interface 5, allowing bidirectional communications on a network 30 of the aircraft, and an internal communication bus 7, connecting the different components of the computing unit 2.

[0034] The memory 4 stores the instructions of computer programs, in particular those of a monitoring program 8 whose execution allows the implementation of a pilot monitoring method, which is based on the generation of an alarm when an incapacitation of the pilot is detected from the signals delivered by all or part of the contact sensors.

[0035] The computing unit 2 is interfaced with the network 30 of the aircraft. The format of the communications on this network depends on the manufacturer of the aircraft and the computing unit is therefore adapted to transmit and receive messages in the required format (AFDX, ARINC, etc.), in particular to transmit alarms in the required format.

[0036] To this network 30 are also connected various ancillary systems capable of providing flight context information, Info_Context, to the calculation unit 2. For example, an autopilot system 31 indicates the state of engagement of the autopilot; a parking brake system 32 indicates the state of application of the parking brake; a flight control system 33 indicates the current flight phase of the aircraft; and an instrumentation system 34 indicates one or more flight parameters, such as the current height of the aircraft or its ground speed.

[0037] And also connected to the aircraft network, an alarm management system 35. When the computing unit 2 detects the incapacitation of the pilot, it emits an alarm on the network 30 to the system 35. Depending on this alarm and the operational situation, the system 35 is capable of initiating the necessary countermeasures to guarantee the safety of the aircraft.

[0038] Each contact sensor is surface mounted on an associated control member.

[0039] A contact sensor is capable of detecting pressure exerted by the pilot on the piloting member that it equips and of generating a signal intended for the calculation unit 2, the signal corresponding to a measurement of pressure exerted by the pilot on the associated steering body.

[0040] A piloting member equipped with a contact sensor is such that the pilot must exert a certain pressure on the piloting member if he wants to manipulate it correctly to pilot the aircraft properly. Piloting members equipped with contact sensors are therefore piloting members that can be actuated by the application of a force by the pilot. Although this force can be applied with the foot or the hand, we will speak of piloting members that can be actuated "manually" in what follows.

[0041] For example, in the embodiment illustrated in [Fig. 1], a first left contact sensor 11 and a first right contact sensor 12 are arranged on the stick 40 of the aircraft (respectively on the left handle 41 and on the right handle 42 of the stick). Note that the invention is not specific to a type of stick (or steering wheel) and can be implemented whether the stick is of the type “arranged in front of the pilot” (as illustrated in [Fig.l]) or of the type “arranged on the side” (mini stick with electrical controls).

[0042] For example, a second left contact sensor 15 and a second right contact sensor 16 are arranged on the rudder pedals, respectively left 45 and right 46.

[0043] For example, a third left contact sensor 17 and a third right contact sensor 18 are arranged on the throttle lever of the left engine 47 and that of the right engine 48 respectively.

[0044] For example, a fifth contact sensor 19 advantageously equips the nose wheel lever 49.

[0045] Alternatively, other control members could carry a contact sensor, such as a landing gear adjustment lever, a dashboard switch or screen, an input tool, such as a mouse or a trackball, etc.

[0046] Contact sensors can be:

[0047] - mechanical sensors, for example with electrical contact (the contact of the hand of the pilot on a moving part of the sensor equipping the piloting member generates a mechanical action making it possible to establish or interrupt an electrical contact);

[0048] - pressure sensors, for example piezoelectric (in particular piezoresistive), resistive or capacitive (contact of the pilot's hand on a surface of the sensor equipping the piloting organ generates a deformation of the sensor causing a modification of an electrical property of the sensor); or again,

[0049] - proximity sensors, for example infrared (the contact of the pilot's hand on the sensor equipping the control unit allows a detection signal to be reflected).

[0050] Different methods for integrating a contact sensor on a control member are known to those skilled in the art. It will be preferable to use contact sensors of the capacitive type, because their integration is simpler. For example, we can cite the possibility of producing a contact sensor (and the associated circuit) by 3D printing on the external surface of the control element. This is particularly interesting for equipping ergonomic control elements, such as handles or joysticks.

[0051] As shown schematically in [Fig.2], the execution of the program 8 makes it possible to provide the calculation unit 2 with different functionalities, in particular a plurality of modeling modules 51 to 59, an aggregation module 70 and an alarm generation module 72.

[0052] Each contact sensor 11 to 19 is associated with at least one modeling module 51 to 59.

[0053] A contact sensor and the associated modeling module together form a pressure detector.

[0054] The modeling module is capable of acquiring the signal generated by the associated sensor and of determining the value of an elementary indicator which indicates whether the pilot is exerting suitable pressure on the corresponding piloting member.

[0055] The modeling module thus implements a contact model which depends on the nature of the corresponding control organ, that is to say the manner in which this organ must be manipulated by the pilot.

[0056] For example, the modeling model compares the pressure measured by the associated contact sensor to a pressure threshold and an indicator is emitted when the pressure exerted is greater than this threshold.

[0057] Alternatively, the contact model takes into account the flight phase to select the contact model to be implemented. For example, the flight phase may make it possible to define the value of the pressure threshold to be taken into account.

[0058] The detection that the pilot is applying the required pressure to the control member is indicative of an intentional action by the pilot and therefore that he is not incapacitated. The invention is therefore based on the assumption that in the event of the pilot losing consciousness, he will no longer be able to apply the pressure normally required to control this equipment to the control member.

[0059] The plurality of pressure detectors comprises at least one contact pressure detector (or contact detector in the following). A contact detector must make it possible to detect a low pressure exerted by the pilot on the corresponding piloting member (for example the model implemented uses a low pressure threshold).

[0060] The plurality of pressure detectors advantageously comprises at least one force pressure detector (or force detector) in the following. A force detector must make it possible to detect a high pressure exerted by the pilot on the corresponding piloting member (for example the model implemented uses a high pressure threshold).

[0061] For example, the first detectors 61, 62 integrating the first sensors 11 and 12 which equip the handle 40 are of the contact detector type: when the pilot correctly grasps the left handle in his left hand, he applies a pressure detectable by the first right detector and when the pilot correctly grasps the right handle in his right hand, he exerts a pressure detectable by the first right detector. These first detectors therefore make it possible to generate the elementary indicators II, 12 indicating the simple fact that the pilot has his hand correctly placed on the handle.

[0062] For example, the second detectors 65, 66 integrating the second sensors 15 and 16 which equip the rudder pedals are of the contact detector type: to turn the aircraft during the taxiing phase on the ground or to make it make a turn in the air, the pilot places his feet on each pedal. These second detectors therefore make it possible to generate the indicators 15, 16 indicating that the pilot has his feet correctly placed on the pedals.

[0063] For example, the third detectors 67, 68 integrating the third sensors 17 and 18 which equip the throttle levers are of the contact detector type: when the pilot correctly grasps one and / or the other of these levers, without moving them, he applies a detectable pressure. Elementary indicators 17 and 18 are generated.

[0064] For example again, the detector 69 integrating the sensor 19 equipping the nose wheel lever is associated with a first modeling module 59 which uses a “straight line” model making it possible to detect that the pilot has his hand placed on this lever. This is still a contact detector. It emits an elementary indicator 19.

[0065] An example of a force sensor is given by the detector 69' integrating the sensor 19 equipping the nose wheel lever and a second modeling module 59' which uses a "turning" model requiring the pilot to exert high pressure to oppose the return force forcing the lever to return to the neutral position. This is a force detector. It emits an elementary indicator 19'.

[0066] A detector therefore makes it possible to determine whether a particular intentional action is carried out by the pilot on the corresponding piloting device and, if so, to emit the associated elementary indicator.

[0067] It should be noted that a contact sensor, for example capacitive, could allow the detection of low pressure without needing to add a modeling module. However, the contact sensor being mounted on a curved surface (such as that of the handle of the stick), it can be subjected to mechanical constraints which lead to the emission by the contact sensor of a signal corresponding to the application of pressure even though no pressure is actually exerted by the pilot on the sensor. The modeling module then makes it possible to calibrate the contact sensor, for example by adjusting the pressure threshold beyond which detection information is emitted.

[0068] The aggregation module 70 is capable of aggregating all or part of the elementary indicators at the current time to determine a current state of the pilot. For example, the current state of the pilot is a global indicator I taking the value unity when it is detected that the pilot is performing an intentional action or the value zero when no intentional action of the pilot is detected, that is to say that the pilot is incapacitated. In other words, a value zero of the global indicator corresponds to the detection of an incapacitation of the pilot.

[0069] The aggregation module 70 advantageously takes into account the flight context information, Info_Context, to select the elementary indicators to be aggregated to calculate the global indicator. Such flight context information comes from one or other of the ancillary systems 31 to 34.

[0070] For example, if the calculation unit 2 receives flight context information according to which the aircraft is in cruise phase and the autopilot is engaged, this means that the pilot has no reason to hold the stick 40. Consequently, it is appropriate to ignore the elementary indicators derived from the signals coming from the contact sensors arranged on the flight control members, such as the sensors 11 and 12.

[0071] Finally, the alarm generation module 72 is capable of generating an alarm and transmitting it to the alarm management system 35. The module 72 executes an algorithm which takes as input the global indicator I at the current time to decide whether to issue an alarm. Preferably, this algorithm also uses context information, Info_Context.

[0072] For example, as shown in [Fig.3], the algorithm, which is implemented by the module 72, compares (step 110) the flight height H to a predefined threshold, for example 1000 feet.

[0073] When the height is greater than or equal to this threshold, the algorithm checks (step 120) whether the autopilot - AP ("AutoPilot" in English) is engaged ("ON" state).

[0074] If so, the algorithm turns off the existing alarm, if there was one (for example by assigning the value zero to the variable ALARM), and loops on step 110.

[0075] If not, the algorithm checks (step 130) the value of the global indicator I to know if the pilot is performing an intentional action.

[0076] If so, the algorithm deletes the existing alarm, if there was one, and loops to step 110.

[0077] If not, a driver incapacitation is detected and an alarm is issued (step 140). For example, the value unity is assigned to the variable ALARM. The algorithm then loops to step 110 for a new iteration.

[0078] On the other hand, when the height H is lower than the threshold of 1000 feet, the algorithm checks (step 150) if the parking brake (“Brakes”) is applied (“ON” state).

[0079] If so, the algorithm deletes the existing alarm, if there was one, and loops to step 110.

[0080] If not, the algorithm checks (step 160) whether the nose wheel steering lever -NWS (“Nose Wheel Steering”) is in neutral position (“Neutral” state).

[0081] If not, the algorithm deletes the existing alarm, if there was one, and loops to step 110.

[0082] If so, the algorithm checks (step 170) the value of the global indicator I to know if the pilot is performing an intentional action.

[0083] If so, the algorithm deletes the existing alarm, if there was one, and loops to step 110.

[0084] If not, a pilot incapacitation is detected and an alarm is issued (step 180). The algorithm loops to step 110 for a new iteration.

[0085] The future operating mode of aircraft when the pilot finds himself alone on board during the landing phase could require having the hands on the stick so as to be ready to react when the height of the aircraft is below a certain value, here taken for example at 1000 feet. Consequently, if the contact sensors arranged on the stick or the throttle do not detect any pressure (the only sensors taken into account in this situation to calculate the overall indicator), an alarm is emitted immediately.

[0086] This constraint of having hands on the stick is not strict above 1000 feet and the elementary indicators derived from the stick contact sensors are not sufficient to detect incapacitation.

[0087] Upon receiving an alarm, the alert management system 35 may begin by emitting an audible alert in the cockpit and wait for the computing unit 2 to suspend the previously triggered alarm, thus indicating that the pilot has regained control of the aircraft, for example by immediately placing his hands back on the stick below the ceiling of 1000 feet.

[0088] On the other hand, if during a predetermined time interval, the calculation unit 2 continues to emit an alarm, the system 35 switches to countermeasures to take control of the piloting of the aircraft, for example by engaging the automatic pilot.

[0089] It should be noted that [Fig.2] could be read as the different stages of a monitoring method. According to this method, the calculation unit 2 carries out:

[0090] - the acquisition of the signals delivered by the different contact sensors,

[0091] - processing these signals in order to determine the current state of the pilot: for example first by calculating an elementary indicator by verifying that the pressure measured by a sensor is adapted to the piloting organ monitored (i.e. the measured pressure conforms to a model), then by aggregating all or part of these elementary indicators in a global indicator as the state of the pilot;

[0092] - and the generation of an alarm when the state of the pilot corresponds to an incapacity- quote.

[0093] Advantageously, the signal processing and alarm generation steps take into account contextual information delivered by other systems on board the aircraft.

[0094] Alternatively, a pressure detector is a device independent of the calculation unit. However, it is preferable that the modeling module associated with a contact sensor is of the software type, in the calculation unit, to be able to adjust the parameters of the model, such as for example the pressure threshold.

[0095] Optionally, a control member, such as a handle, may be equipped with several contact sensors, arranged at different locations to allow different positions of the pilot's hand on this handle. By multiplying the measurement points or by increasing the surface area of ​​the contact sensor, the occurrence of false alarms is limited.

[0096] Other types of sensors may optionally be connected to the computing unit. These may be cameras analyzing the pilot's behavior, accelerometers integrated into the pilot's helmet to detect head movements, physiological sensors such as those measuring heart rate, a microphone analyzing the sound environment inside the cabin (in particular the sounds emitted by the pilots), etc.

[0097] Other indicators may be derived either from the signals delivered by these other sensors equipping the cockpit, or from the actuation of other cockpit equipment. For example, as illustrated in [Fig.l], the handles of the stick 40 are equipped with left and right buttons, 43 and 44. The left button 43 allows for example to disengage the autopilot, while the right button allows the pilot to transmit on the radio. The actuation of one or other of these buttons may, alternatively, serve as an indicator of a voluntary action by the pilot.

Claims

Claims

1. Monitoring system (1) of a pilot of an aircraft, for detecting incapacitation of the pilot, the aircraft being equipped with at least one piloting member (40) manually actuable by the pilot, characterized in that the monitoring system (1) comprises: - at least one pressure detector (62, 65) integrating: a capacitive pressure sensor (11, 12) mounted on the surface of the piloting member (40), so as to generate an elementary indicator when a pressure exerted by the pilot on the piloting member is adapted to a use of this piloting member;and, a modeling module (51 to 59), the modeling module being capable, from the signal delivered by the capacitive pressure sensor, of calculating an elementary indicator, - a calculation unit (2) programmed to process the elementary indicator delivered by the pressure detector and determine a current state of the pilot and, when said current state corresponds to an incapacitation of the pilot, emit an alarm (ALARM).;

2. A system according to claim 1, wherein the pressure sensor is a contact pressure sensor adapted to detect contact of the pilot's hand on the piloting member.

3. A system according to claim 1 or claim 2, wherein the pressure sensor is a force pressure sensor adapted to detect the application of a force by the pilot to the piloting member.

4. System according to any one of claims 1 to 3, in which the calculation unit (2) receives, from an ancillary system (31 to 34) equipping the aircraft, at least one flight context information (Info_Context), and in which the calculation unit (2) processes the elementary indicator delivered by the pressure detector taking into account said flight context information.

5. System according to claim 4, in which the flight context information (Info_Context) is flight phase information of the aircraft or information on a value of at least one flight parameter of the aircraft, the flight phase information being selected from: a taxiing phase, a takeoff phase, a cruising phase, and a landing phase, and the flight parameter information being selected from: a height, a speed, an engagement state of the autopilot, a position of a nose wheel lever, a throttle position, stick position, and rudder position.

6. System according to any one of claims 1 to 5, comprising a plurality of pressure detectors and an aggregation module (70) capable of aggregating all or part of the plurality of indicators delivered by the plurality of pressure detectors into a global indicator as the current state of the pilot.

7. System according to claim 6, in which the aggregation module (70) selects the elementary indicators to be taken into account to calculate the global indicator as a function of flight context information.

8. System according to claim 6 or claim 7, in which the calculation unit (2) comprises an alarm generation module (72) capable of applying an algorithm using one or more flight context information and the value of the global indicator determined by the aggregation module (70) to determine whether to issue an alarm.

Images