Method for determining the physiological states of an operator, and associated computer program and determination system

Abstract

Method for determining the physiological states of an operator, and associated computer program and determination system. The present invention relates to a method for determining the physiological states of an operator, comprising a first preliminary step (110) of determining a correspondence matrix between a mathematical model for determining a physiological state of the operator and a set of data types necessary for implementing the mathematical model. The method further comprises the following steps: - verification (220) of the availability of each sensor (12); - when a sensor (12) is unavailable, exclusion (240) from consideration of each mathematical model using the data type corresponding to that sensor (12), in accordance with the correspondence matrix; - determination (260) of the physiological states of the operator from the mathematical models not excluded from consideration. Figure for the abstract: Figure 2

Description

Title of the invention: Method for determining the physiological states of an operator, and associated computer program and determination system

[0001] The present invention relates to a method for determining the physiological states of an operator.

[0002] The present invention also relates to a computer program and a determination system associated with this determination method.

[0003] The technical field of the invention is that of monitoring operators operating in a critical operational context.

[0004] The invention can thus be used in any highly sensitive field in which monitoring the physiological states of an operator is of interest. This is particularly the case in the aeronautical, aerospace, railway, nuclear, medical, etc. fields.

[0005] In a manner known per se, monitoring the physiological states of an operator is a means of analyzing and evaluating their cognitive behavior in operational conditions, or even outside of these conditions, particularly for longitudinal monitoring.

[0006] Generally, such an analysis relies on means enabling the capture and analysis of physiological and cognitive parameters, supplemented by contextual parameters.

[0007] Physiological data may include any parameter relating to the vitality of the human body such as body temperature, heart rate, oxygen saturation, or respiratory rate.

[0008] This data is generally obtained from a plurality of specific sensors optimized for measuring it. Furthermore, this data is generally associated with signal processing such as filtering to reduce or even eliminate various types of signal noise that could degrade the markers to be observed. The aim is to provide the appropriate level of signal processing to obtain a signal clean enough to be used in models and achieve the desired level of detection or prediction performance.

[0009] According to the methods of the prior art, the problem of monitoring the physiological data of an operator is solved by using traditional means such as specific sensors, detection or prediction models and algorithms for processing signals from the sensors.

[0010] These solutions are based mainly on the extraction of interesting features from physiological signals, followed by the use of detection or prediction algorithms, such as artificial intelligence algorithms, to best separate the detection or prediction of the different physiological states.

[0011] According to the prior art, initially, specific sensors are used to capture physiological signals such as heart rate, oxygen saturation, body temperature, etc. Then, signal processing techniques are applied to these signals to reduce unwanted noise and disturbances in order to obtain higher quality signals.

[0012] When these signals are preprocessed, the prior art proposes to extract relevant features from these physiological signals. These features may include parameters such as heart rate variability, heart rate peaks, oxygen saturation variation, etc. The objective here is to select the most discriminating features that allow for the effective differentiation of different physiological states.

[0013] Subsequently, these features are used as input for detection or prediction algorithms, such as artificial intelligence algorithms, which are trained to recognize the patterns and motifs characteristic of each physiological state. These algorithms may include classification and regression techniques, or even neural networks, to perform the detection or prediction of specific physiological states.

[0014] However, the solutions in the prior art have a number of defects.

[0015] First of all, traditional solutions are often limited by the availability and reliability of sensors.

[0016] In particular, in the event of a sensor failure, the analysis of physiological signals may be compromised, which may lead to errors in detection or prediction or an inability to detect abnormal physiological states.

[0017] Furthermore, existing solutions do not optimally address sensor availability management. If a sensor fails, it is often necessary to completely shut down or recalibrate the system, resulting in monitoring interruptions and delays in restoring normal operation.

[0018] Existing solutions therefore lack flexibility and the ability to handle cases where models for detecting or predicting physiological states are unavailable due to faulty sensors.

[0019] This means that in the absence of certain sensors, it is difficult, if not impossible, to obtain reliable results on the physiological states of the operator.

[0020] The present invention aims to remedy these drawbacks and to propose a solution enabling the determination of an operator's physiological states in a reliable manner even when one or more sensors are no longer available.

[0021] In addition, the solution is particularly flexible insofar as it can easily adapt to a possible loss of one or more sensors.

[0022] To this end, the invention relates to a method for determining the physiological states of an operator;

[0023] the method comprising a first preliminary step of determining a correspondence matrix between a mathematical model enabling the determination of a physiological state of the operator and a set of data types necessary for the implementation of the mathematical model;

[0024] the process further comprising the following steps:

[0025] - verification of the availability of each sensor enabling the provision of a type of data ;

[0026] - when a sensor is unavailable, exclusion from consideration of each model mathematical using the data type corresponding to this sensor, according to the correspondence matrix;

[0027] - determination of the operator's physiological states from the models mathematics not excluded from consideration.

[0028] According to other advantageous features of the invention, the method comprises one or more of the following features, taken individually or in all technically possible combinations:

[0029] - the method further comprising a step of substituting at least one model mathematical excluded from consideration by a mathematical sub-model allowing the determination of the same physiological state from a set of data types without the corresponding unavailable sensor data type;

[0030] - the method further comprising a second preliminary determination step of an incidence matrix indicating the level of incidence of each physiological state on each other physiological state;

[0031] - each incidence level corresponds to the probability of transitioning from a state physiological to another physiological state;

[0032] - the method further comprising a step of determining at least one state physiological corresponding to a mathematical model excluded from consideration, by an aggregation model using the or each physiological state inducing this physiological state corresponding to the mathematical model excluded from consideration, in accordance with the incidence matrix;

[0033] - said physiological state corresponding to the mathematical model excluded from consideration is determined with a level of confidence determined according to the level of incidence of each physiological state used by the aggregation model;

[0034] - each physiological state is chosen from the group comprising at least:

[0035] - hypoxia;

[0036] - loss of consciousness;

[0037] - stress;

[0038] - fatigue;

[0039] - spatial disorientation;

[0040] - the black veil;

[0041] - dehydration;

[0042] - mental load;

[0043] - attentional wandering;

[0044] - hyperventilation;

[0045] - hypoglycemia;

[0046] -visual or attentional tunneling;

[0047] - each data type is chosen from the group comprising at least:

[0048] - Heart rate, in particular at different frequencies;

[0049] - Oxygen saturation;

[0050] - Electroencephalogram;

[0051] - Respiratory rate;

[0052] - Body temperature;

[0053] - Images;

[0054] - Acoustic data;

[0055] - Eye activity;

[0056] - Head position and acceleration;

[0057] - Blood glucose levels;

[0058] - Skin conductance;

[0059] - Muscular activity;

[0060] - Photoplethysmography;

[0061] - Functional near-infrared spectroscopy.

[0062] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a process as defined above.

[0063] The invention finally relates to a system for determining the physiological states of an operator comprising technical means configured to implement a process as defined above.

[0064] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the accompanying drawings in which: - [Fig.1] [Fig.1] is a schematic view of a determination system according to the invention; - [Fig.2] [Fig.2] is a flowchart of a determination process of physiological states of an operator, the determination process being implemented by the determination system of the [Fig.1]; - [Fig.3] [Fig.4] [Fig.5] [Fig.6] [Fig.7] [Fig.8] [Fig.9] Figures 3 to 9 are different illustrations of the implementation of different stages of the process of [Fig.2].

[0065] A system for determining the physiological states of an operator according to the invention has indeed been represented on [Fig.1].

[0066] The term "operator" refers to any person operating a machine, vehicle, or other sensitive system. For example, such an operator pilots an aircraft, such as an airplane. In such a case, this operator is an airplane pilot.

[0067] Alternatively, the operator presents any other crew member or a ground operator performing maintenance on an aircraft.

[0068] By aircraft, we mean any flying machine that can be piloted from its cockpit, as is the case for example with an airplane or a helicopter, or remotely from it, as is the case for example with a drone.

[0069] In general, the concept of operator can be applied to any other person carrying out a critical mission, for example in the field of transport (rail or heavy goods vehicle or any transport for example) or in the nuclear or space field, or in medicine.

[0070] The determination system 10 according to the invention makes it possible to determine a plurality of physiological states of this operator.

[0071] In particular, by physiological state, we mean any state of the operator likely to influence his physical work or his cognitive work.

[0072] Each physiological state is, for example, chosen from the group comprising at least: - hypoxia; - loss of consciousness; - stress; - fatigue; - spatial disorientation; - the black veil (or G-LOC in English); - dehydration; - the mental load; - attentional wandering; - hyperventilation; - hypoglycemia; - visual or attentional tunneling.

[0073] The determination system 10 makes it possible to determine a physiological state of the operator from a plurality of data provided by a plurality of sensors 12.

[0074] These sensors 12 are, for example, located on or near the operator's body, or in the area where they perform their activities, such as the aircraft cockpit. For example, these sensors 12 are fixed and / or removable in the cockpit of the aircraft piloted by the operator.

[0075] Each sensor 12 allows to provide a type of data which is determined according to the nature of this sensor 12.

[0076] Thus, for example, each sensor 12 makes it possible to provide a type of data chosen from the group comprising at least: - Heart rate, for example at different frequencies (60 Hz or 120 Hz for example); - Oxygen saturation; - Electroencephalogram; - Respiratory rate; - Body temperature; - Images; - Acoustic data; - Eye activity (pupillometry, eye opening, direction of gaze); - Head position and acceleration; - Blood glucose levels; - Skin conductance (or GSR from the English "Galvanic Skin Response"); - Muscle activity (electromyogram, EMG); - Photoplethysmography (PPG); - Functional near-infrared spectroscopy (or fNIRS from the English " Functional near-infrared spectroscopy).

[0077] Advantageously, each type of data corresponds to a measurable physiological data in relation to the human body or the environment in which it is located.

[0078] With reference to [Fig.1], the determination system 10 comprises an input module 21, a processing module 22 and an output module 23.

[0079] The input module 21 allows the acquisition of all the data generated by the sensors 12. To do this, the input module 21 is connected to each of these sensors by any technically possible means. This means may include direct or indirect connection cables allowing, for example, the transmission of physiological data acquired by the corresponding sensor via a wired or wireless computer network.

[0080] The input module 21 is further connected to a database 25 for storing a correspondence matrix and an incidence matrix, the meanings of which will be explained in more detail later.

[0081] The processing module 22 allows processing all the data acquired by the input module 21 in order to determine the physiological states of the operator.

[0082] In addition, the processing module 22 allows the correspondence and incidence matrices to be determined, as will be explained in more detail later.

[0083] Finally, the output module 23 allows the processing results of the processing module 22 to be provided to any interested system.

[0084] Such an interested system includes, for example, a human-machine interaction interface or any other external system for which the physiological states of the operator are of interest.

[0085] Each of the modules 21 to 23 advantageously presents at least partially a software implemented by a processor and stored in a memory.

[0086] In such a case, the determination system 10 further comprises such a processor as well as such memory.

[0087] Alternatively or optionally, each of these modules 21 to 23 presents at least partially a programmable logic circuit, for example of the FPGA type (from the English "Field Programmable Gate Array").

[0088] The determination system 10 makes it possible to implement a method for determining the physiological states of the operator which will henceforth be explained with reference to [Fig.2] presenting a flowchart of these steps.

[0089] The determination process first includes a preliminary phase PP during which the correspondence and incidence matrices are determined.

[0090] In particular, during a first preliminary step 110, the processing module 22 defines a mathematical model allowing the determination of a physiological state of the operator and a set of data types necessary for the implementation of this mathematical model.

[0091] This is then done by analyzing each mathematical model used to determine the corresponding physiological state. Each mathematical model is, for example, known in itself and can be obtained theoretically and / or empirically.

[0092] An example of such a determination matrix is ​​illustrated in [Fig.3].

[0093] In particular, in the example of this [Fig.3], the correspondence matrix comprises four mathematical models forming the four columns of these matrices allowing the determination of physiological states of the operator.

[0094] These models are numbered from M1 to M4 and correspond, for example, to models for determining hypoxia, loss of consciousness, stress, and fatigue.

[0095] The columns of this correspondence matrix are formed by the type of data that can be used to implement the corresponding mathematical model.

[0096] In the example in [Fig. 3], five data types numbered from DI to D5 are represented. These types of data correspond, for example, to heart rate at 60 Hz, heart rate at 120 Hz, oxygen saturation, electroencephalogram, and respiratory rate.

[0097] According to the example in [Fig.3], the first model Ml then makes it possible to detect the state hypoxia from data D2, D3 and D5 corresponding respectively to heart rate at 120 Hz, oxygen saturation and respiratory rate.

[0098] According to the same example, the model for determining a loss of consciousness, i.e. the second model M2, makes it possible to determine such a loss of consciousness based on the data D1, D3, D4 and D5 corresponding respectively to the heart rate at 60 Hz, oxygen saturation, electroencephalogram and respiratory rate.

[0099] In other words, when a data type is required to implement the corresponding mathematical model, the correspondence matrix indicates this in the intersection of the corresponding row and column, for example by placing a value of "1". Otherwise, a value of zero is placed in such an intersection.

[0100] In a second preliminary step 120, the processing module 22 determines an incidence matrix indicating the level of incidence of each physiological state to each other physiological state.

[0101] In other words, such an incidence matrix makes it possible to see the correlations between the different states, and in particular, to determine the cases where one physiological state induces another physiological state according to a level of incidence. Such a level of incidence has, for example, a value between 0 and 1 and corresponds to the probability with which a first physiological state induces a second physiological state.

[0102] Alternatively, such a value may represent the percentage of cases where the first physiological state induces the second physiological state.

[0103] When the incidence level is equal to "0", the first state does not induce the second state. In other words, the second state is independent of the first state.

[0104] Conversely, when this incidence level is equal to "1", the first state always induces the second state. For example, it may be the same state or physiological states that systematically occur at the same time.

[0105] An example of such an incidence matrix is ​​given in [Fig.4].

[0106] In the example of this figure, it is then clear that the state of hypoxia, corresponding to the SI state, induces the state of loss of consciousness (state S2) with a probability of 0.4, stress (state S3) with a probability of 0.8 and fatigue (state S4) with a probability of 0.7.

[0107] The incidence matrix is, for example, determined by analyzing statistical data according to their nature and, for example, according to the nature of the operator's task. Such an analysis may, for example, include determining correlations between different physiological states of the operator.

[0108] The two preliminary steps 110 and 120 are therefore implemented at least once before the other steps of the determination process.

[0109] Alternatively, these two steps are implemented at each update of the determination system 10 when, for example, a new state and / or a new sensor are integrated into such a system.

[0110] The subsequent steps of the process then form a PA analysis phase allowing the determination of the operator's physiological states using the matrices as defined above. These two matrices are, for example, stored in the database 25 and thus accessible at any time by the input module 21.

[0111] During an initial step 210 of the PA analysis phase, the input module 21 receives all the measurements provided by the sensors 12.

[0112] Then, in certain embodiments, the input module 21 implements a preprocessing of this data.

[0113] Such preprocessing may include data filtering to eliminate possible noise, as well as any other type of analog and / or digital preprocessing known per se in the prior art.

[0114] According to some embodiments, during this step, the input module 21 further performs contextual preprocessing of the data provided by the sensors to determine a number of characteristics relating to this data.

[0115] These characteristics may include, for example, parameters such as heart rate variability, heart rate peaks, variations in oxygen saturation, etc.

[0116] This type of contextual processing is also known as such and will not be explained further.

[0117] Furthermore, hereafter, data provided by a sensor is understood to mean either raw data provided by the sensor and possibly converted by the input module 21 in a numerical data, i.e. a characteristic generated by this input module 21 from the raw data provided by the corresponding sensor.

[0118] In any event, the nature of the data generated by the input module 21 (i.e. either the raw data or the features) depends on each mathematical model used by the processing module as will be explained in more detail later.

[0119] In a subsequent step 220, the processing module 22 receives the set of data generated and / or received by the input module 21 and checks the availability of each sensor 12.

[0120] For this purpose, the processing module 22 can, for example, analyze the consistency of the data provided by each sensor and, when this data is inconsistent, determine that it is a faulty sensor which is then reconsidered as unavailable thereafter.

[0121] The processing module 22 can further determine that a sensor is unavailable when no data has been provided by such a sensor.

[0122] Alternatively or in addition, the processing module 22 concludes that a sensor is unavailable by using any other information that can be transmitted by external systems.

[0123] Thus, for example, a separate sensor operation monitoring system can be used to provide the operating status of each sensor to the processing module 22.

[0124] Alternatively, the operator can himself indicate a faulty sensor so that it is then considered by the processing module 22 as an unavailable sensor.

[0125] When all sensors are available, the processing module 22 determines physiological states from each mathematical model using the set of data provided by the input module 21.

[0126] Then, in a step 230, the processing module 22 transmits this data to the output module 23, which then provides the determined states to any interested system.

[0127] When, on the contrary, at least one of the sensors is considered unavailable, the processing module 22 implements step 240 in which the processing module 22 excludes from any future consideration each mathematical model using the data type corresponding to such an unavailable sensor.

[0128] For this purpose, the processing module 22 uses the correspondence matrix which indicates for each type of data, the mathematical models using this type of data to determine the corresponding physiological states.

[0129] In the example in [Fig.5], the sensor providing data type D4 is considered unavailable.

[0130] Thus, during step 240, the processing module 22 excludes from consideration the mathematical model M2 corresponding to the only model using the data type D4.

[0131] In an optional subsequent step 250, the processing module 22 substitutes the mathematical model or models excluded from consideration in step 240 with a sub-model that avoids the use of the type of data that should be provided by the unavailable sensor or sensors.

[0132] The implementation of this step 250 is then conditional depending on the mathematical model excluded from consideration during step 240.

[0133] In particular, when such a model excluded from consideration allows the definition of a sub-model which does not use the type of data to be provided by the unavailable sensor, step 250 can be implemented in relation to this sub-model.

[0134] Fig. 6 illustrates such an example in which a sub-model SM2 has been added in the correspondence matrix which can substitute the mathematical model M2 when the data type D4 is unavailable.

[0135] In the next step 260, the processing module 22 determines physiological states of the operator from the non-excluded mathematical models of consideration.

[0136] In particular, when no mathematical model excluded from consideration in step 240 has been substituted by a sub-model in step 250, the processing module 22 uses a reduced number of models to determine the corresponding physiological states.

[0137] This is illustrated in the example of [Fig.7] in which only the mathematical models M1, M3 and M4 are used following the exclusion of consideration of model M2.

[0138] When, following the exclusion of a mathematical model, the processing module 22 has substituted this excluded model with a sub-model, the processing module 22 then determines during this step 260 the physiological states from the set of available mathematical models and sub-models determined during step 250.

[0139] Optionally, the physiological state determined in step 260 by a mathematical sub-model can be marked accordingly. In other words, a marking indicating that this physiological state was determined from a sub-model and not the usual model can be used.

[0140] During the next optional step 270, the processing module 22 determines at least one physiological state corresponding to a mathematical model excluded from consideration during step 240 and not replaced by a sub-model during step 250.

[0141] To do this, the processing module 22 uses the physiological state or each physiological state inducing the corresponding physiological state of the excluded mathematical model, in accordance with the incidence matrix explained previously.

[0142] In the example of [Fig.8], the physiological state S2 cannot be determined by the corresponding mathematical model, i.e. by the mathematical model M2 which was then excluded following the loss of data type D4.

[0143] However, in such a case, the processing module 22 can use the physiological state SI and the physiological state S4 which have respectively the level of incidence on the physiological state S2 equal to 0.4 and 0.1.

[0144] To determine the physiological state from the physiological states inducing such a state according to the incidence matrix, the processing module 22 uses, for example, an aggregation model designed for this purpose.

[0145] This can then be illustrated schematically in [Fig. 9], in which, following the loss of the sensor providing data type D4, the mathematical model M2 cannot be used to determine the physiological state S2. In such a case, the physiological states S4 and SI are used by an aggregation model AM2 to obtain a physiological state S2.

[0146] Advantageously, during this same step 270, the processing module 22 further determines a confidence level of said physiological state determined using other physiological states.

[0147] This confidence level is determined, for example, according to the level of incidence of the or each physiological state used by the corresponding aggregation model.

[0148] In the example of [Fig.9], the confidence level determined for the physiological state S2 then depends on the incidence levels p, 2 and p, 4 determined in accordance with the incidence matrix.

[0149] Finally, in a subsequent step 280, the output module 23 transmits the set of determined physiological states to any interested system.

[0150] Furthermore, when at least one physiological state has been determined using at least one other physiological state by an aggregation model, the output module 23 further provides the confidence level associated with that physiological state. Similarly, when at least one physiological state has been determined using a sub-model instead of the corresponding model, the output module 23 further provides the corresponding marking.

[0151] Next, the steps of the PA analysis phase can be implemented again using other data acquired by sensors 12.

[0152] The present invention therefore has a number of advantages.

[0153] First of all, it is clear that the invention makes it possible to highlight links between the data provided by the different sensors and the physiological states that can be determined from this data.

[0154] Thus, in the event of the unavailability of at least one of the sensors, the invention makes it possible to exclude or easily replace the mathematical model using the type of data from that sensor to determine the corresponding physiological state.

[0155] This then presents a great flexibility of the invention compared to the solutions known in the state of the art.

[0156] In addition, the invention makes it possible to establish links between the different states and in the event of the inability to calculate one of the states by the corresponding mathematical model to use the other states to then estimate this last state.

[0157] In such a case, a confidence level may be provided in order to warn any other interested system that the corresponding physiological state has been obtained by indirect means.

Claims

Demands

1. A method for determining the physiological states of an operator; the method comprising a first preliminary step (110) of determining a correspondence matrix between a mathematical model enabling the determination of a physiological state of the operator and a set of data types necessary for the implementation of the mathematical model; the method further comprising the following steps: - verification (220) of the availability of each sensor (12) enabling the provision of a data type; - when a sensor (12) is unavailable, exclusion (240) from consideration of each mathematical model using the data type corresponding to that sensor (12), in accordance with the correspondence matrix; - determination (260) of the physiological states of the operator from the mathematical models not excluded from consideration.

2. A method according to claim 1, further comprising a step (250) of substituting at least one mathematical model excluded from consideration by a mathematical sub-model enabling the determination of the same physiological state from a set of data types without the corresponding unavailable sensor (12) data type.

3. A method according to any one of the preceding claims, further comprising a second preliminary step (120) of determining an incidence matrix indicating the level of incidence of each physiological state on each other physiological state.

4. A method according to claim 3, wherein each level of incidence corresponds to the probability of passing from one physiological state to another physiological state.

5. A method according to claim 3 or 4, further comprising a step (270) of determining at least one physiological state corresponding to an excluded mathematical model, by an aggregation model using the physiological state or each physiological state inducing that physiological state corresponding to the excluded mathematical model, in accordance with the incidence matrix.

6. Method according to claim 5, wherein said physiological state corresponding to the excluded mathematical model is determined with a level of confidence determined according to the level of incidence of the or each physiological state used by the aggregation model.

7. A method according to any one of the preceding claims, wherein each physiological state is chosen from the group comprising at least: - hypoxia; - loss of consciousness; - stress; - fatigue; - spatial disorientation; - blackout; - dehydration; - mental load; - attentional wandering; - hyperventilation; - hypoglycemia; - visual or attentional tunneling.

8. A method according to any one of the preceding claims, wherein each type of data is selected from the group comprising at least: - Heart rate, in particular at different frequencies; - Oxygen saturation; - Electroencephalogram; - Respiratory rate; - Body temperature; - Images; - Acoustic data; - Eye activity; - Head position and acceleration; - Blood glucose levels; - Skin conductance; - Muscle activity; - Photoplethysmography; 16 - Functional near-infrared spectroscopy.

9. A computer program comprising software instructions which, when executed by a computer, implement a method according to any one of the preceding claims.

10. System for determining (10) physiological states of an operator comprising technical means (21, 22, 23) configured to implement a method according to any one of claims 1 to 8.

Images