Biometric and attitude information collection device for rotorcraft pilot consciousness tracking

Abstract

To prevent an aircraft accident caused by unconsciousness by continuously tracking a consciousness state of an aircraft pilot.SOLUTION: A device for collecting biometric and posture information for tracking consciousness of a rotorcraft pilot, collects a physiological signal, posture information, etc. of a rotorcraft pilot by comprehensively utilizing cameras, on-body wearable devices, control stick pressure detection sensors, etc. while an aircraft pilot is in flight to track a state of consciousness of the rotorcraft pilot in real time, transmits the collected biometric information to an integrated data processing and storage device with a built-in flight data recorder, and generates a warning signal upon identification of singularity associated with loss of consciousness.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a device for collecting physiological signals and attitude information of an aircraft pilot by comprehensively utilizing cameras, body-worn devices, joystick pressure sensors, etc. while the pilot is flying, tracking the pilot's state of consciousness in real time, and transmitting the collected biological information to an integrated data processing and storage device with a built-in flight recorder. [Background technology]

[0002] Military rotorcraft pilots are constantly exposed to stress due to numerous factors such as battlefield conditions, mission characteristics, and aircraft features (maneuvering characteristics, vibrations, etc.), and can lose consciousness during flight missions due to high maneuverability, physical characteristics (fainting due to physical and mental stress), attacks, etc. Pilot loss of consciousness can lead to unstable aircraft attitude, forced landing, or crash, which is why a consciousness tracking device based on biometric and attitude information is needed.

[0003] The prior art has proposed a device and method for tracking the temperature of a pilot's hands, heart rate, etc., using a Hotas-type military aircraft control stick, which is mainly used for fixed-wing aircraft, and has a configuration for monitoring the pilot's condition using this. However, there has been no example of a device that comprehensively collects body and posture information such as the pilot's physiological signals, pressure signals applied to the collective and cyclic control sticks and pedals, and pilot posture information (including head and shoulder positions), and tracks consciousness based on this. Summary of the Invention [Problem to be solved by the invention]

[0004] The purpose of the present invention is to continuously track the consciousness state of an aircraft pilot through a monitoring device (including a camera, signal receiving and processing device, etc.) that can check the pilot's physiological phenomena (pupil response, blood pressure, pulse, electrocardiogram, oxygen saturation, etc.) and the pilot's posture information (including head and shoulder position, etc.), and to prevent aircraft accidents caused by loss of consciousness by transmitting the collected vital signs to a flight recorder and mission computer, etc. [Means for solving the problem]

[0005] In order to achieve the above-mentioned object, the present invention provides a biological and attitude information collecting device for tracking the consciousness of an aircraft pilot, including: a body-worn device worn by an aircraft pilot to collect physiological body information; a flight control system pressure signal sensing device that senses the pressure applied to the stick and pedals used to control the aircraft; a pilot consciousness and attitude tracking and learning device that directly tracks the pilot's pupil reaction and attitude information and receives the physiological body information from the body-worn device to derive a singularity related to the pilot's loss of consciousness; and an integrated data processing and storage device that is responsible for status monitoring functions related to the aircraft and pilot and for recording voice and flight data, processes and records information received from the pilot consciousness and attitude tracking and learning device and the flight control system pressure signal sensing device, and generates a warning signal when a singularity related to loss of consciousness is identified.

[0006] In the present invention, the aircraft may be a military rotorcraft.

[0007] In the present invention, the body worn device may include a physiological signal sensing module; a Bluetooth transceiver module; and a power supply module.

[0008] In the present invention, the physiological signal sensing module includes a blood pressure sensor; a heart rate sensor; an electrocardiogram sensor; a body temperature sensing sensor; and an oxygen saturation sensor, and may be located on a body part including the pilot's wrist.

[0009] In the present invention, the Bluetooth transceiver module can transmit physiological body information collected from the physiological signal sensing module to the pilot awareness and attitude tracking and learning device.

[0010] In the present invention, the power supply module applies power to the physiological signal sensing module and the Bluetooth transceiver module, and may be in the form of a built-in battery.

[0011] In the present invention, the flight control system pressure signal sensing device can include a pressure sensing module; a data transmitting and receiving module; and a power supply module.

[0012] In the present invention, the pressure sensing module can include: a cyclic stick pressure sensing sensor; a collective stick pressure sensing sensor; and a pedal pressure sensing sensor.

[0013] In the present invention, the cyclic stick pressure detection sensor may be located at the lower center of the front of the cockpit adjacent to the pilot, the collective stick pressure detection sensor may be located at the lower left side of the cockpit adjacent to the pilot, and the pedal pressure detection sensor may be located at the lower front of the cockpit adjacent to the pilot.

[0014] In the present invention, the data transceiver module can transmit information collected by the pressure sensing module to the pilot awareness and attitude tracking and learning device, and the integrated data processing and storage device.

[0015] In the present invention, the power supply module can apply power to the pressure sensing module and the data transmitting and receiving module.

[0016] In the present invention, the pilot awareness and attitude tracking and learning device can include: a pupil response and attitude information sensing module; a Bluetooth transceiver module; a signal information processing and learning module; a data transceiver module; and a power supply module.

[0017] In the present invention, the pupil response and posture information sensing module can include a camera for checking the face and posture; and an infrared light for checking the pupil response.

[0018] In the present invention, the face and attitude confirmation camera can collect the change state of the face including the pupils, and attitude information including the pilot's head, shoulders and arms in real time, and provide it to the signal information processing and learning module.

[0019] In the present invention, attitude information related to the pilot's head, shoulders, and arms can be collected by having the face and attitude confirmation cameras track an imaginary vertical line connecting the protrusions of the pilot's face, including the nose and chin, to the center of the torso, and an imaginary horizontal line crossing both shoulders.

[0020] In the present invention, the movement lines collected during pre-flight mission equipment inspection and the movement lines collected during flight stick and equipment operation are learned through a signal information processing and learning module, and when the pilot's attitude deviates from the learning range and approaches an abnormal angle, a pilot attitude abnormality signal can be sent from the signal information processing and learning module.

[0021] In the present invention, the infrared light for checking pupil reflex is equipment for checking the pupil reflex of a pilot, and can induce a change in pupil size by irradiating infrared light with a wavelength in the infrared band that is less harmful to the human body toward the pupil.

[0022] In the present invention, the change in the size of the pilot's pupils is observed by tracking the pupil reaction with a facial and attitude confirmation camera; and after checking the state of the eyes without pupil reaction more than twice during pre-flight mission equipment inspection, the pre-reaction pupil diameter d can be learned through a signal information processing and learning module.

[0023] In the present invention, an infrared light is irradiated two or more times during pre-flight mission equipment inspection or during a flight mission to check the pupil response, and then the post-response pupil diameter d' can be learned through a signal information processing and learning module through an arithmetic average.

[0024] In the present invention, an infrared light is periodically irradiated during flight to check whether a pupil reaction occurs, and if d>d', it is determined that the consciousness is maintained due to a normal pupil reaction, and a signal is sent.

[0025] In the present invention, when infrared light is periodically irradiated during flight, if the pupil diameter d' observed through the camera is identified as being within the pre-reaction pupil diameter d±5%, the infrared light is additionally irradiated three or more times at intervals of 1±0.5 seconds to observe the pupil reaction, and the pupil diameter d" obtained by arithmetically averaging the collected pupil diameters is learned through the signal information processing and learning module. However, if d" is within d±5%, it is determined that the drone is in a state of unconsciousness due to no pupil reaction, and a signal is sent.

[0026] In the present invention, the Bluetooth transceiver module can receive physiological body information transmitted from the body-worn device worn by the pilot and transmit it to the signal information processing and learning module.

[0027] In the present invention, the signal information processing and learning module may have the functions of monitoring transmitted signals and information, learning and storing the physiological body information and attitude information of the pilot based on the collected data, and identifying unusual points related to loss of consciousness confirmed from the body information and attitude information.

[0028] In the present invention, in relation to the pilot's physiological body information; the signal information processing and learning module: receives information related to blood pressure, heart rate, electrocardiogram, body temperature and oxygen saturation from the body-worn device at intervals of 5±2 minutes and learns the average value in a state of stable consciousness by taking the arithmetic average for each item; receives pupil diameter d in a state of stable consciousness from the pupil response and attitude information detection module and learns it; and if signal information outside the learned range is input, it can be determined to be an abnormal input and processed.

[0029] In the present invention, in relation to the pilot's attitude information, the signal information processing and learning module: tracks the movement lines corresponding to the positions of the pilot's head, shoulders, arms and hands through the face and attitude confirmation cameras when inspecting mission equipment before flight or operating equipment during flight, and then learns the normal operating movement lines; tracks the operating direction and pressure applied to the stick and pedals through the flight control system pressure signal sensing device when inspecting mission equipment before flight or operating equipment during flight, and then learns the operating direction and pressure range applied at a normal level of consciousness; and if signal information outside the learned range is input, it can be determined to be an abnormal input and processed.

[0030] In the present invention, in relation to identifying singular points related to loss of consciousness based on body information and attitude information; the signal information processing and learning module: determines loss of consciousness when the pupil diameter d" is d±5% or when the pupil disappears outside the range of the upper and lower eyelids; determines loss of consciousness when the heart rate measured by the body-worn device becomes zero; determines loss of consciousness when the pilot's attitude is tracked to take an abnormal angle and the pressure measured by the flight control system pressure signal sensing device becomes zero; determines loss of consciousness when the pilot's attitude is tracked to take an abnormal angle and the flight control system pressure signal sensing device applies fixed pressure in a direction corresponding to the abnormal operating range; additional judgment criteria related to loss of consciousness can be input from the outside through the data transmission and reception module.

[0031] In the present invention, the data transmission and reception module can transmit the information collected and analyzed by the signal information processing and learning module to an integrated data processing and storage device.

[0032] In the present invention, the power supply module can supply power to the pupil response and posture information sensing module, the Bluetooth transceiver module, the signal information processing and learning module, and the data transceiver module.

[0033] In the present invention, the integrated data processing and storage device may include an attitude information processing and storage module; a biometric information processing and storage module; an audio and flight record storage device; a data transmission and reception module; and a power supply module.

[0034] In the present invention, the attitude information processing and storage module receives and stores data related to the attitude information of the aircraft and the pilot, and can generate a warning signal when a singularity occurs.

[0035] In the present invention, the biometric information processing and storage module can receive and store data related to the pilot's physiological body information and generate a warning signal when an anomaly related to pilot loss of consciousness is identified.

[0036] In the present invention, the voice and flight record storage device may be equipment that acts as a black box for storing pilot communication data and flight records.

[0037] In the present invention, the data transmission and reception module: receives information from the pilot awareness and attitude tracking and learning device, and the flight control system pressure signal sensing device, and transmits it to the attitude information processing and storage module, the bio-information processing and storage module, and the voice and flight record storage device; when a singularity occurs, the signal generated in the attitude information processing and storage module or the bio-information processing and storage module can be transmitted to a device, including a multi-function display device, for the pilot to recognize using audiovisually.

[0038] In the present invention, the power supply module can supply power to the attitude information processing and storage module, the vital sign information processing and storage module, the voice and flight record storage device, and the data transmission and reception module. Effect of the Invention

[0039] The biometric information collecting device proposed in the present invention can continuously track the consciousness state of a pilot during aerial missions and ground training by comprehensively monitoring the physiological body information (blood pressure, heart rate, electrocardiogram, body temperature, oxygen saturation, etc.) of an aircraft pilot, including pupil reaction, head position, piloting posture, and pressure applied to the control stick. In particular, the invented device can observe and collect the pilot's body and posture information during pre-flight mission equipment inspection and simulator training on the ground, so that it is possible to identify in advance any unusual points related to the pilot's loss of consciousness. Based on these device characteristics, it will be possible to ensure the safety of pilots during missions and prevent aircraft accidents caused by the pilot's loss of consciousness. In addition, it is expected that the collected biometric information (limited to matters for which consent has been given for the use of personal information) can be applied to big data analysis, etc., and can be used for personalized health care for pilots and biorhythm management at the aircraft operation unit level. [Brief description of the drawings]

[0040] [Figure 1] This is a block diagram of a biometric information collection device for tracking the consciousness of an aircraft pilot according to the present invention, in which P represents power supply, D represents data transmission and reception according to the MIL-STD-1553B communication protocol, and B represents data transmission and reception via Bluetooth. [Diagram 2] 1 is a conceptual diagram of a biometric information collection device for tracking the consciousness of an aircraft pilot according to the present invention. [Diagram 3] FIG. 1 is an example of pilot pupil response tracking using a facial and attitude confirmation camera and infrared light according to the present invention. (a) shows the state of the eye before pupil response confirmation, (b) shows a conscious state (pupillary response (constriction), d>d'), (c) shows an unconscious state (no pupil response, d≒d"). N1 is the nerve that rotates the eyes, N2 is the optic nerve, d is the pupil diameter before the test, and d' and d" are the pupil diameter after the test. [Figure 4] 1A and 1B are diagrams illustrating stick operation posture tracking using a face unit and a posture confirmation camera according to the present invention, where (a) shows stick operation posture tracking in the front, and (b) shows stick operation posture tracking in the side. [Diagram 5] FIG. 1 illustrates an example of head pose and position tracking using a facial and pose confirmation camera according to the present invention, where (a) shows normal head pose tracking, and (b), (c), and (d) show abnormal head pose tracking. [Figure 6] 1 illustrates an example of physiological body information monitoring through a body-worn device according to the present invention, where I1 indicates blood pressure and heart rate, I2 indicates electrocardiogram and temperature, and I3 indicates oxygen saturation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] The present invention will now be described in detail with reference to the accompanying drawings.

[0042] The present invention relates to a biometric and attitude information collection device for aircraft pilot consciousness tracking. The present invention is preferably applicable to military rotorcraft such as helicopters, but is also applicable to other military aircraft as well as civilian aircraft.

[0043] Loss of consciousness of a pilot can lead to instability of the aircraft's attitude, emergency landing or crash. In the present invention, by monitoring bio-information that combines the pilot's physiological signals and attitude information in real time and storing the collected bio-information, it is possible to ensure the safety of the pilot during missions as well as to use it for future analysis of bio-rhythm signal big data for the performance of pilot duties.

[0044] Referring to FIG. 1 and FIG. 2, the biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot according to the present invention comprises: 1) Pilot awareness and attitude tracking and learning device 10 including a pupil response and attitude information sensing module 11 consisting of a face and attitude confirmation camera 12 and an infrared light 13 for pupil response confirmation, a Bluetooth transmission and reception module 14, a data transmission and reception module 15, a signal information processing and learning module 16, etc.; 2) A Bluetooth-based body-worn device with physiological body information verification function20; 3) A flight control system pressure signal sensing device 30 including a control stick (cyclic stick, collective stick, pedal) pressure sensing module 31, a pressure signal (data) transmission / reception module 35, etc.; and 4) It may be configured as a combination of an integrated data processing and storage device 40 including an attitude information processing and storage module 41, a biometric information processing and storage module 42, an audio and flight record storage device 43, a data transmission and reception module 44, etc.

[0045] The pilot awareness and attitude tracking and learning device 10 may be a subsystem that directly tracks the pilot's pupil response and attitude information, receives physiological body information from the body worn device 20, and performs the role of deriving singularities related to the pilot's loss of consciousness.

[0046] The body worn device 20 may be a subordinate device that is worn on a part of the pilot's body, such as the pilot's wrist, to collect physiological body information and transmit the information to the pilot consciousness and attitude tracking and learning device 10 via Bluetooth transmission and reception.

[0047] The flight control system pressure signal sensing device 30 may be a subsystem that performs the function of sensing pressure applied to the cyclic stick, collective stick, and pedals used to control a rotorcraft, and transmitting the collected information to the pilot awareness and attitude tracking and learning device 10 and the integrated data processing and storage device 40.

[0048] The integrated data processing and storage device 40 may be a subsystem responsible for aircraft and pilot-related status monitoring functions and voice and flight data recording, processing and recording information received from the pilot consciousness and attitude tracking and learning device 10 and the flight control system pressure signal sensing device 30, and generating a warning signal when an anomaly related to loss of consciousness is identified.

[0049] The pilot awareness and attitude tracking and learning device 10 may be comprised of a pupil response and attitude information sensing module 11, a Bluetooth transceiver module 14, a data transceiver module 15, a signal information processing and learning module 16, and a power supply module 17. The pupil response and attitude information sensing module 11 may be comprised of a face and attitude confirmation camera 12, and an infrared light 13 for pupil response confirmation.

[0050] The face and attitude confirmation camera 12 can collect real-time information on changes in key parts of the face, including the pupils, and attitude information such as the positions of the pilot's head, shoulders, and arms when operating various instruments for flying, and provide this information to the signal information processing and learning module 16.

[0051] Referring to Figures 4 and 5, a method of collecting attitude information related to the pilot's head, shoulders and arms may be performed by having the face and attitude confirmation camera 12 track an imaginary vertical line connecting the center of the pilot's torso including facial protrusions such as the pilot's nose and chin, and an imaginary horizontal line crossing both shoulders.

[0052] The traffic patterns collected during pre-flight mission equipment inspection and the traffic patterns collected during flight stick and equipment operation are learned through the signal information processing and learning module 16, and when the pilot's attitude approaches an abnormal angle outside the learning range, a pilot attitude abnormality signal can be sent from the signal information processing and learning module 16.

[0053] The infrared light 13 for checking pupil reflex is equipment for checking the pupil reflex of the pilot, and can play a role of inducing a change in pupil size by irradiating the pupil with infrared band wavelengths that are less harmful to the human body.

[0054] Referring to FIG. 3, a method of observing changes in the size of the pilot's pupils may be accomplished by tracking pupil responses with a facial and attitude camera 12.

[0055] During pre-flight mission equipment inspection, the state of the eyeball (pupil diameter) without pupil reaction is checked at least twice, and then the pre-reaction pupil diameter d can be learned through the signal information processing and learning module 16.

[0056] Thereafter, during pre-flight mission equipment inspection or during the performance of a flight mission, the infrared light 13 is irradiated two or more times to check the pupil reaction, and the post-reaction pupil diameter d' can be learned through the signal information processing and learning module 16 through the arithmetic mean.

[0057] Thereafter, infrared light 13 is periodically illuminated during flight to check whether a pupil reaction occurs, and if d>d', it is determined that consciousness is maintained due to a normal pupil reaction, and a signal can be sent.

[0058] When infrared light 13 is periodically illuminated during flight, if it is determined that the pupil diameter confirmed through camera 12 is almost the same as the pupil diameter before reaction, infrared light 13 is additionally illuminated three or more times at intervals of 1±0.5 seconds to observe the pupil reaction, and the pupil diameter d″, which is the arithmetic mean of the collected pupil diameters, is learned through signal information processing and learning module 16. However, if d ≒ d″ (d″ = d±5%), it can be determined that the pilot is in a state of unconsciousness due to no pupil reaction and a signal can be sent.

[0059] In this manner, the signal information processing and learning module 16 can learn and send signals related to the above-mentioned procedures.

[0060] The Bluetooth transceiver module 14 can be responsible for receiving physiological body information transmitted from a body-worn device 20 worn by the pilot and transmitting the information to the signal information processing and learning module 16 .

[0061] The signal information processing and learning module 16 may have a function of monitoring signals and information transmitted from each module, a function of learning and storing physiological and posture characteristics of the corresponding pilot based on collected data, and a function of identifying anomalies related to loss of consciousness confirmed from physical information and posture. To this end, the signal information processing and learning module 16 may include a processing device and a storage device, etc. Machine learning, deep learning, etc. may be used as a learning method.

[0062] Referring to Figures 1, 3 and 6, in relation to a method of learning and storing a pilot's physiological characteristics, information related to blood pressure, heart rate, electrocardiogram, body temperature and oxygen saturation that can be measured by the body worn device 20 is transmitted to the signal information processing and learning module 16 at intervals of 5±2 minutes, and an arithmetic average is calculated for each item, thereby learning the average value when the pilot is in a stable state of consciousness.

[0063] As described above, the pupil diameter d in a stable state of consciousness can be transmitted to and learned by the signal information processing and learning module 16. Furthermore, when signal information that does not fall within the learned range is input, the signal information processing and learning module 16 can process the input to determine that it is abnormal.

[0064] Referring to Figures 4 and 5, in relation to a method of learning and storing the pilot's posture characteristics, the face and posture confirmation camera 12 can track the movements of the pilot's head, shoulders, arms, hands, etc. during pre-flight mission equipment inspection or during in-flight equipment operation, and then transmit the movements to the signal information processing and learning module 16 to learn the normal operating movements.

[0065] In addition, the control direction and pressure applied to the cyclic stick, collective stick, and pedals during pre-flight mission equipment check or during in-flight equipment operation can be tracked through the flight control pressure signal sensing device 30, and then transmitted to the signal information processing and learning module 16 to learn the control direction and pressure range that can be applied at a normal level of consciousness. In addition, when signal information that does not fall within the learned range is input, the signal information processing and learning module 16 can process it to determine that the input is abnormal.

[0066] To enable the signal information processing and learning module 16 to identify anomalies associated with loss of consciousness based on input body and posture information, the following procedure can be performed.

[0067] First, as mentioned above, if the pupil diameter is d ≒ d″ or if the pupil disappears beyond the range of the upper and lower eyelids, loss of consciousness can be determined.

[0068] Secondly, if the heart rate measured as described above becomes zero, loss of consciousness can be determined.

[0069] Thirdly, if the pilot's attitude is tracked to be at an abnormal angle as described above, and if the pressure measured by the flight control system pressure signal sensing device 30 becomes zero as described above, it can be determined that the pilot has lost consciousness.

[0070] Fourth, as described above, if the pilot's attitude is tracked to be at an abnormal angle, and as described above, if pressure is applied fixedly in a direction that falls within the abnormal operating range in the flight control system pressure signal sensing device 30, it can be determined that the pilot has lost consciousness.

[0071] In addition to the above examples, the signal information processing and learning module 16 may receive external criteria via the data transmission and reception module 15 to enable additional judgments related to loss of consciousness based on the collected body and posture information.

[0072] The data transmission / reception module 15 can be responsible for transmitting information collected and analyzed by the signal information processing and learning module 16 to the integrated data processing and storage device 40 in accordance with the MIL-STD-1553B communication protocol.

[0073] The power supply module 17 can be responsible for supplying power to the pupil response and posture information sensing module 11, the Bluetooth transceiver module 14, the signal information processing and learning module 16, and the data transceiver module 15.

[0074] Referring to FIG. 6, the body-worn device 20 is a device worn on the pilot's wrist, etc., to collect physiological body information, and may be composed of: a physiological signal sensing module 21 responsible for sensing and collecting blood pressure, heart rate, electrocardiogram, body temperature, oxygen saturation, etc.; and a Bluetooth transceiver module 27 responsible for transmitting the collected information to the pilot consciousness and attitude tracking and learning device 10, etc.

[0075] Specifically, the body worn device 20 may be comprised of a physiological signal sensing module 21; a Bluetooth transceiver module 27; and a power supply module 28, and may be configured in the form of a watch or a band.

[0076] The physiological signal sensing module 21 is a device located on the pilot's wrist, etc., and is responsible for sensing and collecting blood pressure, heart rate, electrocardiogram, body temperature, oxygen saturation, etc., and may be provided so as to be located on a surface that comes into direct contact with the pilot. Since the physiological signal sensing module 21 is intended for tracking health conditions, it does not need to have the accuracy of a medical device, but it can provide a sensing function at the level of tracking an individual's health condition through trend confirmation.

[0077] Specifically, the physiological signal sensing module 21 may be composed of a blood pressure sensor 22, a heart rate sensor 23, an electrocardiogram sensor 24, a body temperature sensing sensor 25, and an oxygen saturation sensor 26, etc., and these sensors may be centrally located in a watch, band, etc. and configured in a modularized form.

[0078] The Bluetooth transceiver module 27 may be responsible for transmitting information collected from the physiological signal sensing module 21 to the pilot awareness and attitude tracking and learning device 10 via Bluetooth communication.

[0079] The power supply module 28 may be responsible for applying power to the physiological signal sensing module 21 and the Bluetooth transceiver module 27, and may be configured in the form of a built-in battery.

[0080] The flight control system pressure signal sensing device 30 may be composed of a pressure sensing module 31 consisting of sensors for sensing and collecting pressure applied to the cyclic stick, collective stick, and pedals; and a data transmission and reception module 35 responsible for transmitting the collected information to the pilot awareness and attitude tracking and learning device 10 and the integrated data processing and storage device 40.

[0081] Specifically, the flight control system pressure signal sensing device 30 may be comprised of a pressure sensing module 31; a data transmitting and receiving module 35; and a power supply module 36.

[0082] The pressure sensing module 31 may be comprised of a cyclic stick pressure sensing sensor 32; a collective stick pressure sensing sensor 33; and a pedal pressure sensing sensor 34 to sense pressure applied to the joystick and pedals.

[0083] A cyclic stick pressure sensor 32 may be provided located at a lower center front end of the cockpit adjacent to the pilot to detect the pilot's operating pressure applied to the cyclic control stick.

[0084] The collective stick pressure sensor 33 may be provided at a lower left end of the cockpit adjacent to the pilot to detect the pilot's operating pressure applied to the collective control stick.

[0085] A pedal pressure sensor 34 may be provided at a lower front end of the cockpit adjacent to the pilot to detect the pilot's operating pressure applied to the pedal.

[0086] The data transmission and reception module 35 can be responsible for transmitting the information collected by the pressure sensing module 31 to the pilot awareness and attitude tracking and learning device 10 and the integrated data processing and storage device 40 in accordance with the MIL-STD-1553B communication protocol.

[0087] The power supply module 36 may be responsible for applying power to the pressure sensing module 31 and the data transmitting / receiving module 35 .

[0088] The integrated data processing and storage device 40 can be responsible for aircraft and pilot-related status monitoring functions; and voice and flight data recording, etc. The integrated data processing and storage device 40 can be composed of an attitude information processing and storage module 41; a vital sign information processing and storage module 42; a voice and flight record storage device 43; a data transmission and reception module 44; and a power supply module 45.

[0089] The attitude information processing and storage module 41 may receive and store data related to the attitude information of the aircraft and the pilot, and may play a role in generating a warning signal when a singularity occurs. To this end, the attitude information processing and storage module 41 may include a processing device (processor) and a storage device.

[0090] The biometric information processing and storage module 42 may receive and store data related to the pilot's physiological body information, and generate a warning signal when a singularity related to the pilot's loss of consciousness is identified. To this end, the biometric information processing and storage module 42 may include a processing device and a storage device.

[0091] The voice and flight record storage device 43 may be equipment that acts as a black box for storing pilot communication data and flight records.

[0092] The data transmission / reception module 44 can receive information from the pilot awareness and attitude tracking and learning device 10 and the flight control system pressure signal sensing device 30 and transmit it to the attitude information processing and storage module 41, the bio-information processing and storage module 42, and the voice and flight record storage device 43. In addition, when a singularity occurs, the attitude information processing and storage module 41 or the bio-information processing and storage module 42 can generate a signal and transmit it to a device that the pilot can recognize using his / her sense of sight and hearing, such as a multi-function display device.

[0093] The power supply module 45 can be responsible for applying power to the attitude information processing and storage module 41, the vital sign information processing and storage module 42, the voice and flight record storage device 43, and the data transmission and reception module 44. [Explanation of symbols]

[0094] 10: Pilot Awareness and Attitude Tracking and Learning Device 11: Pupil response and posture information sensing module 12: Face and posture confirmation camera 13: Infrared light for checking pupil reflex 14: Bluetooth transceiver module 15: Data transmission and reception module 16: Signal information processing and learning module 17: Power supply module 20: Body-worn devices 21: Physiological signal sensing module 22: Blood pressure sensor 23: Heart rate sensor 24: Electrocardiogram sensor 25: Body temperature sensor 26: Oxygen saturation sensor 27: Bluetooth transceiver module 28: Power supply module 30: Flight control system pressure signal sensing device 31: Pressure sensing module 32: Cyclic stick pressure sensor 33: Collective stick pressure sensor 34: Pedal pressure sensor 35: Data transmission and reception module 36: Power supply module 40: Integrated data processing and storage device 41: Posture information processing and storage module 42: Biometric information processing and storage module 43: Voice and flight recorder 44: Data transmission and reception module 45: Power supply module

Claims

1. Body worn devices worn by aircraft pilots to collect physiological body information; Flight control system pressure signal sensing devices which sense pressure applied to the sticks and pedals used to control the aircraft; a pilot awareness and attitude tracking and learning device that directly tracks the pilot's pupil response and attitude information and receives physiological body information from a body worn device to derive singularities associated with the pilot's loss of consciousness; and an integrated data processing and storage device that is responsible for aircraft and pilot-related status monitoring functions and voice and flight data recording, processes and records information received from the pilot consciousness and attitude tracking and learning device and the flight control system pressure signal sensing device, and generates a warning signal when a singularity related to loss of consciousness is identified; The aircraft is a rotorcraft; The body worn device includes a physiological signal sensing module; the physiological signal sensing module includes a blood pressure sensor; a heart rate sensor; an electrocardiogram sensor; a temperature sensing sensor; and an oxygen saturation sensor; [0023] The flight control system pressure signal sensing device includes a pressure sensing module; The pressure sensing module includes a cyclic stick pressure sensing sensor; a collective stick pressure sensing sensor; and a pedal pressure sensing sensor; The pilot awareness and attitude tracking and learning device includes a pupil response and attitude information sensing module; and a signal information processing and learning module; The pupil response and posture information sensing module includes a camera for confirming face and posture; and an infrared light for confirming pupil response; A biometric and attitude information collection device for tracking pilot consciousness.

2. 2. The biometric and attitude information gathering device for aircraft pilot consciousness tracking as claimed in claim 1, wherein the aircraft is a military rotorcraft.

3. The biometric and attitude information collecting apparatus for aircraft pilot consciousness tracking as claimed in claim 1 , wherein the body worn device includes: a Bluetooth transceiver module; and a power supply module.

4. 4. The biometric and attitude information gathering device for aircraft pilot consciousness tracking as claimed in claim 3, wherein the physiological signal sensing module is located on a body part including the pilot's wrist.

5. The biometric and attitude information collecting device for tracking pilot consciousness of an aircraft as claimed in claim 3, wherein the Bluetooth transceiver module transmits the physiological body information collected from the physiological signal sensing module to the pilot consciousness and attitude tracking and learning device.

6. 4. The biometric and attitude information collecting device for tracking pilot consciousness of an aircraft according to claim 3, wherein the power supply module applies power to the physiological signal sensing module and the Bluetooth transceiver module and is in the form of a built-in battery.

7. 2. The vital signs and attitude information collecting device for tracking aircraft pilot consciousness as claimed in claim 1, wherein the flight control system pressure signal sensing device comprises: a data transmitting and receiving module; and a power supply module.

8. 2. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 1, wherein the cyclic stick pressure detection sensor is located at a lower center in front of the cockpit adjacent to the pilot, the collective stick pressure detection sensor is located at a lower left end of the cockpit adjacent to the pilot, and the pedal pressure detection sensor is located at a lower front end of the cockpit adjacent to the pilot.

9. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 7, wherein the data transmitting and receiving module transmits the information collected by the pressure sensing module to the pilot consciousness and attitude tracking and learning device, and the integrated data processing and storage device.

10. 8. The vital signs and attitude information collecting device for tracking pilot consciousness of an aircraft as claimed in claim 7, wherein the power supply module applies power to the pressure sensing module and the data transmitting and receiving module.

11. The biometric and attitude information collecting device for aircraft pilot consciousness tracking as claimed in claim 1, wherein the pilot consciousness and attitude tracking and learning device includes: a Bluetooth transceiver module; a data transceiver module; and a power supply module.

12. The biometric and attitude information collection device for tracking the consciousness of an aircraft pilot as described in claim 1, wherein the facial and attitude confirmation camera collects changes in the facial state including the pupils, and attitude information including the pilot's head, shoulders and arms in real time and provides it to the signal information processing and learning module.

13. The biometric and posture information collecting device for tracking the consciousness of an aircraft pilot as described in claim 12, wherein posture information related to the pilot's head, shoulders and arms is collected by a facial and posture confirmation camera tracking an imaginary vertical line connecting the pilot's facial protrusions, including the pilot's nose and chin, to the center of the torso, and an imaginary horizontal line crossing both shoulders.

14. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 13, further comprising a signal information processing and learning module that learns the movement lines collected during pre-flight mission equipment inspection and the movement lines collected during flight stick and equipment operation, and sends a pilot attitude abnormality signal from the signal information processing and learning module when the pilot's attitude deviates from the learning range and approaches an abnormal angle.

15. The biological and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 1, wherein the infrared light for checking pupil reflex is equipment for checking the pupil reflex of the pilot, and irradiates infrared band wavelengths that are less harmful to the human body toward the pupil to induce a change in pupil size.

16. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 15, wherein the change in the size of the pilot's pupils is observed by tracking the pupil reaction with a facial and attitude confirmation camera; and after checking the state of the eyeballs without a pupil reaction two or more times during pre-flight mission equipment inspection, the pre-reaction pupil diameter d is learned through a signal information processing and learning module.

17. The biological and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 16, wherein an infrared light is irradiated at least twice during a pre-flight mission equipment inspection or during a flight mission to check the pupil reaction, and then a post-reaction pupil diameter d' is learned through an arithmetic average through a signal information processing and learning module.

18. The biological and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 17, wherein an infrared light is periodically irradiated during flight to check whether a pupil reaction occurs, and if d>d', it is determined that the consciousness is maintained due to a normal pupil reaction and a signal is sent.

19. 20. The device for collecting biological and attitude information for tracking the consciousness of an aircraft pilot as claimed in claim 18, wherein when infrared light is periodically irradiated during flight, if the pupil diameter d' confirmed through the camera is identified as being within the pre-reaction pupil diameter d±5%, the infrared light is additionally irradiated three or more times at intervals of 1±0.5 seconds to observe the pupil reaction, and the pupil diameter d" obtained by arithmetically averaging the collected pupil diameters is learned through a signal information processing and learning module, and if d" is within d±5%, the device determines that the pilot is in a state of unconsciousness due to no pupil reaction and sends a signal.

20. The biometric and attitude information collection device for tracking the consciousness of an aircraft pilot as described in claim 11, wherein the Bluetooth transceiver module receives physiological body information transmitted from a body-worn device worn by the pilot and transmits it to the signal information processing and learning module.

21. The signal information processing and learning module has a function of monitoring transmitted signals and information, a function of learning and storing the physiological body information and attitude information of the pilot based on the collected data, and a function of identifying singular points related to loss of consciousness confirmed from the body information and attitude information.

22. 22. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 21, in which, in relation to the pilot's physiological body information, the signal information processing and learning module: receives information related to blood pressure, heart rate, electrocardiogram, body temperature and oxygen saturation from the body-worn device at intervals of 5±2 minutes and learns the average value in a stable state of consciousness by taking the arithmetic average for each item; receives pupil diameter d in a stable state of consciousness from the pupil response and attitude information sensing module and learns it; and if signal information outside the learned range is input, it is determined to be an abnormal input and processed.

23. 22. The biometric and attitude information collection device for tracking aircraft pilot consciousness as described in claim 21, wherein the signal information processing and learning module: tracks the movement lines corresponding to the positions of the pilot's head, shoulders, arms and hands during pre-flight mission equipment inspection or during in-flight equipment operation through the facial and attitude confirmation cameras, and learns the normal operation movement lines; tracks the operation direction and pressure applied to the stick and pedals during pre-flight mission equipment inspection or during in-flight equipment operation through the flight control system pressure signal sensing device, and learns the operation direction and pressure range applied at a normal consciousness level; and if signal information outside the learned range is input, determines it to be an abnormal input and processes it.

24. 22. The biological and attitude information collecting device for tracking the consciousness of an aircraft pilot as claimed in claim 21, in relation to identifying singular points related to loss of consciousness based on the biological information and attitude information; the signal information processing and learning module: determines loss of consciousness when the pupil diameter d" is d±5% or the pupil disappears outside the range of the upper and lower eyelids; determines loss of consciousness when the heart rate measured by the body worn device becomes zero; determines loss of consciousness when the pilot's attitude is tracked to take an abnormal angle and the pressure measured by the flight control system pressure signal sensing device becomes zero; determines loss of consciousness when the pilot's attitude is tracked to take an abnormal angle and the flight control system pressure signal sensing device applies fixed pressure in a direction corresponding to the abnormal operating range; and additional criteria related to loss of consciousness are externally input through the data transmitting and receiving module.

25. The biometric and attitude information collecting device for tracking pilot consciousness as claimed in claim 11, wherein the data transmitting and receiving module transmits the information collected and analyzed by the signal information processing and learning module to the integrated data processing and storage device.

26. The biometric and attitude information collecting device for tracking the consciousness of an aircraft pilot as described in claim 11, wherein the power supply module applies power to the pupil response and attitude information sensing module, the Bluetooth transceiver module, the signal information processing and learning module, and the data transceiver module.

27. The biometric and attitude information collection device for aircraft pilot consciousness tracking as described in claim 1, wherein the integrated data processing and storage device includes: an attitude information processing and storage module; a biometric information processing and storage module; a voice and flight record storage device; a data transmission and reception module; and a power supply module.

28. The biometric and attitude information collection device for tracking the consciousness of an aircraft pilot as described in claim 27, wherein the attitude information processing and storage module receives and stores data related to the attitude information of the aircraft and the pilot, and generates a warning signal when a singularity occurs.

29. 28. The biometric and attitude information collection device for tracking aircraft pilot consciousness as described in claim 27, wherein the biometric information processing and storage module receives and stores data related to the pilot's physiological body information and generates a warning signal upon identifying a singularity related to pilot loss of consciousness.

30. The vital signs and attitude information collecting device for tracking the consciousness of an aircraft pilot according to claim 27, wherein the voice and flight record storage device is an equipment that acts as a black box for storing pilot communication data and flight records.

31. The data transmission and reception module: receives information from the pilot consciousness and attitude tracking and learning device, and the flight control system pressure signal sensing device, and transmits it to the attitude information processing and storage module, the biometric information processing and storage module, and the voice and flight record storage device; and transmits the signal generated in the attitude information processing and storage module or the biometric information processing and storage module when a singularity occurs to a device that includes a multi-function display device and allows the pilot to perceive using audiovisual information. The biometric and attitude information collection device for tracking aircraft pilot consciousness as described in claim 27.

32. 28. The biometric and attitude information collecting device for tracking aircraft pilot consciousness as described in claim 27, wherein the power supply module applies power to the attitude information processing and storage module, the biometric information processing and storage module, the voice and flight record storage device, and the data transmission and reception module.

Images