A monitoring system for the disabled state of a unit based on contact sensors

By installing contact sensors on the pilot, monitoring its various physiological signals, and using AI to identify models to analyze and identify disability status in real time, the problem of pilot disability status monitoring is solved, the flight safety is improved, and key technical support is provided for a single pilot's driving mode.

CN117508616BActive Publication Date: 2025-06-17LUOYANG INST OF ELECTRO OPTICAL EQUIP OF AVIC +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311493231.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-09
Publication Date
2025-06-17
Estimated Expiration
2043-11-09

AI Technical Summary

Technical Problem

How to reasonably and efficiently monitor the pilot's status, including fatigue, behavior, psychology, emotions, workload and disability, to reduce the risk of flight accidents, especially in a single pilot's driving mode.

Method used

The unit's disability status monitoring system based on contact sensors is adopted, including wearable wrist sensors, wearable chest strap sensors, seat pressure sensors, data synchronization modules, intelligent computing platform, voice acquisition and recognition modules and status alarm modules. Through the comprehensive analysis of a variety of physiological signals and AI recognition models, the pilot's disability status is monitored and identified in real time.

Benefits of technology

It realizes natural and uninfluential disability status monitoring, improves crew flight safety, reduces the risk of flight accidents, and provides key support for commercial aircraft intelligent assisted flight technology.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117508616B_ABST
    Figure CN117508616B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of monitoring the health status of flight crews in the cockpit, and particularly relates to a monitoring system for the incapacitation status of flight crews based on contact sensors, including a wearable wrist sensor, a wearable chest strap sensor, a seat pressure sensor, a data synchronization module, an intelligent computing platform, a voice acquisition and recognition module, and a status warning module. Through contact multi-sensor fusion technology and algorithms, the present invention can accurately monitor and analyze the ECG electrocardiogram signal, PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, SKT skin temperature signal, and HR heart rate signal of pilots, realizing real-time monitoring of the incapacitation status of flight crews and improving the safety of the single-pilot driving mode.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of monitoring the health status of flight crew in the cockpit, and particularly relates to a monitoring system for crew incapacitation status based on contact sensors. Background Art

[0002] The operation of large commercial civil airliners requires at least two pilots. With the continuous development of the aviation industry, the proportion of aircraft maintenance and fuel costs in the operating costs of airlines is gradually decreasing, while the proportion of crew costs is gradually increasing, which has an increasing impact on the economy of airlines. And according to the data of the International Air Transport Association (IATA), air transportation volume is expected to double, and the industry will face a shortage of pilots. Therefore, in order to control crew costs and solve the problem of pilot shortage, the aviation industry has carried out a large number of technical research and future plans on intelligent flight technology for commercial aircraft, including Single Pilot Operation (SPO), Autonomous Taxi, Takeoff and Landing (ATTOL), Airbus' DragonFly project, NASA's "Aviation Strategic Implementation Plan", and EASA's "Artificial Intelligence Roadmap", etc., to build a vision for intelligent flight of future commercial aircraft.

[0003] However, relying on advanced airborne automation equipment or air-ground collaborative operation modes to achieve, for example, Single Pilot Operation, the workload and driving pressure of a single pilot should not be higher than that of the current two-pilot crew mode. Therefore, how to reasonably and efficiently monitor the status of pilots, including fatigue, behavior, psychology, emotion, workload, and incapacitation, etc., not only helps to improve flight safety for pilots, but also helps to promote commercial aircraft towards the era of intelligent flight.

[0004] Pilot incapacitation refers to the reduction of physical fitness to a level that may endanger flight safety. According to its severity, it can be divided into two types: total incapacitation and partial incapacitation. When a pilot is incapacitated, their flight control ability will decline to varying degrees, either obviously or secretly, and in severe cases, it will lead to flight accidents. According to the "1% rule", the fatal accident rate caused by the incapacitation of a single pilot should not exceed 1 / 10 -9 hours. For an aircraft with two pilots, the risk of pilot incapacitation does not exceed 1 / 10 -6 hours. Therefore, for the single-pilot operation mode, a reasonable incapacitation status monitoring method needs to be adopted to reduce the risk rate caused by pilot incapacitation to the risk level of two-pilot operation, which is a key technology urgently needed to be solved for the intelligent flight of future aircraft. Summary of the Invention

[0005] In view of this, the present invention provides a crew incapacitation state monitoring system based on contact sensors, which can realize natural and imperceptible incapacitation state monitoring, comprehensively analyze and identify various physiological signals of pilots, analyze the crew incapacitation state, improve the flight safety of the crew, and lay a foundation for the key technologies of intelligent assisted flight of commercial aircraft.

[0006] In order to achieve the above technical objectives, the specific technical solutions adopted by the present invention are as follows:

[0007] A crew incapacitation state monitoring system based on contact sensors, including a wearable wrist sensor, a wearable chest strap sensor, a seat pressure sensor, a data synchronization module, an intelligent computing platform, a voice acquisition and recognition module, and a status warning module, wherein:

[0008] The wearable wrist sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time;

[0009] The wearable chest strap sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time;

[0010] The seat pressure sensor is used to collect the pilot's body pressure distribution signal and wirelessly transmit the collected body pressure distribution signal data to the data synchronization module in real time;

[0011] The data synchronization module is used to receive the physiological parameter data of the pilot sent by the wearable wrist sensor and the wearable chest strap sensor and the body pressure distribution signal data sent by the seat pressure sensor, and synchronize the data sent by each sensor to achieve synchronous acquisition of multi-sensor data;

[0012] The intelligent computing platform is used to receive the physiological parameter data of the pilot and the body pressure distribution data synchronously processed by the data synchronization module in real time, deploy a crew incapacitation state monitoring model, and output the crew incapacitation state to the status warning module; the intelligent computing platform is cross-linked with the voice acquisition and recognition module to receive the pilot's voice information, which is used for the pilot to actively report his own status and to reconfirm the pilot's status when the system detects crew incapacitation;

[0013] The voice acquisition and recognition module is used to collect the active report information, identify the pilot's active report information and the confirmation of the secondary voice information, and send the recognized information to the intelligent computing platform;

[0014] The status warning module receives the crew incapacitation state information output by the intelligent computing platform and generates a warning message for the pilot's incapacitation state.

[0015] Further, the physiological parameters collected by the wearable wrist sensor include PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, SKT skin temperature signal, and HR heart rate signal.

[0016] The physiological parameters collected by the wearable chest strap sensor include ECG electrocardiogram signal, RESP respiration signal, RPM respiratory rate signal, and HR heart rate signal.

[0017] Further, the crew incapacitation state includes total incapacitation, partial incapacitation, and normal, where incapacitation means that the pilot's physical fitness is reduced to a level that may endanger flight safety.

[0018] Further, the wearable wrist sensor uses the reflective optoelectronic detection method to measure the pilot's PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, and HR heart rate signal.

[0019] The wearable wrist sensor uses the Bluetooth wireless transmission method to send data to the data synchronization and fusion module.

[0020] The wearable chest strap sensor uses patch-type measurement electrodes to measure the pilot's ECG electrocardiogram signal, RESP respiration signal, RPM respiratory rate signal, and HR heart rate signal.

[0021] Further, the wearable chest strap sensor uses the Bluetooth wireless transmission method to send data to the data synchronization and fusion module.

[0022] Further, the intelligent computing platform is used to perform multi-modal data preprocessing, feature extraction, AI incapacitation state recognition model deployment, and incapacitation state output.

[0023] Further, the signal of the incapacitation state output is transmitted based on the UDP network protocol.

[0024] Further, the voice acquisition and recognition module consists of a voice microphone array and artificial intelligence voice recognition software.

[0025] Further, the status warning module consists of a speaker and a display, and the status warning module reminds the pilot in an auditory and visual manner.

[0026] Further, the crew incapacitation state monitoring system of the contact sensor determines whether the pilot is in an incapacitation state based on the following steps:

[0027] S1: After collecting physiological parameter data, perform feature index extraction and data normalization, specifically including: performing heart rate variability analysis based on electrocardiogram signals, blood volume pulse signals, and heart rate signals, including three analysis methods: time-domain analysis, frequency-domain analysis, and non-linear analysis, and extracting their characteristic index parameters; performing time-domain analysis and frequency-domain analysis based on respiratory signals, and extracting their characteristic index parameters; performing data normalization on the extracted physiological parameter eigenvalues;

[0028] S2: According to the extraction results of physiological parameter eigenvalues, as well as blood oxygen saturation parameters, skin temperature parameters, and simultaneously collected seat pressure distribution data, perform data annotation to form a data set, and divide the data set into a training set and a test set;

[0029] S3: Construct a deep learning network for identifying the disabled state of the AI crew, and perform training and testing based on the training set and the test set to obtain a trained model for identifying the disabled state of the AI crew;

[0030] S4: If the AI crew disabled state recognition model determines that the crew is disabled, send visual and auditory alarm information in the status alarm module, and pop up a pilot secondary status confirmation page on the display unit. Receive the pilot's voice information and touch feedback information in real time according to the pilot's ability, and reconfirm the pilot's status. If the pilot's status is normal, cancel the visual and auditory alarm information. If not, confirm that the pilot is disabled and send the pilot disability information to the crew and ground operation support.

[0031] Advantages of the present invention:

[0032] The present invention realizes the collection of multi-modal physiological data of pilots through wearable wrist sensors, wearable chest strap sensors, and seat pressure sensors, and the wireless data transmission technology avoids the interference of a large number of cables to the operation, ensuring that the pilot is not interfered and realizing natural and non-intrusive monitoring of the disabled state. Secondly, by deploying the AI crew disabled state monitoring model, the present invention can comprehensively analyze and identify various physiological signals of the pilot, analyze the disabled state of the crew, improve the flight safety of the crew, and lay a foundation for the key technologies of intelligent assisted flight of commercial aircraft. Description of the Drawings

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a schematic diagram of the system composition of the present invention;

[0035] Figure 2Sensor arrangement and wearing position of the present invention;

[0036] Figure 3 Monitoring and identification process of the unit disablement state of the present invention. Specific implementation manners

[0037] The following describes the embodiments of the present disclosure in detail with reference to the accompanying drawings.

[0038] The following illustrates the implementation manners of the present disclosure through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present disclosure.

[0039] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present disclosure, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement the device and / or practice the method. In addition, this device can be implemented and this method can be practiced using other structures and / or functions in addition to one or more of the aspects described herein.

[0040] It should also be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present disclosure schematically. The drawings only show the components related to the present disclosure and are not drawn according to the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in its actual implementation can be an arbitrary change, and the component layout type may also be more complex.

[0041] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0042] In an embodiment of the present invention, a monitoring system for the incapacitated state of an aircrew based on contact sensors is proposed, which includes a wearable wrist sensor, a wearable chest strap sensor, a seat pressure sensor, a data synchronization module, an intelligent computing platform, a voice acquisition and recognition module, and a status warning module, where:

[0043] The wearable wrist sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time;

[0044] The wearable chest strap sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time;

[0045] The seat pressure sensor is used to collect the human body pressure distribution signal of the pilot and wirelessly transmit the collected human body pressure distribution signal data to the data synchronization module in real time;

[0046] The data synchronization module is used to receive the physiological parameter data of the pilot sent by the wearable wrist sensor and the wearable chest strap sensor, and the human body pressure distribution signal data sent by the seat pressure sensor, and synchronize the data sent by each sensor to achieve synchronous acquisition of multi-sensor data;

[0047] The intelligent computing platform is used to receive the physiological parameter data of the pilot and the human body pressure distribution data after synchronous processing by the data synchronization module in real time, deploy a monitoring model for the incapacitated state of the aircrew, and output the incapacitated state of the aircrew to the status warning module; the intelligent computing platform is cross-linked with the voice acquisition and recognition module to receive the voice information of the pilot, which is used for the pilot to actively report their own status and to reconfirm the pilot's status when the system detects the incapacitation of the aircrew;

[0048] The voice acquisition and recognition module is used to collect the active report information, identify the active report information of the pilot and the confirmation of the secondary voice information, and send the recognized information to the intelligent computing platform;

[0049] The status warning module receives the incapacitated state information of the aircrew output by the intelligent computing platform and generates a warning message for the incapacitated state of the pilot.

[0050] In an embodiment, the physiological parameters collected by the wearable wrist sensor include PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, SKT skin temperature signal, and HR heart rate signal,

[0051] The physiological parameters collected by the wearable chest strap sensor include ECG electrocardiogram signal, RESP respiration signal, RPM respiratory rate signal, and HR heart rate signal

[0052] In one embodiment, the crew incapacitation states include total incapacitation, partial incapacitation, and normal, where incapacitation means that the pilot's physical fitness has been reduced to a level that may endanger flight safety.

[0053] The wearable wrist sensor uses the reflective optoelectronic detection method to measure the pilot's PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, and HR heart rate signal.

[0054] In one embodiment, the wearable wrist sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module.

[0055] In one embodiment, the wearable chest strap sensor uses patch-type measurement electrodes to measure the pilot's ECG electrocardiogram signal, RESP respiration signal, RPM respiratory rate signal, and HR heart rate signal.

[0056] In one embodiment, the wearable chest strap sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module.

[0057] In one embodiment, the intelligent computing platform is used to perform multimodal data preprocessing, feature extraction, AI incapacitation state recognition model deployment, and incapacitation state output.

[0058] In one embodiment, the signal of the incapacitation state output is transmitted based on the UDP network protocol.

[0059] In one embodiment, the voice acquisition and recognition module consists of a voice microphone array and artificial intelligence voice recognition software.

[0060] In one embodiment, the status warning module consists of a speaker and a display, and the status warning module reminds the pilot based on auditory and visual means.

[0061] In one embodiment, the crew incapacitation state monitoring system of the contact sensor determines whether the pilot is in an incapacitated state based on the following method:

[0062] S1: After collecting the physiological parameter data, perform feature index extraction and data normalization, specifically including: performing heart rate variability analysis based on the electrocardiogram signal, blood volume pulse signal, and heart rate signal, including three analysis methods: time-domain analysis, frequency-domain analysis, and non-linear analysis, and extracting their characteristic index parameters; performing time-domain analysis and frequency-domain analysis on the respiration signal and extracting its characteristic index parameters; performing data normalization on the extracted physiological parameter eigenvalues;

[0063] S2: Extract the results of physiological parameter eigenvalue extraction, along with blood oxygen saturation parameters, skin temperature parameters, and simultaneously collected seat pressure distribution data, and perform data annotation to form a dataset. Divide the dataset into a training set and a test set.

[0064] S3: Construct a deep learning network for identifying the disabled state of the AI crew, and train and test it based on the training set and the test set to obtain a trained model for identifying the disabled state of the AI crew.

[0065] S4: If the AI crew disabled state recognition model determines that the crew is disabled, send visual and auditory alarm messages in the status alarm module, and pop up a secondary status confirmation page for the pilot on the display unit. Receive the pilot's voice information and touch feedback information in real time according to the pilot's ability, and confirm the pilot's status for the second time. If the pilot's status is normal, eliminate the visual and auditory alarm messages. If not, confirm that the pilot is disabled and send the pilot disability information to the crew and ground operation support.

[0066] See the appendix Figure 1 As shown, the embodiment of the present invention discloses a monitoring system for the disabled state of the crew based on contact sensors. The system includes a wearable wrist sensor, a wearable chest strap sensor, a seat pressure sensor, a data synchronization module, an intelligent computing platform, a voice collection and recognition module, and a status alarm module.

[0067] Among them:

[0068] See the appendix Figure 2 As shown, the wearable wrist sensor is used to collect the pilot's physiological parameters including PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, SKT skin temperature signal, and HR heart rate signal, and transmit the collected physiological parameter data to the data synchronization module in real time wirelessly.

[0069] See the appendix Figure 2 As shown, the wearable chest strap sensor is used to collect the pilot's physiological parameters including ECG electrocardiogram signal, RESP respiration signal, and HR heart rate signal, and transmit the collected physiological parameter data to the data synchronization module in real time wirelessly.

[0070] See the appendix Figure 2 As shown, the seat pressure sensor is used to collect the pilot's body pressure distribution signal, and transmit the collected body pressure distribution signal data to the data synchronization module in real time wirelessly.

[0071] The data synchronization module is used to receive the pilot's physiological parameter data sent by the wearable wrist sensor and the wearable chest strap sensor, and the body pressure distribution signal data sent by the seat pressure sensor, and perform data synchronization and fusion processing on the data sent by each sensor to achieve synchronous acquisition and multi-source fusion of multi-sensor data.

[0072] An intelligent computing platform is used to receive in real time the physiological parameter data of the pilot and the human body pressure distribution data processed by the data synchronization module, and deploy a crew incapacitation state monitoring model to output the crew incapacitation state to the status warning module; the intelligent computing platform is also cross-linked with the voice collection and recognition module to receive the pilot's voice information for secondary confirmation of the pilot's status.

[0073] A voice collection and recognition module is used to collect and recognize the pilot's active report information and secondary confirmation voice information, and send the recognized information to the intelligent computing platform.

[0074] A status warning module receives the crew incapacitation state information output by the intelligent computing platform and generates a warning message for the pilot's incapacitation state.

[0075] In some embodiments, the crew incapacitation state includes total incapacitation, partial incapacitation, and normal. Among them, incapacitation means that the pilot's physical fitness is reduced to a level that may endanger flight safety.

[0076] In some embodiments, the wearable wrist sensor uses the reflective optoelectronic detection method to measure the pilot's PPG blood volume pulse signal, SKT skin temperature signal, SPO2 blood oxygen saturation signal, and HR heart rate signal.

[0077] In some embodiments, the wearable wrist sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module.

[0078] In some embodiments, the wearable chest strap sensor uses patch-type measurement electrodes to measure the pilot's ECG electrocardiogram signal, RESP respiratory signal, and HR heart rate signal.

[0079] In some embodiments, the wearable chest strap sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module.

[0080] In some embodiments, the seat pressure sensor is a flexible pressure sensor that can be arranged on the seat cushion and backrest to collect the pilot's human body pressure distribution signal.

[0081] In some embodiments, the data synchronization module can perform multi-source data synchronization according to the sampling data timestamp and send the synchronized data to the intelligent computing platform.

[0082] In some embodiments, the intelligent computing platform can perform multiple functions such as multi-mode data preprocessing, feature extraction, AI incapacitation state recognition model deployment, and incapacitation state output.

[0083] In some embodiments, the function of deploying the AI incapacitation state recognition model should meet the requirements of real-time operation.

[0084] In some embodiments, the incapacitation state output function uses the UDP network protocol to send the crew incapacitation state data outward in a broadcast form.

[0085] In some embodiments, the voice acquisition and recognition module consists of a voice microphone array and artificial intelligence voice recognition software.

[0086] In some embodiments, the status warning module consists of a touch display unit that can emit sound, and can remind the pilot of the incapacitation state in both auditory and visual ways.

[0087] In some embodiments, the touch display unit can execute the pilot interaction function. When the pilot has partial interaction ability, the pilot can click the corresponding button on the display unit to confirm their own state again.

[0088] In some embodiments, see the appendix Figure 3 As shown, the specific process for the system to judge whether the pilot is in an incapacitated state is as follows:

[0089] S1: First, collect the physiological parameter data and then perform feature index extraction and data normalization. Specifically, it includes: performing heart rate variability (HRV) analysis based on electrocardiogram signals, blood volume pulse signals, and heart rate signals, including three analysis methods: time-domain analysis, frequency-domain analysis, and non-linear analysis, and extracting their characteristic index parameters; performing time-domain analysis and frequency-domain analysis based on respiratory signals (RESP) and extracting their characteristic index parameters; performing data normalization on the extracted physiological parameter eigenvalues.

[0090] S2: According to the extraction results of physiological parameter eigenvalues, as well as blood oxygen saturation parameters, skin temperature parameters, and simultaneously collected seat pressure distribution data, perform data annotation on them to form a data set, and divide it into a training set and a test set.

[0091] S3: Construct a three-class (fully incapacitated, partially incapacitated, normal) AI crew incapacitation state recognition deep learning network, and perform training and testing based on the training set and the test set to obtain a trained AI crew incapacitation state recognition model.

[0092] S4: If the AI crew incapacitation state recognition model determines that the crew is incapacitated, visual and auditory warning information should be issued in the status warning module, and a pilot secondary status confirmation page ("Confirm Incapacitation" button, "Status Normal" button) should pop up on the display unit. According to the pilot's ability, receive the pilot's voice information and touch feedback information in real time to confirm the pilot's state again. If the pilot's state is normal, eliminate the visual and auditory warning information. Otherwise, confirm the pilot's incapacitation and send the pilot incapacitation information to the crew and ground operation support.

[0093] As described above, it is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present disclosure should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A monitoring system for the disabled state of a unit based on a contact sensor, characterized in that, It includes a wearable wrist sensor, a wearable chest strap sensor, a seat pressure sensor, a data synchronization module, an intelligent computing platform, a voice acquisition and recognition module, and a status warning module, where: The wearable wrist sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time; The wearable chest strap sensor is used to collect the physiological parameters of the pilot and wirelessly transmit the collected physiological parameter data to the data synchronization module in real time; The seat pressure sensor is used to collect the pilot's body pressure distribution signal and wirelessly transmit the collected body pressure distribution signal data to the data synchronization module in real time; The data synchronization module is used to receive the pilot's physiological parameter data sent by the wearable wrist sensor and the wearable chest strap sensor and the body pressure distribution signal data sent by the seat pressure sensor, and synchronize the data sent by each sensor to achieve synchronous acquisition of multi-sensor data; The intelligent computing platform is used to receive the pilot's physiological parameter data and body pressure distribution data after synchronous processing by the data synchronization module in real time, deploy a crew incapacitation state monitoring model, and output the crew incapacitation state to the status warning module; the intelligent computing platform is cross-linked with the voice acquisition and recognition module to receive the pilot's voice information, which is used for the pilot to actively report his own status and to reconfirm the pilot's status when the system detects crew incapacitation; The voice acquisition and recognition module is used to collect the active report information, recognize the pilot's active report information and the confirmation of the secondary voice information, and send the recognized information to the intelligent computing platform; The status warning module receives the crew incapacitation state information output by the intelligent computing platform and generates a warning message for the pilot's incapacitation state; Among them: the physiological parameters collected by the wearable wrist sensor include PPG blood volume pulse signal, SPO2 blood oxygen saturation signal, SKT skin temperature signal, and HR heart rate signal; The voice acquisition and recognition module consists of a voice microphone array and artificial intelligence voice recognition software; The physiological parameters collected by the wearable chest strap sensor include ECG electrocardiogram signal, RESP respiration signal, RPM respiration rate signal, and HR heart rate signal; The crew incapacitation state monitoring system of the contact sensor determines whether the pilot is in an incapacitated state based on the following steps of the system: S1: After collecting the physiological parameter data, perform feature index extraction and data normalization, specifically including: performing heart rate variability analysis according to the electrocardiogram signal, blood volume pulse signal, and heart rate signal, including three analysis methods: time domain analysis, frequency domain analysis, and non-linear analysis, and extracting feature index parameters; performing time domain analysis and frequency domain analysis on the respiration signal and extracting feature index parameters; normalizing the extracted physiological parameter feature values; S2: According to the extraction results of the physiological parameter feature values, as well as the blood oxygen saturation parameter, skin temperature parameter, and the simultaneously collected seat pressure distribution data, perform data annotation to form a data set, and divide the data set into a training set and a test set; S3: Construct a deep learning network for identifying the disabled state of the AI flight crew, and train and test it based on the training set and the test set to obtain a trained AI flight crew disabled state recognition model; S4: If the AI flight crew disabled state recognition model determines that the flight crew is disabled, send visual and auditory alarm messages in the status alarm module, and pop up a pilot secondary status confirmation page on the display unit. Receive the pilot's voice information and touch feedback information in real time according to the pilot's ability, and confirm the pilot's status for the second time. If the pilot's status is normal, cancel the visual and auditory alarm messages. Otherwise, confirm that the pilot is disabled and send the pilot disability information to the flight crew and ground operation support.

2. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 1, characterized in that, The disabled state of the flight crew includes total disability, partial disability and normal, where disability means that the pilot's physical fitness has decreased to a level that may endanger flight safety.

3. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 1, characterized in that, The wearable wrist sensor uses the reflective photoelectric detection method to measure the pilot's PPG blood volume pulse signal, SPO2 blood oxygen saturation signal and HR heart rate signal; The wearable wrist sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module; The wearable chest strap sensor uses patch-type measurement electrodes to measure the pilot's ECG electrocardiogram signal, RESP respiration signal, RPM respiration rate signal and HR heart rate signal.

4. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 2, characterized in that, The wearable chest strap sensor uses the Bluetooth wireless transmission method to send data to the data synchronization module and the fusion module.

5. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 4, characterized in that, The intelligent computing platform is used to perform multi-modal data preprocessing, feature extraction, AI disability state recognition model deployment and disability state output.

6. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 5, characterized in that, The signal of the disability state output is transmitted based on the UDP network protocol.

7. The monitoring system for the disabled state of a unit based on a contact sensor according to claim 6, characterized in that, The status alarm module consists of a speaker and a display, and the status alarm module reminds the pilot in an auditory and visual way.

Citation Information

Patent Citations

  • Driver health fatigue and prevention monitoring system

    CN111354162A

  • Electronic device for monitoring a neurophysiological state of an operator in an aircraft control station, associated monitoring method and associated computer program

    US20230293114A1