A method for monitoring the physiological state of a pilot during flight

Abstract

The present application relates to a kind of physiological state monitoring method of pilot in flight process, belong to aviation medical field, solve the inaccurate problem of physiological state monitoring of pilot in flight process in existing method. Including in the process of pilot flight, using high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment real-time monitoring pilot's blood oxygen, electrocardiogram and respiratory signal, and using airborne camera to shoot face video;Real-time monitoring blood oxygen, electrocardiogram and respiratory signal, and face video, after data encryption, transmission to ground command center;Ground command center is based on real-time monitoring blood oxygen, electrocardiogram, respiratory signal and face video, input trained flight fatigue prediction model, obtain the comprehensive physiological fatigue level of pilot, and based on comprehensive physiological fatigue level generation decision instruction.The high-precision real-time monitoring of pilot blood oxygen, electrocardiogram, respiration and face video under dynamic flight environment is realized, accurately assesses and warns pilot comprehensive fatigue state.

Description

Technical Field

[0001] This invention relates to the field of aviation medicine technology, and in particular to a method for monitoring the physiological state of pilots during flight. Background Technology

[0002] In modern aviation, monitoring pilots' physiological state is crucial for ensuring flight safety. During long flights, especially under high-intensity missions or complex weather conditions, pilots are prone to physiological problems such as fatigue, hypoxia, and arrhythmia, which can seriously affect the accuracy and safety of flight operations. Therefore, real-time and accurate monitoring of pilots' physiological state, timely detection of potential health risks, and implementation of appropriate intervention measures are important aspects of aviation safety management.

[0003] Monitoring physical and mental states during flight is becoming increasingly important. However, current blood oxygen, electrocardiogram, and respiration monitoring devices typically perform well in static situations but show significant shortcomings in dynamic scenarios. Traditional monitoring devices, while achieving real-time monitoring, have limited measurement accuracy and rely solely on heart rate as a judgment indicator, resulting in assessment lag and difficulty in reflecting individual differences and providing early warning functions.

[0004] Existing flight physiological monitoring equipment often fails to provide high-precision monitoring data in dynamic environments (such as acceleration and vibration during flight). For example, traditional blood oxygen monitoring equipment may experience signal drift or false alarms in high-acceleration environments. A pilot's physiological state requires a comprehensive assessment of multiple physiological signals (such as blood oxygen, ECG, and respiration) and behavioral characteristics (such as facial expressions and head posture). However, current technologies employ relatively simple methods for fusing these multimodal data, failing to fully utilize the advantages of each modality, resulting in insufficient accuracy and reliability of the monitoring results. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide a method for monitoring the physiological state of pilots during flight, in order to solve the technical problem of inaccurate monitoring of the physiological state of pilots during flight in existing methods.

[0006] The objective of this invention is mainly achieved through the following technical solutions:

[0007] This invention provides a method for monitoring the physiological state of a pilot during flight, comprising the following steps:

[0008] During flight, high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment are used to monitor the pilot's blood oxygen, electrocardiogram and respiratory signals in real time, and airborne cameras are used to capture facial videos;

[0009] The real-time monitored blood oxygen, ECG and respiratory signals, as well as facial video, are transmitted to the ground command center after data encryption.

[0010] The ground command center, based on real-time monitoring of the pilot's blood oxygen, electrocardiogram, respiratory signals, and facial video, inputs a trained flight fatigue prediction model to obtain the pilot's comprehensive physiological fatigue level, and generates decision instructions based on the comprehensive physiological fatigue level.

[0011] Furthermore, the flight fatigue prediction model is trained through the following process:

[0012] After obtaining and preprocessing the blood oxygen, electrocardiogram, respiratory signals and facial videos of multiple pilots within a certain historical period, the data are combined with sample labels to form a flight fatigue sample set; where the sample labels are the pilots' fatigue levels.

[0013] Construct a flight fatigue prediction model;

[0014] The flight fatigue prediction model is trained using the flight fatigue sample set. The forward propagation uses the cross-entropy loss function, and the backpropagation uses the AdamW optimizer to update the model parameters.

[0015] The model parameters are saved as a trained flight fatigue prediction model until the joint loss function converges.

[0016] Furthermore, the high-precision blood oxygen, electrocardiogram, and respiration monitoring equipment was obtained through the following evaluation steps:

[0017] The blood oxygen, electrocardiogram (ECG), and respiratory signals of a multimodal dummy are simulated using simulation equipment, and the corresponding blood oxygen, ECG, and respiratory monitoring signals of the multimodal dummy are monitored using blood oxygen, ECG, and respiratory monitoring equipment; wherein, the simulation equipment includes a blood oxygen simulator, an ECG simulator, and a respiratory motion module;

[0018] The resultant acceleration of the six-axis motion platform is obtained; wherein the multimodal dummy is disposed on the six-axis motion platform;

[0019] Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the dynamic accuracy evaluation result is obtained.

[0020] Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, as well as the combined acceleration of motion, the performance of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the performance evaluation results are obtained.

[0021] If both the dynamic accuracy assessment results and the performance assessment results meet the predetermined requirements, a high-precision blood oxygen, electrocardiogram, and respiration monitoring device will be obtained.

[0022] Furthermore, the network structure of the flight fatigue prediction model includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer;

[0023] The feature extraction layer includes parallel first, second, and third deep residual networks and EfficientNet networks; the first, second, and third deep residual networks and EfficientNet networks are followed by corresponding first, second, third, and fourth global pooling layers and first, second, third, and fourth fully connected layers in sequence;

[0024] The first, second, and third deep residual networks are used to extract blood oxygen, electrocardiogram, and respiratory signal features, respectively.

[0025] The EfficientNet network is used to extract facial video features;

[0026] The first, second, third, and fourth global pooling layers are used to perform global average pooling on blood oxygen, ECG, respiratory signal features, and facial video features, respectively, converting the feature maps into fixed-length feature vectors, which are then input to the corresponding first, second, third, and fourth fully connected layers.

[0027] The first, second, third, and fourth fully connected layers are used to concatenate the fixed-length feature vectors output by the first, second, third, and fourth global pooling layers into corresponding long vectors for blood oxygen, ECG, respiration, and facial video.

[0028] The feature fusion layer is a Transformer fusion module, used to fuse long vectors of blood oxygen, ECG, respiration and facial video to generate a comprehensive feature vector;

[0029] The output layer includes a parallel Softmax layer and a Sogmoid layer, which are used to output fatigue level and fatigue value based on the comprehensive feature vector, respectively.

[0030] Furthermore, evaluating the dynamic accuracy of the blood oxygen, electrocardiogram, and respiratory monitoring devices includes: calculating the simulated blood oxygen, electrocardiogram, and respiratory signals and the corresponding Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index of the blood oxygen, electrocardiogram, and respiratory monitoring signals, and then normalizing them;

[0031] The dynamic accuracy assessment results of blood oxygen, electrocardiogram and respiration are obtained by weighted summation based on the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence and complex wavelet structural similarity index.

[0032] Furthermore, the resultant acceleration of the six-axis motion platform is obtained, including:

[0033] Obtain the maximum values ​​of the lateral, longitudinal, and vertical displacement distances of the six-axis motion platform at the current moment, and obtain the maximum distance of the six-axis motion platform from the equilibrium position;

[0034] The displacement of the six-axis motion platform is calculated based on the maximum distance of the six-axis motion platform from its equilibrium position.

[0035] The velocity of the six-axis motion platform is obtained by differentiating its displacement with respect to time.

[0036] The resultant acceleration of the six-axis motion platform is obtained by differentiating its velocity with respect to time.

[0037] Further, obtain performance evaluation results for blood oxygenation, electrocardiogram, and respiration, including:

[0038] The ratio of the absolute error of the simulated signals of blood oxygen, electrocardiogram and respiration to the corresponding monitored signals at each time moment, as well as the resultant acceleration of motion at the corresponding time moment, are calculated. Based on the RANSAC algorithm, a polynomial fitting is performed to obtain the error-acceleration fitting relationship curve.

[0039] Based on the resultant acceleration, the resultant acceleration modulus is calculated;

[0040] Using the maximum and minimum values ​​of the combined acceleration modulus as the integration interval, the error-acceleration fitting curve is integrally integrated to obtain the area S enclosed by the error-acceleration fitting curve and the horizontal axis, which is used to evaluate the performance of the blood oxygen, electrocardiogram and respiratory monitoring devices to be evaluated.

[0041] Furthermore, the calculation of the ratio of the absolute error between the analog signals and the corresponding monitoring signals of blood oxygen, electrocardiogram, and respiration at each moment includes:

[0042] Calculate the errors between the simulated and monitored signals for blood oxygen, electrocardiogram, and respiration at each time point;

[0043] The average values ​​of blood oxygen, electrocardiogram and respiratory simulated signals were calculated separately and used as the average standard data for blood oxygen, electrocardiogram and respiratory.

[0044] Based on the errors between the simulated and detected signals of blood oxygen, electrocardiogram, and respiration at each moment, and the average standard data of blood oxygen, electrocardiogram, and respiration, the absolute value ratio of the errors of blood oxygen, electrocardiogram, and respiration at each moment is calculated respectively.

[0045] Furthermore, the results of the dynamic accuracy assessment are as follows:

[0046] S accuracy =a*S PCC +b*S SAM+c*S SID +d*S SAM +e*S SID +f*S CWSSIM

[0047] Among them, S accuracy S represents the score for the accuracy assessment results. PCC S SAM S SID S SAM S SID and S CWSSIM , respectively, are the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index of the analog signal and the monitoring signal; a, b, c, d, e, and f are the weights corresponding to the Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index of the analog signal and the monitoring signal, respectively.

[0048] Furthermore, the respiratory motion module includes a thoracic push plate, a push rod, a guide rail, a base, a high-precision stepper motor, and an eccentric connector;

[0049] The thoracic cavity push plate is fixedly connected to the ribcage of the dummy and is used to simulate the expansion and contraction of the human thoracic cavity;

[0050] The push rod is connected to an eccentric connector at one end and to a thoracic push plate at the other end, and is used to transmit the motion of the high-precision stepper motor to the thoracic push plate.

[0051] The guide rail extends along the axial direction of the push rod, providing stable guidance for the push rod;

[0052] The base is used to support the push rod, guide rail, and high-precision stepper motor, as well as to deploy a high-precision MCU;

[0053] The high-precision stepper motor is fixed to the base by a bracket, with a minimum step size of m° and a rotation period of one revolution. Each pulse controls the reciprocating motion of the push rod;

[0054] The eccentric connector is used to connect to the output shaft of the high-precision stepper motor. The central shaft of the high-precision stepper motor drives the eccentric connector to rotate, which in turn drives the push rod to move.

[0055] The high-precision MCU is used to control the pulse period of the high-precision stepper motor;

[0056] The high-precision MCU is based on the highest respiratory rate of the respiratory signal and... The minimum pulse period is obtained by multiplying the results, which serves as the pulse period requirement for the simulated respiratory signal.

[0057] Calculate a single clock cycle based on the operating frequency of the high-precision MCU;

[0058] If the single clock cycle is less than or equal to the pulse cycle requirement of the simulated breathing signal, then the breathing signal of the multimodal dummy simulated by the breathing motion module is used; otherwise, it is discarded.

[0059] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0060] 1. This invention uses simulation equipment and a six-axis motion platform to dynamically evaluate the accuracy of monitoring equipment, ensuring that the equipment can still provide high-precision monitoring data under highly dynamic environments (such as acceleration and vibration during flight). It employs multimodal data fusion, utilizing a deep residual network (ResNet) to extract features from blood oxygen, electrocardiogram, and respiratory signals, and using EfficientNet to extract features from facial videos. The multimodal features are then fused using a Transformer module. This fusion method fully leverages the advantages of each modality of data, overcomes the lag problem of single-modal data evaluation, and comprehensively and accurately assesses the pilot's physiological state, providing a more reliable guarantee for flight safety.

[0061] 2. This invention employs encrypted data transmission to enhance data security and real-time performance. It uses encrypted data chains (such as SM4 encryption) to encrypt and transmit real-time monitored blood oxygen, ECG, respiratory signals, and facial video, ensuring data security and integrity during transmission. This effectively prevents data leakage or malicious attacks, protecting pilot privacy and flight data security. Real-time monitoring and decision-making are performed by transmitting real-time monitored multimodal data to the ground command center and inputting it into a trained flight fatigue prediction model to quickly obtain the pilot's comprehensive physiological fatigue level, generating decision instructions based on this. This process enables real-time monitoring and rapid decision-making of the pilot's physiological state, allowing for timely detection of potential health risks and the implementation of corresponding intervention measures, thereby improving flight safety and efficiency.

[0062] 3. The flight fatigue prediction model of this invention adopts a network architecture that includes a parallel deep residual network, an EfficientNet network, and a Transformer fusion module. This architecture can not only effectively extract features from each modality of data, but also fully integrate information between different modalities through the Transformer fusion module, improving the model's ability to identify and predict pilot fatigue states. The model is trained using a flight fatigue sample set consisting of blood oxygen, ECG, respiratory signals, and facial videos from multiple pilots over a certain historical period. This allows the model to learn the fatigue characteristics of different pilots, different flight missions, and different environmental conditions, improving the accuracy and generalization ability of the fatigue assessment model. This means that the model can more accurately assess pilot fatigue states in actual flight environments, providing stronger support for flight safety.

[0063] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0064] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0065] Figure 1 This is a flowchart of a method for monitoring the physiological state of a pilot during flight, as described in an embodiment of the present invention.

[0066] Figure 2 This is a schematic diagram illustrating the dynamic accuracy evaluation and performance evaluation in an embodiment of the present invention;

[0067] Figure 3 This is a schematic diagram of the error-acceleration fitting curve results in an embodiment of the present invention;

[0068] Figure 4 This is a schematic diagram of the respiratory motion module structure in an embodiment of the present invention;

[0069] Figure 5 This is a schematic diagram of the flight fatigue prediction model structure in an embodiment of the present invention.

[0070] Figure label:

[0071] 1-Thoracic push plate; 2-Push rod; 3-Guide rail; 4-Base; 5-High-precision stepper motor; 6-Eccentric connector; 7-High-precision MCU. Detailed Implementation

[0072] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0073] To address the aforementioned issues, it is necessary to integrate a multi-degree-of-freedom motion platform and a multi-physiological index synchronous simulation dummy, establish relevant dynamic performance evaluation methods, and verify the reliability and accuracy of key vital signs such as electrocardiogram, respiration, and blood oxygen saturation measurements under simulated rapid motion and complex three-dimensional motion conditions.

[0074] One specific embodiment of the present invention discloses a method for monitoring the physiological state of a pilot during flight, such as... Figure 1 As shown, it includes the following steps:

[0075] Step S1: During the pilot's flight, use high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment to monitor the pilot's blood oxygen, electrocardiogram and respiratory signals in real time, and use an airborne camera to capture facial video;

[0076] Step S2: The real-time monitored blood oxygen, ECG and respiratory signals, as well as facial video, are transmitted to the ground command center after data encryption.

[0077] Step S3: The ground command center, based on the real-time monitoring of the pilot's blood oxygen, electrocardiogram, respiratory signals and facial video, inputs the trained flight fatigue prediction model to obtain the pilot's comprehensive physiological fatigue level, and generates decision instructions based on the comprehensive physiological fatigue level.

[0078] Step S1 includes steps S11-S12.

[0079] Step S11: Monitor the pilot's blood oxygen, electrocardiogram and respiratory signals in real time, as well as the pilot's facial video.

[0080] During flight, high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment is used to monitor the pilot's blood oxygen, electrocardiogram and respiratory signals in real time, and airborne cameras are used to capture facial videos of the pilot.

[0081] The blood oxygen, ECG, respiratory signals, and facial video are synchronized with timestamps, signal integrity is verified, and data is compressed. Then, the data is cached in an encrypted data frame format in the onboard secure storage.

[0082] Step S12: Obtain high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment.

[0083] Step S12 includes steps S121-S124.

[0084] The high-precision blood oxygen, electrocardiogram, and respiration monitoring equipment was obtained through the following evaluation steps:

[0085] The blood oxygen, electrocardiogram (ECG), and respiratory signals of a multimodal dummy are simulated using simulation equipment, and the corresponding blood oxygen, ECG, and respiratory monitoring signals of the multimodal dummy are monitored using blood oxygen, ECG, and respiratory monitoring equipment; wherein, the simulation equipment includes a blood oxygen simulator, an ECG simulator, and a respiratory motion module;

[0086] The resultant acceleration of the six-axis motion platform is obtained; wherein the multimodal dummy is disposed on the six-axis motion platform;

[0087] Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the dynamic accuracy evaluation result is obtained.

[0088] Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, as well as the combined acceleration of motion, the performance of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the performance evaluation results are obtained.

[0089] If both the dynamic accuracy assessment results and the performance assessment results meet the predetermined requirements, a high-precision blood oxygen, electrocardiogram, and respiration monitoring device will be obtained.

[0090] Step S121: Simulate the blood oxygen, electrocardiogram and respiratory signals of the multimodal dummy using simulation equipment, and monitor the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals of the multimodal dummy using the blood oxygen, electrocardiogram and respiratory monitoring equipment to be evaluated.

[0091] The blood oxygen, electrocardiogram, and respiratory monitoring devices to be evaluated are wearable vital sign monitoring devices.

[0092] The simulation equipment includes a blood oxygen simulator, an electrocardiogram simulator, and a respiratory movement module.

[0093] (1) A blood oxygen simulator, for example, is selected as the SKX-1000FIII simulator, which is based on a reflective blood oxygen signal simulation fixture, installed on the head of a simulated dummy to simulate blood oxygen signals for monitoring by a reflective blood oxygen monitoring device, and used to test wearable blood oxygen products.

[0094] Simulated blood oxygen saturation range: 70.0%–100.0%, initial blood oxygen saturation value: 98.0%;

[0095] 100.0%~70.0%, error ≤1%;

[0096] Heart rate simulation range: not less than 30 bpm-240 bpm, resolution of 1 lbpm, error ≤ 1 bpm, initial value 80 bpm.

[0097] (2) Electrocardiogram (ECG) simulator, for example, the SK-2000T simulator is selected, and the ECG simulation function is embedded in the surface of the dummy's chest cavity through device integration. The signal is output using dry electrodes on the chest, which are fixed at specific detection positions to adapt to different detection methods.

[0098] Heart rate simulation range: not less than 30 bpm-240 bpm, resolution of 1 lbpm, error ≤ 1 bpm, initial value 80 bpm.

[0099] (3) The respiratory movement module, designed according to ergonomics, simulates the rib arch within the chest cavity and uses an electric actuator as the driving component to drive the rise and fall of the chest cavity contour. During simulated inhalation, the electric actuator is activated to push outwards, raising the entire chest wall, resulting in a forward and upward chest cavity position. The range of motion is freely controlled by the stroke of the electric actuator. During exhalation, the electric actuator contracts, and the chest wall returns to its original position. It meets the following main technical specifications: not less than 7 bpm-60 bpm, resolution of 1 lbpm, error ≤2 bpm, and initial value of 20 bpm.

[0100] like Figure 4 As shown, the respiratory motion module includes a thoracic push plate, a push rod, a guide rail, a base, a high-precision stepper motor, and an eccentric connector;

[0101] The thoracic cavity push plate is fixedly connected to the ribcage of the dummy and is used to simulate the expansion and contraction of the human thoracic cavity;

[0102] The push rod is connected to an eccentric connector at one end and to a thoracic push plate at the other end, and is used to transmit the motion of the high-precision stepper motor to the thoracic push plate.

[0103] The guide rail extends along the axial direction of the push rod, providing stable guidance for the push rod;

[0104] The base is used to support the push rod, guide rail, and high-precision stepper motor, as well as to deploy a high-precision MCU;

[0105] The high-precision stepper motor is fixed to the base by a bracket, with a minimum step size of m° and a rotation period of one revolution. Each pulse controls the reciprocating motion of the push rod;

[0106] The eccentric connector is used to connect to the output shaft of the high-precision stepper motor. The central shaft of the high-precision stepper motor drives the eccentric connector to rotate, which in turn drives the push rod to move.

[0107] For example, m° is 0.9°, but in specific applications, it can be changed according to specific needs; This means that a high-precision stepper motor needs to generate 400 steps, or 400 pulses, to complete one revolution. The eccentric connector is an eccentric wheel.

[0108] The high-precision MCU is used to control the pulse period of the high-precision stepper motor;

[0109] The high-precision MCU is based on the highest respiratory rate of the respiratory signal and... The minimum pulse period is obtained by multiplying the results, which serves as the pulse period requirement for the simulated respiratory signal.

[0110] Calculate a single clock cycle based on the operating frequency of the high-precision MCU;

[0111] If the single clock cycle is less than or equal to the pulse cycle requirement of the simulated breathing signal, then the breathing signal of the multimodal dummy simulated by the breathing motion module is used; otherwise, it is discarded.

[0112] For example, if the maximum respiratory rate is 70 bpm, the number of steps in 1 minute is 400 * 70 = 28000, then the minimum pulse period is 60000ms / 28000 = 2.142ms; the pulse period requirement for the simulated respiratory signal is 2.142ms.

[0113] For example, this invention uses a 48MHz high-precision MCU (Microcontroller Unit), with a single clock cycle of 1,000,000μs / 48,000,000 = 0.02μs. That is, the minimum pulse cycle generated by the 48MHz high-precision MCU is 0.02μs. 0.02μs is much smaller than the pulse cycle requirement of 2.142ms for the simulated breathing signal, so the output simulated breathing signal can be used; otherwise, if the single clock cycle generated by the MCU does not meet the pulse cycle requirement of the simulated breathing signal, the generated simulated breathing signal is discarded.

[0114] The high-precision MCU is placed inside the base and connected to the high-precision stepper motor via a shielded cable.

[0115] A multimodal mannequin wears blood oxygen, electrocardiogram, and respiratory monitoring equipment to be evaluated.

[0116] The blood oxygen monitoring device is a head-mounted device worn on the head of a multimodal dummy to obtain blood oxygen monitoring signals.

[0117] The electrocardiogram monitoring device is a dynamic electrocardiogram recorder, which is worn on the chest of a multimodal dummy to monitor electrocardiogram and respiratory signals.

[0118] After the device to be tested is put on, the monitoring data of blood oxygen, electrocardiogram and respiratory signals are automatically recorded and stored to the preset file path of the host computer.

[0119] The head-mounted pulse oximeter automatically records the generated pulse oximeter data file; the electrocardiogram (ECG) monitoring device automatically records the generated ECG and respiratory monitoring data file.

[0120] Use the host computer software to set the values ​​of simulated blood oxygen, electrocardiogram, and respiration signals.

[0121] The host computer software is used to set the blood oxygen saturation of the blood oxygen simulator, the respiratory rate of the respiratory exercise module, and the electrocardiogram waveform and heart rate of the electrocardiogram simulator.

[0122] The simulated ECG, respiration, and blood oxygenation signals are transmitted to the blood oxygen simulator, ECG simulator, and respiratory motion module on the multimodal dummy.

[0123] The blood oxygen simulator, electrocardiogram simulator, and respiratory motion module generate standard analog signals.

[0124] The aforementioned ECG simulator, respiratory exercise module, and blood oxygen simulator generate standard signals. The monitoring range and resolution of the blood oxygen, ECG, and respiratory simulation signals are no less than those of commonly used physiological monitoring equipment, and can meet the verification requirements.

[0125] During the monitoring process, simulated ECG, respiration, and blood oxygenation signal data files are generated and stored in a preset file path on the host computer.

[0126] The purpose of step S121 is to construct a multimodal dummy monitoring environment using simulation equipment to simulate blood oxygen, electrocardiogram and respiratory signals, and to acquire the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals through the wearable vital sign monitoring device, so as to provide a data basis for subsequent evaluation.

[0127] Step S122: Obtain the combined acceleration of the six-axis motion platform; wherein the multimodal dummy is mounted on the six-axis motion platform.

[0128] A multimodal dummy is mounted on the six-axis motion platform. The six-axis motion platform is placed vertically and is physically connected to a half-body multimodal dummy including its head.

[0129] The six-axis motion platform mainly consists of an electrical control box, host computer software, connecting cables, an upper platform, hinge components, and a lower platform.

[0130] The electrical control box provides the hardware platform for operation and control, including controllers, servo drives, etc. It is a convergence point for motion signals and servo drive signals, completing the exchange of these signals. After the control box is powered on, the simulation operation is completed.

[0131] The host computer software controls the motion state of the six-axis motion platform. This software is deployed on an external computer and is a standalone application connected to the six-axis motion platform via a data cable.

[0132] Connecting cables are used to enable electrical signal communication transmission between the electrical control cabinet and the six-axis motion platform and multimodal dummy.

[0133] The upper platform is constructed by bolting together profiles, and three sets of upper hinge components are connected by these profiles.

[0134] The hinge assembly includes three sets of upper hinges and three sets of lower hinges. It adopts a silent, high-precision Hooke hinge structure, which features a large swing angle, small fit clearance, stable transmission, flexible movement, and convenient maintenance.

[0135] The lower platform is constructed by bolting together profiles and is designed with connecting and positioning iron plates to enhance the connection strength and positioning accuracy of the six-axis motion platform. At the same time, the lower platform is fixedly connected to the pre-embedded high-strength anchor bolts to ensure the safe and reliable operation of the motion platform.

[0136] The host computer software sets the attitude and motion frequency of the six-axis motion platform for lateral movement, longitudinal movement, lifting, pitching, tilting, and yaw, thereby controlling the movement of the six-axis motion platform.

[0137] The motion parameters of the six-axis motion platform include the attitude of lateral movement, longitudinal movement, heave, pitch, tilt, yaw, and motion frequency. By adjusting the platform parameters to simulate the real dynamic scene of the flight cockpit, the six-axis motion platform generates different motion resultant acceleration values ​​in real time.

[0138] The platform parameters include maximum lateral displacement, maximum longitudinal displacement, maximum vertical displacement, maximum pitch angle, maximum tilt angle, maximum heading angle, maximum angular acceleration, angular position accuracy, linear position accuracy, and motion frequency.

[0139] The parameters of the six-axis motion platform are set via host computer software, and its movement is controlled. The six-axis motion platform generates different acceleration values. Finally, the six-axis motion platform generates a time-series acceleration file package.

[0140] The resultant acceleration of the six-axis motion platform is calculated as follows:

[0141] Obtain the resultant acceleration of the six-axis motion platform, including:

[0142] Obtain the maximum values ​​of the lateral, longitudinal, and vertical displacement distances of the six-axis motion platform at the current moment, and obtain the maximum distance of the six-axis motion platform from the equilibrium position;

[0143] The displacement of the six-axis motion platform is calculated based on the maximum distance of the six-axis motion platform from its equilibrium position.

[0144] The velocity of the six-axis motion platform is obtained by differentiating its displacement with respect to time.

[0145] The resultant acceleration of the six-axis motion platform is obtained by differentiating its velocity with respect to time.

[0146] The maximum distance A that the six-axis motion platform deviates from its equilibrium position is measured by the lateral displacement distance |X. Lat |, longitudinal displacement distance|Y Lon |, Vertical displacement distance|Z Axi | is obtained by taking the maximum value among the three.

[0147] A = MAX(|X Lat |,|Y Lon |,|Z Axi |) Formula (1)

[0148] The displacement y of the six-axis motion platform changes with time t as follows:

[0149] y=Asin(wt)=Asin(2πft) Formula (2)

[0150] Wherein, frequency f is the number of vibrations completed per second by the six-axis motion platform (unit: Hertz, Hz), and time is t (unit: seconds, s).

[0151] The velocity of the six-axis motion platform is obtained from the derivative of displacement with respect to time:

[0152] v=2Aπf*cos(2πft) Formula (3)

[0153] The resultant acceleration of a six-axis motion platform is obtained by the reciprocal of velocity with respect to time.

[0154] a = -4Aπ 2 f 2 *sin(2πft) formula (4)

[0155] The direction of the resultant acceleration is always opposite to the direction of the displacement, and the magnitude of the velocity changes continuously during the vibration, with a maximum value of 4Aπ. 2 f 2 The minimum value is 0. With a fixed amplitude A, the magnitude of the resultant acceleration is proportional to the square of the frequency, which means that the higher the frequency, the greater the resultant acceleration.

[0156] The purpose of step S122 is to simulate real dynamic scenes using a six-axis motion platform and a multimodal dummy to generate motion resultant acceleration, providing necessary resultant acceleration data for evaluating the performance of wearable vital sign monitoring devices in motion.

[0157] Step S123: Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, evaluate the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring device to be evaluated, and obtain the evaluation result of the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring device to be evaluated.

[0158] The evaluation of the dynamic accuracy of the blood oxygen, electrocardiogram (ECG), and respiration monitoring devices includes: calculating the simulated blood oxygen, ECG, and respiration signals and their corresponding Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index, and then normalizing them; based on the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index, a weighted summation is performed to obtain the dynamic accuracy evaluation results for blood oxygen, ECG, and respiration, respectively.

[0159] like Figure 2 As shown, the methods for assessing the dynamic accuracy of blood oxygen, electrocardiogram, or respiration are consistent.

[0160] Let X be the analog signal of blood oxygen, electrocardiogram or respiration, and let Y be the monitoring signal obtained by the blood oxygen, electrocardiogram or respiration monitoring device to be evaluated.

[0161] (1) Calculate the Pearson correlation coefficient between the analog signal and the monitoring signal of the blood oxygen, electrocardiogram and respiratory monitoring device to be evaluated.

[0162] The Pearson correlation coefficient ranges from -1 to +1. +1 indicates that the two compared signals are completely identical, while -1 indicates that the two compared signals are completely opposite. The calculation is as follows:

[0163]

[0164] Where conv(X,Y) is the covariance of the X and Y signals, and σ X Let σ be the standard deviation of the signal X. Y Let Y be the standard deviation of the signal Y.

[0165] When calculating the Pearson correlation coefficient score, the Pearson correlation coefficient is mapped to the interval [0,1], denoted as S. PCC ,as follows:

[0166]

[0167] The Pearson correlation coefficient S between the analog signal and the monitoring signal was obtained. PCC .

[0168] (2) Calculate the mean square error of the analog signal and the monitoring signal.

[0169] The mean square error (MSE) is the expected value of the square of the difference between the analog signal and the monitored signal. The MSE between the analog signal and the monitored signal, calculated using n sampled data points, is as follows:

[0170]

[0171] Among them, X iY is the i-th sampling point of the analog signal. i Let be the i-th sampling point of the monitored signal. The Mean-Square Error (MSE) has no upper or lower limit. When calculating the MSE score, this invention uses the absolute value of the MSE divided by the absolute value of the average value of the analog signal to obtain the average deviation ratio of the monitored signal, denoted as S. MSE ,as follows:

[0172]

[0173] The mean square error S of the analog signal and the monitoring signal is obtained. MSE .

[0174] (3) Calculate the structural similarity index between the analog signal and the monitoring signal.

[0175] The structural similarity index takes into account the structural information of the object being analyzed. Originally used for comparative analysis of images, it has been transferred to signals, as follows:

[0176]

[0177] Where, μ X and μ Y The averages of signals X and Y are σ and σ', respectively. X and σ Y Let σ be the standard deviation of signals X and Y, respectively. XY Let X be the covariance between signals X and Y. c1, c2, and c3 are small constants introduced to avoid the denominator being zero, and are usually the squares of a small value relative to the dynamic range of the signal.

[0178] The structural similarity index ranges from -1 to 1. A higher value indicates that the two input signals are more similar. Mapping it to the interval [0,1], it is denoted as S. SSIM ,as follows:

[0179]

[0180] The structural similarity index S is obtained after normalizing the analog signal and the monitoring signal. SSIM .

[0181] (4) Calculate the spectral angle mapping between the analog signal and the monitoring signal.

[0182] Treating the signal as a high-dimensional vector, the similarity between the spectra is measured by calculating the angle between two vectors; the smaller the angle, the more similar the two spectra. The spectra t and r of the analog signal X and the corresponding monitored signal Y are obtained using Fast Fourier Transform, as follows:

[0183]

[0184] Where C is the spectrum length, and the range of the inverse cosine function is [0,π]. Therefore, dividing the result of the spectrum angle mapping by π will map the score to the interval [0,1], as follows:

[0185]

[0186] The normalized spectral angle mapping S of the analog signal and the monitoring signal is obtained. SAM .

[0187] (5) Calculate the spectral information divergence of the analog signal and the monitoring signal.

[0188] Spectral information divergence is a spectral classification method based on information theory that measures the difference between two spectra. It treats spectral vectors as random variables and analyzes the similarity between two random vectors based on probability and statistics theory. That is, the smaller the value of spectral information divergence, the more similar the two sets of spectra are. To define spectral information divergence, the probability vectors p and q of the analog signal and the monitored signal X, Y are calculated.

[0189]

[0190] Where L is the number of sampling points in the analog signal and the monitoring signal X,Y, and l is the sampling point index.

[0191] Formula (13) represents the proportion of the l-th sampling point of the analog signal X among all sampling points, i.e., the probability distribution of the analog signal;

[0192] Formula (14) represents the proportion of the l-th sampling point of the monitoring signal Y among all sampling points, i.e. the probability distribution of the monitoring signal.

[0193] SID(X,Y)=D(X||Y)+D(Y||X) Formula (15)

[0194] in,

[0195] SID(X,Y) represents the total divergence between the analog signal and the monitoring signal.

[0196] D(X||Y) and D(Y||X) are relative entropies, also known as KL (Kullback-Leibler divergence). D(X||Y) represents the KL divergence of the distribution of the analog signal X relative to the distribution of the monitored signal Y; D(Y||X) represents the KL divergence of the distribution of the monitored signal Y relative to the distribution of the analog signal X.

[0197] Since the spectral information divergence has no upper or lower bounds, it is mapped to the range [0,1] using min-max scaling within the same batch of data, as follows:

[0198] SSID =MinMax(SID(X,Y)) Formula (16)

[0199] The normalized spectral information divergence S of the analog signal and the monitoring signal is obtained. SID .

[0200] (6) Calculate the complex wavelet structure similarity index between the analog signal and the monitoring signal.

[0201] The complex wavelet structure similarity index is a metric used to evaluate image similarity. When applied to signal analysis, it works as follows:

[0202]

[0203] Among them, c X,i c represents the i-th layer complex wavelet coefficients of the X signal after complex wavelet transformation; Y,i These are the complex wavelet coefficients of the i-th layer of the Y signal after complex wavelet transformation; For c Y,i The conjugate of ; N is the number of coefficients obtained after the signal undergoes complex wavelet transform; K is a small constant used to avoid the denominator being zero, and is usually taken as the square of a small value relative to the dynamic range of the signal.

[0204] The complex wavelet structure similarity index ranges from 0 to 1, so there is no need to unify the dimensions.

[0205] (7) Dynamic accuracy assessment result value.

[0206] To unify the dimensions, the results are mapped to the [0,1] interval using the following method.

[0207] The results of the dynamic accuracy assessment are as follows:

[0208] S accuracy =a*S PCC +b*S SAM +c*S SID +d*S SAM +e*S SID +f*S CWSSIM Formula (18)

[0209] Among them, S accuracy S represents the score for the accuracy assessment results. PCC S SAM S SID S SAM S SID and S CWSSIM, respectively, are the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index of the analog signal and the monitoring signal; a, b, c, d, e, and f are the corresponding weights.

[0210] Specifically, the Pearson correlation coefficient and mean squared error excel at measuring the large-scale and trend consistency of signals, and are therefore given greater weight, emphasizing the similarity of signals at the macroscopic level. In contrast, the structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index focus more on the consistency of signals at the small scale and in detail. By assigning weights to these, the matching of signals at the level of detail can be better reflected.

[0211] The weighted average is a + b + c + d + e + f = 1. For example, a = 0.37, b = 0.21, c = 0.14, d = 0.12, e = 0.10, and f = 0.06. In practical applications, this weighting can be adjusted according to specific needs. This weighting balances the consistency of large-scale trends with the consistency of small-scale details, ensuring that the evaluation results fully consider the overall matching degree of the signal while not neglecting differences in detail.

[0212] If 0.9 < S accuracy If the value is ≤1.0, the accuracy of blood oxygen, electrocardiogram, or respiratory monitoring equipment is relatively high;

[0213] If 0.7 < S accuracy If the value is ≤0.9, the accuracy of blood oxygen, electrocardiogram, or respiratory monitoring equipment is generally low.

[0214] If 0 ≤ S accuracy If the value is ≤0.7, the accuracy of blood oxygen, electrocardiogram, or respiratory monitoring equipment is poor.

[0215] When the accuracy score of the metrological evaluation software test is between 0.9 and 1.0, the accuracy performance of the monitoring equipment test is considered excellent; when it is between 0.7 and 0.9, the accuracy performance of the monitoring equipment test is considered average; and when it is between 0 and 0.7, the accuracy performance of the monitoring equipment test is considered poor.

[0216] The purpose of step S123 is to quantify the dynamic accuracy of wearable vital sign monitoring devices in dynamic scenarios by calculating multiple evaluation indicators and weighting them together.

[0217] Step S124: Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, as well as the combined acceleration of motion, evaluate the performance of the wearable vital signs monitoring device and obtain the performance evaluation results.

[0218] Based on the obtained simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, as well as the combined acceleration of motion, the performance of the blood oxygen, electrocardiogram and respiratory monitoring device to be evaluated is assessed, and the performance evaluation results are obtained.

[0219] (1) Calculate the ratio of the absolute error of the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals at each time point.

[0220] The calculation method for the absolute value ratio of errors between simulated blood oxygen, electrocardiogram, and respiratory signals and their corresponding monitoring signals is consistent.

[0221] The calculation of the ratio of the absolute error between the analog signals and the corresponding monitored signals of blood oxygen, electrocardiogram, and respiration at each moment includes:

[0222] Calculate the errors between the simulated and monitored signals for blood oxygen, electrocardiogram, and respiration at each time point;

[0223] The average values ​​of blood oxygen, electrocardiogram and respiratory simulated signals were calculated separately and used as the average standard data for blood oxygen, electrocardiogram and respiratory.

[0224] Based on the errors between the simulated and detected signals of blood oxygen, electrocardiogram, and respiration at each moment, and the average standard data of blood oxygen, electrocardiogram, and respiration, the absolute value ratio of the errors of blood oxygen, electrocardiogram, and respiration at each moment is calculated respectively.

[0225] Based on the resultant acceleration, monitoring signal data, and analog signal data, a polynomial fitting method is used to fit the polynomial relationship between the percentage of absolute error and the resultant acceleration modulus. Outliers are processed using the RANSAC (Random Sample Consensus) algorithm, and the fitted polynomial coefficients are output.

[0226] Given analog signal X and monitored signal Y, calculate the percentage of the absolute value of the error as follows:

[0227] ① For each time moment, calculate the error ∈ between the monitored signal and the analog signal, as follows:

[0228] Error = |Monitoring signal - Analog signal| (Formula 19)

[0229] ∈=|XY| Formula (20)

[0230] ② Calculate the average value of the analog signal, denoted as the average standard data, as follows:

[0231]

[0232] Where n is the total number of sampled data points.

[0233] ③ Obtain the proportion of the absolute value of the error at each time step ∈ I The calculation is as follows:

[0234]

[0235] ④ Obtain the complete sequence of absolute error proportions, which will serve as input data for subsequent polynomial fitting.

[0236] (3) Obtain performance evaluation results of blood oxygenation, electrocardiogram and respiration.

[0237] Obtain performance evaluation results for blood oxygenation, electrocardiogram, and respiration, including:

[0238] The ratio of the absolute error of the simulated signals of blood oxygen, electrocardiogram and respiration to the corresponding monitored signals at each time moment, as well as the resultant acceleration of motion at the corresponding time moment, are calculated. Based on the RANSAC algorithm, a polynomial fitting is performed to obtain the error-acceleration fitting relationship curve.

[0239] Based on the resultant acceleration, the resultant acceleration modulus is calculated;

[0240] Using the maximum and minimum values ​​of the combined acceleration modulus as the integration interval, the error-acceleration fitting curve is integrally integrated to obtain the area S enclosed by the error-acceleration fitting curve and the horizontal axis, which is used to evaluate the performance of the blood oxygen, electrocardiogram and respiratory monitoring devices to be evaluated.

[0241] Calculate the resultant acceleration modulus, |a|, which represents the resultant acceleration of an object in three-dimensional space. It is calculated using a six-axis motion platform.

[0242] Polynomial fitting, as follows:

[0243] Input: The error absolute value ratio sequence is y, and the resultant acceleration modulus sequence is x.

[0244] Find an nth-degree polynomial function y = p(x) that describes the underlying relationship between the independent variable x (the resultant acceleration modulus) and the dependent variable y (the proportion of the absolute value of the error) with the minimum error.

[0245] The function has the following form:

[0246] y = p(X) = c n *x n +c n-1 *x n-1 Formula (23) is: +...+c1*x+c0

[0247] Where c0, c1……c n These are the model coefficients that need to be solved using a fitting algorithm. c0, c1……c nThe confirmation is based on the RANSAC (RANdom Sampling Consensus) algorithm.

[0248] The RANSAC algorithm is used to detect and remove outliers in data. During the fitting process, the RANSAC algorithm repeatedly selects a subset of data for fitting based on a given tolerance, and finally selects the optimal fitting model and removes outliers.

[0249] After polynomial fitting, the coefficients of the fitted model are output. These coefficients will form the final polynomial equation, which describes the relationship between the percentage of the absolute value of the error and the modulus of the resultant acceleration.

[0250] A smaller tolerance may result in more data points being labeled as outliers, which could exclude some data that is actually consistent with the model, thus affecting the model's accuracy; a larger tolerance may incorrectly label some real outliers as inliers, thus affecting the model's robustness and reliability.

[0251] For example, the given tolerance is tolerance = 0.5, which can be adjusted according to specific needs in a particular application.

[0252] If a quadratic polynomial (degree = 2) is fitted, and the coefficient array is [c0, c1, c2], then the final fitted function is:

[0253] y = c0 + c1x + c2x 2 Formula (24)

[0254] Where: x is the resultant acceleration modulus, y is the percentage of the absolute value of the predicted error, c0 is the intercept term, representing the percentage of the absolute value of the error when the resultant acceleration is 0, c1 is the coefficient of the first term, reflecting the trend of the percentage of the absolute value of the error changing linearly with the resultant acceleration, and c2 is the coefficient of the second term, reflecting the trend of the percentage of the absolute value of the error changing nonlinearly (accelerating or decelerating) with the resultant acceleration.

[0255] This function allows for the prediction or compensation of measurement errors based on any given resultant acceleration value. While the polynomial function itself provides a detailed error-acceleration relationship, a single, intuitive, and comprehensive performance metric is needed for cross-sectional comparisons between different devices. Therefore, the cumulative error level of the device across the entire range of motion intensity is quantified by calculating the area under the fitted curve.

[0256] This is achieved by performing a definite integral on the finally fitted polynomial function p(x) over a specified range of resultant accelerations [a, b]. For example, the integration interval consists of the minimum and maximum resultant acceleration values ​​obtained, covering the entire dynamic range of the test.

[0257] like Figure 3 As shown, the performance evaluation score is calculated as follows:

[0258]

[0259] Where p(x) is a polynomial function fitted by the RANSAC algorithm, a is the minimum value of the resultant acceleration modulus, and b is the maximum value of the resultant acceleration modulus.

[0260] This integral value represents the area enclosed by the absolute value of the error curve and the horizontal axis, intuitively reflecting the overall magnitude of the error throughout the entire process from rest to high-intensity motion.

[0261] Figure 3 In the figure, the horizontal axis represents the magnitude of the resultant acceleration, and the vertical axis represents the percentage of the absolute value of the error.

[0262] The smaller the S score, the better the performance.

[0263] A lower S-score means that the fitted curve is closer to the x-axis, indicating that the device has a lower cumulative error across the entire motion intensity range, stronger resistance to motion interference, and higher overall data quality.

[0264] The S score is used to assess the anti-interference capability of the blood oxygen, ECG, and respiratory monitoring devices being evaluated, and is used to quickly and objectively compare the robustness of different devices.

[0265] If 0 < S ≤ 4, then the blood oxygen, ECG and respiratory monitoring equipment to be evaluated has excellent performance;

[0266] If 4 < S ≤ 12, the performance of the blood oxygen, ECG and respiratory monitoring equipment to be evaluated is generally average.

[0267] If S > 12, the performance of the blood oxygen, ECG, and respiratory monitoring equipment to be evaluated is poor.

[0268] The purpose of step S124 is to comprehensively simulate the influence of physiological signals and motion acceleration, and to evaluate the performance of the blood oxygen, electrocardiogram and respiratory monitoring devices to be evaluated by quantifying the error and performing fitting analysis.

[0269] Step S1 involves dynamic calibration and multimodal synchronous acquisition, using high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment to acquire the pilot's blood oxygen, electrocardiogram and respiratory signals, as well as using an airborne camera to capture facial video data of the pilot, and ensuring that the data is complete, safe and traceable.

[0270] Step S2, specifically.

[0271] The real-time monitored blood oxygen, electrocardiogram, and respiratory signals, as well as facial video, are transmitted to the ground command center after data encryption.

[0272] For example, the pilot's blood oxygen, electrocardiogram and respiratory signals will be monitored in real time, and transmitted to the ground command center via a 5G private network after being encrypted with SM4 data.

[0273] The pilot's blood oxygen, electrocardiogram, and respiratory physiological signals, as well as facial video frames collected in step S1, are aligned according to UTC timestamps to form a 1-second cycle of synchronous data packets.

[0274] Each data packet is appended with a frame sequence number, a CRC32 checksum, and the pilot's anonymous ID to ensure integrity.

[0275] Data packets are encrypted using the SM4 block cipher algorithm, with each 512 bytes forming an encryption unit, to avoid delays caused by excessively large single packets.

[0276] The airborne terminal accesses satellites or ground base stations via a 5G aviation private network, with a link bandwidth of ≥10Mbps and an end-to-end latency of ≤80ms.

[0277] Redundancy and retransmission: UDP+FEC (forward error correction code RS(255,223)) hybrid mode is enabled. No retransmission is required when the packet loss rate is <1%. If the link is interrupted momentarily, the onboard buffer can store 120 seconds of encrypted data, which will be retransmitted after the link is restored.

[0278] The ground control center completed the SM4 decryption, and the decrypted data was directly put into the isolated intranet for the flight fatigue prediction model to use in real time.

[0279] Step S2 encapsulates, SM4 encrypts, transmits via a highly reliable 5G private network, and decrypts multimodal physiological data and facial video data in real time throughout the flight, providing the ground command center with complete, low-latency, and secure input data for a well-trained flight fatigue prediction model.

[0280] Step S3 includes steps S31-S32.

[0281] Step S31: Based on the real-time monitoring of the pilot's blood oxygen, electrocardiogram, respiratory signals and facial video, the ground command center inputs the trained flight fatigue prediction model to obtain the pilot's comprehensive physiological fatigue level, and generates decision instructions based on the comprehensive physiological fatigue level.

[0282] The comprehensive physiological fatigue level (5-level quantitative system) is exemplarily shown in Table 1.

[0283] Table 1: Comprehensive Physiological Fatigue Levels

[0284]

[0285] The definition of comprehensive physiological fatigue levels can be modified according to specific needs in practical applications.

[0286] The overall physiological fatigue level and corresponding decision instructions are exemplarily shown in Table 2.

[0287] Table 1: Comprehensive Physiological Fatigue Levels

[0288] Overall physiological fatigue level Decision instructions L0 No action required, maintain the current monitoring frequency. L1 The voice prompt suggests doing stretching exercises in 10 minutes. L2 Activate co-pilot standby, prompting a message to prepare for takeover. L3 Immediately trigger automatic flight takeover and push the nearest alternate landing site. L4 Emergency descent procedures were initiated, and the control tower prioritized clearing the runway.

[0289] All raw blood oxygen, ECG, and respiration data, as well as facial video data, along with corresponding feature data, comprehensive physiological fatigue levels, and decision instructions, are written to the time-series database InfluxDB and retained for 90 days to support post-event playback and model retraining.

[0290] Step S31 involves the ground command center completing a closed-loop decision-making process in real time, from "ECG, blood oxygen, respiration, facial video → flight fatigue prediction model → comprehensive physiological fatigue level → decision command". All raw ECG, blood oxygen, respiration, facial video data, and derived data are written into the time-series database InfluxDB for immediate intervention and post-event traceability.

[0291] Step S32: The flight fatigue prediction model is trained through the following process:

[0292] After obtaining and preprocessing the blood oxygen, electrocardiogram, respiratory signals and facial videos of multiple pilots within a certain historical period, the data are combined with sample labels to form a flight fatigue sample set; where the sample labels are the pilots' fatigue levels.

[0293] Construct a flight fatigue prediction model;

[0294] The flight fatigue prediction model is trained using the flight fatigue sample set. The forward propagation uses the cross-entropy loss function, and the backpropagation uses the AdamW optimizer to update the model parameters.

[0295] The model parameters are saved as a trained flight fatigue prediction model until the joint loss function converges.

[0296] (1) Obtain blood oxygen, electrocardiogram, respiratory signals and facial video data of multiple pilots on multiple flight segments within a certain historical period.

[0297] For example, the data collection period was 18 months, and the data included blood oxygen, electrocardiogram, respiratory signals and facial video data of 260 pilots and 9420 flight segments.

[0298] The sample labels are shown in Table 1, which lists five discrete fatigue levels, L0-L4.

[0299] (2) Data preprocessing.

[0300] The three physiological signals, blood oxygen, electrocardiogram and respiration, are first resampled at 1Hz, then bandpassed at 0.5-4Hz (i.e., noise below 0.5 Hz and above 4 Hz is filtered out), and then Z-score normalized.

[0301] The facial video is cropped to 224×224 pixels, and frames are captured at 30fps (i.e., 30 images per second). Frame-level PERCLOS calculation (i.e., eyelid closure time) is performed.

[0302] Construct a flight fatigue prediction model, with the network structure as follows: Figure 5 As shown.

[0303] The network structure of the flight fatigue prediction model includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer.

[0304] The feature extraction layer includes parallel first, second, and third deep residual networks and EfficientNet networks; the first, second, and third deep residual networks and EfficientNet networks are followed by corresponding first, second, third, and fourth global pooling layers and first, second, third, and fourth fully connected layers in sequence;

[0305] The first, second, and third deep residual networks are used to extract blood oxygen, electrocardiogram, and respiratory signal features, respectively.

[0306] The EfficientNet network is used to extract facial video features;

[0307] The first, second, third, and fourth global pooling layers are used to perform global average pooling on blood oxygen, ECG, respiratory signal features, and facial video features, respectively, converting the feature maps into fixed-length feature vectors, which are then input to the corresponding first, second, third, and fourth fully connected layers.

[0308] The first, second, third, and fourth fully connected layers are used to concatenate the fixed-length feature vectors output by the first, second, third, and fourth global pooling layers into corresponding long vectors for blood oxygen, ECG, respiration, and facial video.

[0309] The feature fusion layer is a Transformer fusion module, used to fuse long vectors of blood oxygen, ECG, respiration and facial video to generate a comprehensive feature vector;

[0310] The output layer includes a parallel Softmax layer and a Sogmoid layer, which are used to output fatigue level and fatigue value based on the comprehensive feature vector, respectively.

[0311] The input layer receives preprocessed blood oxygen, ECG, and respiratory signals, as well as facial video features.

[0312] (1) Input layer

[0313] Preprocessed blood oxygen signal: feature dimension is (1,60), that is, 1 channel, 60 time points for each sample.

[0314] Preprocessed ECG signals: The input feature dimension is (1,250), that is, 1 channel, and 250 time points for each sample.

[0315] Preprocessed respiratory signals: The input feature dimension is (1,60), that is, 1 channel, and 60 time points for each sample.

[0316] Preprocessed facial video: The input feature dimension is (3,224,224,T), that is, 3 channels, each channel is 224x224 pixels, and T is the number of video frames.

[0317] (2) Feature extraction layer

[0318] The first, second, and third deep residual networks (corresponding to blood oxygen, ECG, and respiration) have the following inputs: blood oxygen and respiration signals are (1, 60), and ECG signals are (1, 250); the output feature dimension is (256), i.e., a 256-dimensional feature vector.

[0319] EfficientNet network (corresponding to facial video): input feature dimension (3,224,224,T), output feature dimension (256);

[0320] (3) The first, second, third and fourth global pooling layers are global average pooling layers. The input is the feature vectors from the first, second and third deep residual networks and the EfficientNet network. The feature dimension is 256. The output feature dimension is still 256 because the effect of global average pooling is to reduce the spatial dimension, but the feature dimension remains unchanged.

[0321] (4) The first, second, third and fully connected layers have an input feature dimension of 1024 and an output feature dimension of 512 after processing by the fully connected layers.

[0322] (5) Transformer fusion module, input feature dimension 512, which is the output of the fully connected network; after processing by Transformer, the output feature dimension is still 512.

[0323] (6) Output layer: Softmax classification head, input feature dimension 512, output probability values ​​of five fatigue levels; Sigmoid regression head outputs continuous fatigue values, ranging from 0 to 1.

[0324] The flight fatigue prediction model, from multimodal input to final fatigue level and continuous fatigue value output, achieves accurate prediction of pilot fatigue status through a series of feature extraction, fusion, and classification / regression tasks.

[0325] The cross-entropy loss and mean squared error loss are combined to obtain the joint loss function.

[0326] Cross-entropy loss measures the difference between the probability distribution predicted by the flight fatigue prediction model and the actual sample labels. For multi-class classification problems (fatigue level classification), cross-entropy loss is expressed as follows:

[0327]

[0328] Where C represents the number of categories, which in this invention are 5 fatigue levels, y i The true label of class i is usually represented by one-hot encoding. Predict the probability of class i for the model.

[0329] Mean squared error loss measures the squared difference between the model's predicted values ​​and the actual values. For continuous value predictions, such as regression of pilot fatigue levels, the mean squared error loss function is expressed as follows:

[0330]

[0331] Where N is the number of samples.

[0332] The joint loss function is expressed as follows:

[0333] L total =L CE +λ·L MSE

[0334] Here, λ is a weighting coefficient used to balance the influence of the two loss terms. For example, λ is set to 0.2, indicating that the weight of the MSE loss is 20% of the cross-entropy.

[0335] The purpose of step S3 is to use the trained flight fatigue prediction model to process and integrate the multi-source physiological and facial behavior data of the pilot during flight, to assess the pilot's fatigue level in real time, and to generate corresponding decision instructions to ensure flight safety.

[0336] In summary, the method for monitoring a pilot's physiological state during flight according to an embodiment of the present invention has the following beneficial effects:

[0337] 1. This invention uses simulation equipment and a six-axis motion platform to dynamically evaluate the accuracy of monitoring equipment, ensuring that the equipment can still provide high-precision monitoring data under highly dynamic environments (such as acceleration and vibration during flight). It employs multimodal data fusion, utilizing a deep residual network (ResNet) to extract features from blood oxygen, electrocardiogram, and respiratory signals, and using EfficientNet to extract features from facial videos. The multimodal features are then fused using a Transformer module. This fusion method fully leverages the advantages of each modality of data, overcomes the lag problem of single-modal data evaluation, and comprehensively and accurately assesses the pilot's physiological state, providing a more reliable guarantee for flight safety.

[0338] 2. This invention employs encrypted data transmission to enhance data security and real-time performance. It uses encrypted data chains (such as SM4 encryption) to encrypt and transmit real-time monitored blood oxygen, ECG, respiratory signals, and facial video, ensuring data security and integrity during transmission. This effectively prevents data leakage or malicious attacks, protecting pilot privacy and flight data security. Real-time monitoring and decision-making are performed by transmitting real-time monitored multimodal data to the ground command center and inputting it into a trained flight fatigue prediction model to quickly obtain the pilot's comprehensive physiological fatigue level, generating decision instructions based on this. This process enables real-time monitoring and rapid decision-making of the pilot's physiological state, allowing for timely detection of potential health risks and the implementation of corresponding intervention measures, thereby improving flight safety and efficiency.

[0339] 3. The flight fatigue prediction model of this invention adopts a network architecture that includes a parallel deep residual network, an EfficientNet network, and a Transformer fusion module. This architecture can not only effectively extract features from each modality of data, but also fully integrate information between different modalities through the Transformer fusion module, improving the model's ability to identify and predict pilot fatigue states. The model is trained using a flight fatigue sample set consisting of blood oxygen, ECG, respiratory signals, and facial videos from multiple pilots over a certain historical period. This allows the model to learn the fatigue characteristics of different pilots, different flight missions, and different environmental conditions, improving the accuracy and generalization ability of the fatigue assessment model. This means that the model can more accurately assess pilot fatigue states in actual flight environments, providing stronger support for flight safety.

[0340] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0341] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring the physiological state of a pilot during flight, characterized in that, include: During flight, high-precision blood oxygen, electrocardiogram and respiratory monitoring equipment are used to monitor the pilot's blood oxygen, electrocardiogram and respiratory signals in real time, and airborne cameras are used to capture facial videos; The real-time monitored blood oxygen, ECG and respiratory signals, as well as facial video, are transmitted to the ground command center after data encryption. The ground command center, based on real-time monitoring of the pilot's blood oxygen, electrocardiogram, respiratory signals, and facial video, inputs a trained flight fatigue prediction model to obtain the pilot's comprehensive physiological fatigue level, and generates decision instructions based on the comprehensive physiological fatigue level. The high-precision blood oxygen, electrocardiogram, and respiration monitoring equipment was obtained through the following evaluation steps: The blood oxygen, electrocardiogram (ECG), and respiratory signals of a multimodal dummy are simulated using simulation equipment, and the corresponding blood oxygen, ECG, and respiratory monitoring signals of the multimodal dummy are monitored using blood oxygen, ECG, and respiratory monitoring equipment; wherein, the simulation equipment includes a blood oxygen simulator, an ECG simulator, and a respiratory motion module; The resultant acceleration of the six-axis motion platform is obtained; wherein the multimodal dummy is disposed on the six-axis motion platform; Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the dynamic accuracy evaluation result is obtained. Based on the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding blood oxygen, electrocardiogram and respiratory monitoring signals, as well as the combined acceleration of motion, the performance of the blood oxygen, electrocardiogram and respiratory monitoring device is evaluated, and the performance evaluation results are obtained. If both the dynamic accuracy assessment results and the performance assessment results meet the predetermined requirements, a high-precision blood oxygen, electrocardiogram and respiratory monitoring device will be obtained. The evaluation of the dynamic accuracy of the blood oxygen, electrocardiogram and respiratory monitoring equipment includes: calculating the simulated blood oxygen, electrocardiogram and respiratory signals and the corresponding Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence and complex wavelet structural similarity index of the blood oxygen, electrocardiogram and respiratory monitoring signals, and normalizing them; The dynamic accuracy assessment results of blood oxygen, electrocardiogram and respiration are obtained by weighted summation based on the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence and complex wavelet structural similarity index. Obtain performance evaluation results for blood oxygenation, electrocardiogram, and respiration, including: The ratio of the absolute error of the simulated signals of blood oxygen, electrocardiogram and respiration to the corresponding monitored signals at each time moment, as well as the resultant acceleration of motion at the corresponding time moment, are calculated. Based on the RANSAC algorithm, a polynomial fitting is performed to obtain the error-acceleration fitting relationship curve. Based on the resultant acceleration, the resultant acceleration modulus is calculated; Using the maximum and minimum values ​​of the combined acceleration modulus as the integration interval, the error-acceleration fitting curve is integrally integrated to obtain the area S enclosed by the error-acceleration fitting curve and the horizontal axis, which is used to evaluate the performance of the blood oxygen, electrocardiogram and respiratory monitoring devices to be evaluated.

2. The method according to claim 1, characterized in that, The flight fatigue prediction model is trained through the following process: After obtaining and preprocessing the blood oxygen, electrocardiogram, respiratory signals and facial videos of multiple pilots within a certain historical period, the data are combined with sample labels to form a flight fatigue sample set; where the sample labels are the pilots' fatigue levels. Construct a flight fatigue prediction model; The flight fatigue prediction model is trained using the flight fatigue sample set. The forward propagation uses the cross-entropy loss function, and the backpropagation uses the AdamW optimizer to update the model parameters. The model parameters are saved as a trained flight fatigue prediction model until the joint loss function converges.

3. The method according to claim 2, characterized in that, The network structure of the flight fatigue prediction model includes an input layer, a feature extraction layer, a feature fusion layer, and an output layer. The feature extraction layer includes parallel first, second, and third deep residual networks and EfficientNet networks; the first, second, and third deep residual networks and EfficientNet networks are followed by corresponding first, second, third, and fourth global pooling layers and first, second, third, and fourth fully connected layers in sequence; The first, second, and third deep residual networks are used to extract blood oxygen, electrocardiogram, and respiratory signal features, respectively. The EfficientNet network is used to extract facial video features; The first, second, third, and fourth global pooling layers are used to perform global average pooling on blood oxygen, ECG, respiratory signal features, and facial video features, respectively, converting the feature maps into fixed-length feature vectors, which are then input to the corresponding first, second, third, and fourth fully connected layers. The first, second, third, and fourth fully connected layers are used to concatenate the fixed-length feature vectors output by the first, second, third, and fourth global pooling layers into corresponding long vectors for blood oxygen, ECG, respiration, and facial video. The feature fusion layer is a Transformer fusion module, used to fuse long vectors of blood oxygen, ECG, respiration and facial video to generate a comprehensive feature vector; The output layer includes a parallel Softmax layer and a Sogmoid layer, which are used to output fatigue level and fatigue value based on the comprehensive feature vector, respectively.

4. The method according to claim 1, characterized in that, Obtain the resultant acceleration of the six-axis motion platform, including: Obtain the maximum values ​​of the lateral, longitudinal, and vertical displacement distances of the six-axis motion platform at the current moment, and obtain the maximum distance of the six-axis motion platform from the equilibrium position; The displacement of the six-axis motion platform is calculated based on the maximum distance of the six-axis motion platform from its equilibrium position. The velocity of the six-axis motion platform is obtained by differentiating its displacement with respect to time. The resultant acceleration of the six-axis motion platform is obtained by differentiating its velocity with respect to time.

5. The method according to claim 1, characterized in that, The calculation of the ratio of the absolute error between the analog signals and the corresponding monitored signals of blood oxygen, electrocardiogram, and respiration at each moment includes: Calculate the errors between the simulated and monitored signals for blood oxygen, electrocardiogram, and respiration at each time point; The average values ​​of blood oxygen, electrocardiogram and respiratory simulated signals were calculated separately and used as the average standard data for blood oxygen, electrocardiogram and respiratory. Based on the errors between the simulated and detected signals of blood oxygen, electrocardiogram, and respiration at each moment, and the average standard data of blood oxygen, electrocardiogram, and respiration, the absolute value ratio of the errors of blood oxygen, electrocardiogram, and respiration at each moment is calculated respectively.

6. The method according to any one of claims 4-5, characterized in that, The results of the dynamic accuracy assessment are as follows: ； in, The score is used to assess the accuracy of the results; , , , , and These are the normalized Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index for the analog and monitoring signals, respectively. These are the weights corresponding to the Pearson correlation coefficient, mean square error, structural similarity index, spectral angle mapping, spectral information divergence, and complex wavelet structural similarity index for the analog and monitored signals, respectively.

7. The method according to claim 1, characterized in that, The respiratory motion module includes a thoracic push plate, a push rod, a guide rail, a base, a high-precision stepper motor, and an eccentric connector; The thoracic cavity push plate is fixedly connected to the ribcage of the dummy and is used to simulate the expansion and contraction of the human thoracic cavity; The push rod is connected to an eccentric connector at one end and to a thoracic push plate at the other end, and is used to transmit the motion of the high-precision stepper motor to the thoracic push plate. The guide rail extends along the axial direction of the push rod, providing stable guidance for the push rod; The base is used to support the push rod, guide rail, and high-precision stepper motor, as well as to deploy a high-precision MCU; The high-precision stepper motor is fixed to the base by a bracket, with a minimum step size. One week of operation Each pulse controls the reciprocating motion of the push rod; The eccentric connector is used to connect to the output shaft of the high-precision stepper motor. The central shaft of the high-precision stepper motor drives the eccentric connector to rotate, which in turn drives the push rod to move. The high-precision MCU is used to control the pulse period of the high-precision stepper motor; The high-precision MCU is based on the highest respiratory rate of the respiratory signal and... The minimum pulse period is obtained by multiplying the results, which serves as the pulse period requirement for the simulated respiratory signal. Calculate a single clock cycle based on the operating frequency of the high-precision MCU; If the single clock cycle is less than or equal to the pulse cycle requirement of the simulated breathing signal, then the breathing signal of the multimodal dummy simulated by the breathing motion module is used; otherwise, it is discarded.

Images