A non-contact firefighter vital sign anomaly monitoring and early warning method and system
By integrating multimodal sensors of millimeter-wave radar and infrared thermal imaging, combined with environmental perception and inertial measurement units, the problem of firefighters' vital sign monitoring equipment being easily detached and false alarms in fire scenes has been solved, achieving high-precision vital sign monitoring and graded early warning, thus improving the safety of firefighters.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 霍小齐
- Filing Date
- 2026-05-19
- Publication Date
- 2026-06-30
AI Technical Summary
Existing contact-based vital sign monitoring equipment is prone to detachment and displacement in fire scenes, and single radar modes are difficult to accurately measure vital signs under high temperature, dense smoke and strong movement. Existing early warning methods have a high false alarm and false alarm rate and cannot reliably extract multi-parameter vital signs and provide effective early warning in extreme environments.
Employing a multimodal sensor integrating millimeter-wave radar and infrared thermal imaging, combined with environmental perception and inertial measurement units, and through dynamic baseline prediction and LSTM trend analysis, it achieves non-contact monitoring and graded early warning of heart rate, respiratory rate, and body temperature. Environmental adaptive compensation and motion artifact suppression technologies are used to improve signal accuracy and early warning accuracy.
It has achieved high-precision heart rate, respiratory rate and body temperature measurement in the extreme environment of fire scene, reduced false alarm rate, improved the timeliness and accuracy of early warning, and reduced the burden of clothing on firefighters.
Smart Images
Figure CN122296841A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to vital sign monitoring and abnormal early warning technology, specifically to a non-contact method and system for monitoring and early warning of abnormal vital signs of firefighters, which is adapted to the extreme environment of fire scenes and employs multimodal sensor fusion, environmental perception compensation and dynamic baseline prediction. It belongs to the interdisciplinary field of intelligent fire-fighting equipment and signal processing. Background Technology
[0002] Real-time monitoring of firefighters' vital signs, such as heart rate, respiration, and body temperature, is a core requirement for reducing casualties on fire scenes. Current technologies primarily rely on contact-based wearable devices, such as smart bracelets, chest straps, or fabric electrodes embedded in fire suits. These solutions have revealed significant shortcomings in real-world applications: contact sensors are prone to detachment, displacement, or poor electrical contact under high temperature, high humidity, and strenuous activity, increasing the physical burden on firefighters and reducing data reliability.
[0003] In recent years, radar-based non-contact vital sign detection technologies have emerged, extracting respiratory and heartbeat signals by emitting electromagnetic waves and receiving echoes from chest cavity micro-motions. However, directly applying general radar monitoring to fire scenes faces serious obstacles: dense smoke and high temperatures cause performance drift in the radio frequency front-end; vigorous movement of firefighters produces strong motion artifacts; single radar modes are difficult to accurately measure body temperature and are easily obstructed by posture, making reliable early warning difficult. Existing early warning systems mostly use fixed thresholds, ignoring individual differences and dynamic physiological loads, resulting in high false alarm and false negative rates. Therefore, there is an urgent need for an intelligent method that can withstand extreme fire conditions, reliably extract multi-parameter vital signs, and provide early warnings. Summary of the Invention
[0004] The technical problem to be solved by this invention is to provide a robust non-contact vital sign monitoring and early warning method for special conditions such as high temperature, dense smoke and strong motion interference in fire scenes. This method reduces the burden on firefighters while achieving high-precision measurement of heart rate, respiratory rate and body temperature, and issues graded alarms in advance through dynamic baseline and trend prediction.
[0005] To address the aforementioned problems, this invention proposes a non-contact method for monitoring and early warning of abnormal vital signs in firefighters, the core of which includes the following steps:
[0006] S1. Use a millimeter-wave radar integrated into the firefighter's head or shoulder to emit frequency-modulated continuous waves toward the chest cavity and receive the echoes, while simultaneously using an infrared thermal imaging module to collect images of the body surface temperature distribution.
[0007] S2. Process the echo signal to extract chest cavity micro-movements and estimate heart rate and respiratory rate; simultaneously segment human body regions from the body surface temperature image and extract representative body surface temperature features.
[0008] S3. Acquire environmental parameters, including at least temperature and smoke concentration, collected by environmental sensors, and motion parameters collected by the inertial measurement unit; perform environmental adaptive compensation on the echo or thoracic micromotion signal based on the environmental parameters, and perform adaptive motion artifact suppression on the thoracic micromotion signal based on the motion parameters to obtain a pure physiological signal;
[0009] S4. Input the heart rate and respiratory rate estimates obtained based on the pure physiological signals, the body surface temperature features, and the environmental and motion parameters into a cross-modal attention fusion network to generate a fused vital sign feature vector;
[0010] S5. Based on the fusion feature vector and exercise intensity, the physiological deviation is calculated through a pre-established individualized physiological baseline model and dynamic calibration; at the same time, the temporal feature vector is input into the Long Short-Term Memory (LSTM) network to predict the trend; when the deviation exceeds the limit or the predicted trend indicates that it will enter the danger zone within a predetermined time, a graded warning including the warning level, abnormality type and confidence level is output.
[0011] Accordingly, the present invention also provides a system comprising a multimodal front-end sensing module, an edge computing module, and a cloud analysis engine. The front-end sensing module integrates millimeter-wave radar, an infrared thermal imaging module, environmental sensors, and an inertial measurement unit (IMU), and is mounted on a firefighter's helmet or shoulder; the edge computing module performs real-time signal processing, feature extraction, and local early warning; the cloud engine runs a dynamic baseline model and a trend prediction network to generate comprehensive early warnings.
[0012] Beneficial effects
[0013] By fusing millimeter-wave radar with infrared thermal imaging in a multimodal manner, heart rate, respiration, and body surface temperature can be acquired simultaneously without contact, overcoming the limitations of single-modality imaging in fire situations. By introducing temperature and smoke compensation and IMU-based adaptive motion artifact suppression, the detection accuracy and robustness in extreme environments are greatly improved. By adopting a hierarchical early warning mechanism that combines individualized dynamic baselines with LSTM trend prediction, a leap from "post-event threshold alarm" to "pre-event trend prediction" is achieved, significantly improving the timeliness and accuracy of early warnings and reducing the false alarm rate. Attached Figure Description
[0014] Figure 1 This is an overall flowchart of the method of the present invention.
[0015] Figure 2 This diagram illustrates the system architecture and the installation of the front-end perception module.
[0016] Figure 3 This is a block diagram of signal processing for environmental adaptive compensation and motion artifact suppression.
[0017] Figure 4This is a schematic diagram of the dynamic baseline adaptive early warning model.
[0018] In the diagram: 10: Firefighter helmet, 100: Multimodal front-end sensing module, 110: Millimeter-wave radar module, 120: Infrared thermal imaging module, 130: Environmental sensor, 140: Inertial measurement unit, 200: Edge computing module, 300: Cloud analysis engine. Detailed Implementation
[0019] The present invention will now be described in further detail with reference to the accompanying drawings. It should be understood that the following specific embodiments are only for explaining the present invention and do not limit the scope of protection.
[0020] like Figure 1 As shown, the non-contact firefighter vital sign abnormality monitoring and early warning method of the present invention includes five core steps.
[0021] Step S1: Multimodal signal acquisition
[0022] like Figure 2 As shown, the multimodal front-end sensing module 100 is installed inside the firefighter's helmet 10 or at a shoulder mounting point, and includes a millimeter-wave radar module 110, an infrared thermal imaging module 120, an environmental sensor 130, and an inertial measurement unit 140. The millimeter-wave radar adopts a 60GHz FMCW system with a 4-transmit, 4-receive MIMO antenna array, continuously locking onto the firefighter's chest area through beamforming. The radar transmits a frequency-modulated continuous wave with a bandwidth of 4GHz, and the received echo is mixed with the transmitted wave to obtain an intermediate frequency (IF) signal. The infrared thermal imaging module selects the long-wave infrared band 8-14μm, acquiring a body surface temperature image with a resolution of 160×120 at a frame rate of 9fps. The environmental sensor includes a thermocouple temperature probe and an optical scattering smoke concentration sensor, and the inertial measurement unit 140 provides three-axis acceleration and three-axis angular velocity.
[0023] Step S2: Preliminary extraction of vital signs parameters
[0024] Within the edge computing module 200, range FFT and Doppler FFT are performed on the radar IF signal to separate the phase changes caused by thoracic cavity micro-movements. A bandpass filter is used to extract respiration at frequencies of 0.1-0.5 Hz and heart rate at frequencies of 0.8-2.5 Hz. Real-time respiratory rate (RR) and heart rate (HR) are estimated using spectral peak tracking and an adaptive notch filter, respectively. For infrared thermal images, lightweight networks such as YOLO are used to detect facial regions. Regions of interest (ROIs) are selected on the forehead and neck, and the average temperature of these regions is calculated as the body surface temperature (T_skin). Simultaneously, statistical features of temperature distribution within the ROIs are extracted.
[0025] Step S3: Environmental Perception Compensation and Motion Artifact Suppression
[0026] Figure 3The detailed process for this step is provided. The radar IF signal first passes through the environmental adaptive compensation module. The temperature compensator, based on the real-time ambient temperature T_env collected by the thermocouple, calls a pre-calibrated "temperature-RF front-end gain / phase deviation" lookup table (obtained through a high and low temperature test chamber) to perform amplitude scaling and phase rotation on the IF signal. The smoke attenuation compensator estimates the additional attenuation using the smoke concentration C_smoke, dynamically adjusting the radar transmit power or directly compensating for the IF amplitude to maintain the signal-to-noise ratio.
[0027] The motion artifact suppression module employs an adaptive filter based on a reference signal. The triaxial synthetic acceleration a(t) acquired by the inertial measurement unit 140 is used as the motion reference signal and, together with the environmentally compensated thoracic micromotion signal d(t), is fed into the least mean square (LMS) adaptive filter. The filter iteratively updates the weight coefficients, and the output error signal e(t) is the pure thoracic micromotion signal after removing motion interference. When the inertial measurement unit 140 detects intense motion exceeding 6 METs, the signal quality may severely degrade. In this case, the predictive hold function is activated, using the effective heart rate and respiratory rate values and their trends from the previous moment for short-term fill-in, and the measurement is quickly resumed after the motion subsides.
[0028] After this step, the clean HR_clean and RR_clean are recalculated.
[0029] Step S4: Cross-modal attention feature fusion
[0030] Heart rate, respiratory rate, and heart rate variability features are integrated into a radar feature vector F_radar, while mean body surface temperature and gradient features extracted from infrared images are integrated into an infrared feature vector F_ir. Both are input into a cross-modal attention fusion network. The network first embeds F_radar and F_ir into the same dimension via fully connected layers. Then, using radar features as the query and infrared features as the key and value, an attention weight matrix is calculated. An adjustment vector generated from the encoded environmental and motion parameters is additionally introduced as a bias term. This bias term is added element-wise to the cross-modal attention weight matrix, or multiplied element-wise using a gating unit, to adjust the attention distribution and increase the contribution of less affected modes. The final fused feature vector is F_fused. This design allows the network to automatically increase the weight of the infrared mode when the radar signal is significantly affected by smoke, and vice versa.
[0031] Step S5: Dynamic baseline prediction and graded early warning
[0032] Figure 4The early warning decision-making process was demonstrated. An individualized physiological baseline model was pre-established: upon joining the team, each firefighter underwent a power vehicle or treadmill test, collecting HR, RR, and T_skin data under different exercise loads such as resting, 3 MET, 6 MET, and 9 MET. A polynomial regression baseline function with exercise intensity as the independent variable was fitted. In actual operation, the edge computing module estimated the current metabolic equivalent MET_curr based on inertial measurement unit and radar motion detection. The dynamic calibration module output the reference heart rate HR_ref, reference respiration RR_ref, and reference body surface temperature T_ref corresponding to the current intensity from the baseline model. Their calculation formulas are HR_ref = f(MET), RR_ref = g(MET), and T_skin_ref = h(MET), respectively. The actual physiological state deviation Dev was calculated as: Dev = w1 × (HR_clean - HR_ref) + w2 × (RR_clean - RR_ref) + w3 × (T_skin - T_skinref).
[0033] Simultaneously, a 30-second sequence of F_fused vectors is input into a pre-trained LSTM trend prediction network. This network outputs predicted heart rate, respiratory rate, and body temperature for the next 15 and 30 seconds. A tiered early warning decision module integrates current deviation and future predicted trends: a yellow warning is issued if the predicted value will exceed personal safety limits within 20 seconds (e.g., HR > 180 bpm or T_skin > 39°C); an orange warning is issued if the limit will be exceeded within 10 seconds or the current deviation is already high; and a red warning is issued if the situation is critical or a coma-level abnormality is predicted. Warning information is transmitted back to the command terminal via the communication unit, carrying an anomaly type label and confidence score.
[0034] In terms of system architecture, the edge computing module 200 is an embedded processing unit carried by firefighters, equipped with a DSP and a lightweight GPU, executing steps S2-S4 and short-term local alarms. The cloud analysis engine 300 is a remote server cluster that runs a personalized baseline database, LSTM network, and situation display interface, receiving F_fused and MET data uploaded by multiple firefighters, completing step S5, and pushing tiered early warnings.
[0035] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications, equivalent substitutions, and improvements to the technical solutions of the present invention without departing from the principles and spirit of the invention, and all such modifications, equivalent substitutions, and improvements should be covered within the protection scope of the present invention.
Claims
1. A non-contact method for monitoring and early warning of abnormal vital signs in firefighters, characterized in that, Includes the following steps: S1: The millimeter-wave radar module (110) installed on the firefighter's helmet (10) transmits frequency-modulated continuous waves to the firefighter's chest cavity and receives the echoes, while the infrared thermal imaging module (120) collects body surface temperature images. S2: Extract thoracic cavity micromotion signals from the echo to estimate real-time heart rate and respiratory rate; extract human body surface temperature features from the body surface temperature image; S3: Acquire environmental parameters, including at least temperature and smoke concentration, collected by the environmental sensor (130), and motion parameters collected by the inertial measurement unit (140); perform environmental adaptive compensation on the echo or thoracic micromotion signal based on the environmental parameters, and perform adaptive motion artifact suppression on the thoracic micromotion signal based on the motion parameters to obtain a pure physiological signal; S4: Input the heart rate and respiratory rate estimates obtained based on the pure physiological signals, the body surface temperature features, and the environmental and motion parameters into the cross-modal attention fusion network to generate a fused vital sign feature vector; S5: Determine the current exercise intensity based on the fused vital sign feature vector and the exercise parameters; perform dynamic baseline calibration through a pre-established individualized physiological baseline model; calculate the deviation of physiological state; input the historical fused feature vector sequence into the time-series prediction model to obtain the predicted trend of vital signs at future moments; When the deviation exceeds a preset threshold or the predicted trend indicates that it will enter a dangerous zone within a predetermined time, a graded early warning message is output.
2. The method according to claim 1, characterized in that, The millimeter-wave radar module (110) operates in the 60GHz or 77GHz frequency band and uses a multi-input multi-output MIMO antenna array. Through beamforming technology, the radar beam is continuously directed towards the firefighter's chest area.
3. The method according to claim 1, characterized in that, The environmental adaptive compensation in step S3 includes: using a pre-calibrated RF front-end temperature characteristic curve to perform amplitude and phase compensation on the received radar echo based on the real-time temperature; compensating the echo amplitude based on the electromagnetic wave attenuation estimated by the smoke concentration; the adaptive motion artifact suppression includes: using the acceleration signal output by the inertial measurement unit (140) as a reference, using an adaptive filter to cancel noise in the chest cavity micro-motion signal, and enabling a prediction and maintenance mode based on historical physiological characteristics when the motion intensity exceeds a set threshold.
4. The method according to claim 1, characterized in that, The cross-modal attention fusion network in step S4 processes the following: heart rate and respiratory rate related features are encoded as first modal features, and body surface temperature related features are encoded as second modal features. Calculate the cross-modal attention weight between the first and second modalities, and use this weight to weightedly fuse the features of the two modalities. The attention weight is adjusted by the bias of the current environmental parameters and motion parameters to increase the contribution of the less disturbed modality. Specifically, this includes: encoding the environmental parameters and motion parameters into adjustment vectors, and adding the adjustment vectors as bias terms to the cross-modal attention weight matrix element by element, or multiplying them element by element through a gating unit, to adjust the attention distribution.
5. The method according to claim 1, characterized in that, The individualized physiological baseline model mentioned in step S5 is a multinomial regression model established by collecting heart rate and respiratory rate data under different exercise loads through standardized physical fitness tests when firefighters join the team; the dynamic baseline calibration queries the corresponding reference heart rate and reference respiratory rate from the baseline model according to the current exercise intensity; the baseline model not only outputs reference values, but also outputs the adaptive adjustment range of the reference values; the physiological state deviation is the weighted deviation between the actual measured value and the reference value.
6. The method according to claim 5, characterized in that, The time-series prediction model is a Long Short-Term Memory (LSTM) network. Its input is a fused vital sign feature vector of the past N time windows, and its output is the predicted values of heart rate, respiratory rate, and body temperature for the next M time windows. The predicted trend indicates that the area will enter the danger zone within a predetermined time period. This means that any predicted vital sign value within the next M time windows output by the LSTM network exceeds the safety upper limit threshold corresponding to that moment obtained by the dynamic baseline calibration. The graded warning includes at least three levels: yellow warning, orange warning, and red alert, and simultaneously outputs the anomaly type and confidence level evaluation.
7. A non-contact firefighter vital sign abnormality monitoring and early warning system, characterized in that, include: The firefighter helmet (10) and multimodal front-end sensing module (100) integrate a millimeter-wave radar module (110), an infrared thermal imaging module (120), an environmental sensor (130), and an inertial measurement unit (140). The millimeter-wave radar module (110) is used to transmit frequency-modulated continuous waves into the chest cavity and receive echoes. The infrared thermal imaging module (120) is used to acquire body surface temperature images. The environmental sensor (130) collects temperature and smoke concentration. The inertial measurement unit (140) collects motion parameters. The edge computing module (200) is connected to the multimodal front-end perception module (100) and is configured to perform echo signal processing, environmental adaptive compensation, motion artifact suppression, heart rate and respiratory rate estimation, body surface temperature extraction and cross-modal attention fusion, generate a fused vital sign feature vector, and implement a first-level early warning according to preset rules; The cloud-based analysis engine (300) is wirelessly connected to the edge computing module (200) to receive the fused vital sign feature vector and motion parameters, run dynamic baseline calibration and time series prediction models, generate graded early warning information and push it to the command terminal.
8. The system according to claim 7, characterized in that, The edge computing module (200) further includes: an environmental compensation unit for performing amplitude and phase compensation on radar echoes based on real-time temperature and smoke concentration; a motion interference suppression unit for eliminating motion artifacts in chest cavity micromotion signals using adaptive filters and inertial measurement unit (140) signals; a feature extraction unit for estimating heart rate and respiratory rate from the compensated and suppressed signals, and calculating body surface temperature from infrared thermal images; and a front-end early warning unit for issuing an alarm via local sound and light or vibration when vital signs parameters are detected to exceed safety thresholds.
9. The system according to claim 7, characterized in that, The cloud-based analysis engine (300) includes: an individualized baseline database that stores the baseline model parameters established by each firefighter through testing; a dynamic baseline calibration module that retrieves the corresponding reference physiological values from the database based on real-time exercise intensity; an LSTM trend prediction network that takes a fused feature vector sequence as input and outputs predicted values of physiological parameters for future moments; and a graded early warning decision module that combines the current physiological deviation and the predicted trend to generate graded early warnings and anomaly type determinations.
10. The system according to claim 7, characterized in that, The multimodal front-end sensing module (100) is installed on the inner lining of the firefighter's helmet (10) or on the shoulder equipment, and the beam direction of the millimeter-wave radar module (110) is adjusted to point towards the wearer's chest area; the environmental sensor (130) includes a contact or non-contact temperature sensor and an optical smoke sensor.