Multi-source data fusion non-contact respiration monitoring system and method

Abstract

According to the multi-source data fusion non-contact respiration monitoring system and method, high-precision, high-adaptability and real-time respiration frequency monitoring is achieved through core algorithms such as body position self-adaptive calibration, multi-source data fusion and an abnormal respiration recognition model. Millimeter-wave radar and infrared thermal imaging technologies are adopted, the respiratory rate can be monitored without direct contact with the human body, discomfort and interference caused by a traditional contact sensor are avoided, and the system is particularly suitable for intensive care, postoperative recovery, infectious wards and other scenes. Through a multi-source data fusion algorithm, the advantages of millimeter wave radar and infrared thermal imaging are combined, and the signal-to-noise ratio and monitoring precision of respiratory signals are improved. Experiments show that the signal-to-noise ratio after fusion is improved by 42%, and high accuracy of respiratory rate monitoring is ensured.

Description

Technical Field

[0001] This invention relates to the field of medical monitoring technology, specifically to a non-contact respiratory rate monitoring system and method based on the fusion of multi-source data from millimeter-wave radar and infrared thermal imaging. It can be applied to respiratory rate monitoring in clinical environments, such as intensive care units (ICUs), sleep apnea syndrome screening, and nurse rounds. Background Technology

[0002] Respiratory rate is an important indicator reflecting the physiological state of the human body. Traditional contact-based respiratory rate monitoring methods, such as chest strap sensors, suffer from problems such as discomfort and susceptibility to interference. Non-contact monitoring methods can avoid these problems, but existing non-contact methods still need improvement in terms of accuracy, adaptability, and real-time performance. Summary of the Invention

[0003] The purpose of this invention is to provide a non-contact respiratory rate monitoring system and method based on the fusion of multi-source data from millimeter-wave radar and infrared thermal imaging. Through core algorithms such as body position adaptive calibration, multi-source data fusion, and abnormal breathing identification model, it achieves high-precision, highly adaptable, and real-time respiratory rate monitoring.

[0004] This invention achieves non-contact respiratory rate monitoring through body position adaptive calibration, multi-source data fusion, and an abnormal breathing recognition model. First, infrared thermal imaging technology is used to acquire human body images. These images are then input into an improved ResNet-18 model to identify body positions, outputting recognition results for supine, lateral, and prone positions. The body position adaptive calibration algorithm dynamically adjusts the radar monitoring area based on the position recognition results, improving the integrity and accuracy of the acquired signals.

[0005] Based on the adjusted monitoring area, millimeter-wave radar signals are collected. The millimeter-wave radar signals and infrared thermal imaging signals are preprocessed to extract the chest displacement waveform and the temperature change in the nasal cavity. The radar signals and nasal cavity infrared thermal imaging signal features are fused by establishing semantic associations between the radar and thermal imaging features to generate high-precision respiratory waveforms.

[0006] Then, the abnormal breathing recognition model analyzes the fused waveform in real time, identifies abnormal breathing patterns, and triggers an early warning.

[0007] In the body position adaptive calibration algorithm, the first input is the acquired infrared image. An improved ResNet-18 model is used to identify the body position, outputting the identification results for three positions: supine, lateral, and prone. Then, the scanning angle of the radar beam is dynamically adjusted based on the body position identification results to ensure that the radar signal can accurately capture the human respiratory signal. Specifically, if the body position is identified as supine, the radar beam covers the central chest area (scanning angle ±5°); if the body position is identified as supine or lateral, the beam is adjusted to the back and chest (scanning angle ±15°), and beam deflection is achieved through a phased array antenna. Finally, the body position identification results are updated every 30 seconds, triggering radar parameter adjustments with a response time of less than 0.5 seconds. Using the improved model to identify body positions solves the signal attenuation problem of traditional non-contact monitoring when body position changes (e.g., the chest displacement signal is weakened by 50% when lying on one's side), ensuring full body position coverage.

[0008] In the multi-source data fusion algorithm, millimeter-wave radar signals and infrared thermal imaging signals are first preprocessed to extract the thoracic displacement waveform and temperature changes in the nasal cavity region. Then, the fusion weights are adjusted in real time based on the signal-to-noise ratio, prioritizing the mode with higher current signal quality. Finally, outliers are removed using the 3σ principle, retaining 99.7% of the valid data within the confidence interval.

[0009] The signal preprocessing includes: millimeter-wave radar signal: the thoracic displacement waveform is extracted using Fast Fourier Transform (FFT) with a frequency domain resolution of 0.1 Hz, and the respiratory frequency band is preserved by bandpass filtering (0.1-0.5 Hz).

[0010] Infrared thermal imaging signal: The nasal cavity region is located based on the Otsu threshold segmentation algorithm, and the temperature change during exhalation / inhalation is dynamically tracked by optical flow method (sensitivity ±0.05℃).

[0011] The radar signal and the nasal cavity infrared thermal imaging signal features are fused by establishing semantic associations between the radar and thermal imaging features to suppress redundant information between modes.

[0012] In the abnormal breathing recognition model, features are first extracted from the fused breathing signal, including time-domain and frequency-domain features. Then, a bidirectional LSTM network is used to recognize these features, outputting recognition results for three breathing modes: normal, Cheyne-Stokes, and apnea. Finally, the model is trained and optimized using cross-entropy loss and the Adam optimizer to improve its recognition accuracy. The LSTM network architecture includes: an input layer containing fused breathing signals at 32 time steps (sampling rate 50Hz), a hidden layer containing 64-node LSTM units with a dropout rate of 0.3 to prevent overfitting, and an output layer where Softmax identifies the three modes as normal, Cheyne-Stokes, or apnea.

[0013] The beneficial effects of this invention are as follows: 1. Non-contact monitoring: Using millimeter-wave radar and infrared thermal imaging technology, respiratory rate can be monitored without direct contact with the human body, avoiding the discomfort and interference caused by traditional contact sensors. It is particularly suitable for scenarios such as intensive care, postoperative recovery and infectious disease wards.

[0014] 2. High-precision monitoring: By employing a multi-source data fusion algorithm, combining the advantages of millimeter-wave radar and infrared thermal imaging, the signal-to-noise ratio and monitoring accuracy of respiratory signals are improved. Experiments show that the signal-to-noise ratio is improved by 42% after fusion, ensuring high accuracy in respiratory rate monitoring.

[0015] 3. Adaptive Position Calibration: An improved ResNet-18 model is used to identify patient positions in real time and dynamically adjust the radar beam scanning angle to ensure signal integrity under different positions, solving the signal attenuation problem of traditional non-contact monitoring when position changes. Improvements include: introducing the CBAM attention mechanism into the improved ResNet-18 model to adjust the processing order and structure of channel and spatial attention; using dilated convolutions or multi-scale convolutions to expand the receptive field; optimizing activation functions, such as replacing ReLU with Leaky ReLU to prevent neuronal death; adding global context modules (such as GCNet) and Local Importance Pooling (LIP) to enhance feature preservation; and data augmentation and regularization strategies to prevent overfitting.

[0016] 4. Abnormal breathing identification: A bidirectional LSTM network is used to extract and identify respiratory signals. It can accurately identify three breathing modes: normal, Cheyne-Stokes, and sleep apnea, with an F1-score of 96.7%, which significantly reduces the false alarm rate and provides reliable early warning information for medical staff.

[0017] 5. Real-time and dynamic feedback: The system can process and analyze respiratory signals in real time, trigger early warnings in a timely manner, and adjust system parameters in real time through the dynamic feedback module to ensure the real-time and effectiveness of monitoring. Attached Figure Description Figure 1 This is a schematic diagram of the overall process flow of the method of the present invention. Figure 2 This is an application scenario diagram of the present invention. Detailed Implementation The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0018] This invention achieves non-contact respiratory rate monitoring through body position adaptive calibration, multi-source data fusion, and an abnormal breathing recognition model. First, infrared thermal imaging technology is used to acquire human body images. These images are then input into an improved ResNet-18 model to identify body positions, outputting recognition results for supine, lateral, and prone positions. The body position adaptive calibration algorithm dynamically adjusts the radar monitoring area based on the position recognition results, improving the integrity and accuracy of the acquired signals.

[0019] Based on the adjusted monitoring area, millimeter-wave radar signals are collected. The millimeter-wave radar signals and infrared thermal imaging signals are preprocessed to extract the chest displacement waveform and the temperature change in the nasal cavity. The radar signals and nasal cavity infrared thermal imaging signal features are fused by establishing semantic associations between the radar and thermal imaging features to generate high-precision respiratory waveforms.

[0020] Then, the abnormal breathing recognition model analyzes the fused waveform in real time, identifies abnormal breathing patterns, and triggers an early warning.

[0021] In the body position adaptive calibration algorithm, the first input is the acquired infrared image. An improved ResNet-18 model is used to identify the body position, outputting the identification results for three positions: supine, lateral, and prone. Then, the scanning angle of the radar beam is dynamically adjusted based on the body position identification results to ensure that the radar signal can accurately capture the human respiratory signal. Specifically, if the body position is identified as supine, the radar beam covers the central chest area (scanning angle ±5°); if the body position is identified as supine or lateral, the beam is adjusted to the back and chest (scanning angle ±15°), and beam deflection is achieved through a phased array antenna. Finally, the body position identification results are updated every 30 seconds, triggering radar parameter adjustments with a response time of less than 0.5 seconds. Using the improved model to identify body positions solves the signal attenuation problem of traditional non-contact monitoring when body position changes (e.g., the chest displacement signal is weakened by 50% when lying on one's side), ensuring full body position coverage.

[0022] In the multi-source data fusion algorithm, millimeter-wave radar signals and infrared thermal imaging signals are first preprocessed to extract the thoracic displacement waveform and temperature changes in the nasal cavity region. Then, the fusion weights are adjusted in real time based on the signal-to-noise ratio, prioritizing the mode with higher current signal quality. Finally, outliers are removed using the 3σ principle, retaining 99.7% of the valid data within the confidence interval.

[0023] The signal preprocessing includes: millimeter-wave radar signal: the thoracic displacement waveform is extracted using Fast Fourier Transform (FFT) with a frequency domain resolution of 0.1 Hz, and the respiratory frequency band is preserved by bandpass filtering (0.1-0.5 Hz).

[0024] Infrared thermal imaging signal: The nasal cavity region is located based on the Otsu threshold segmentation algorithm, and the temperature change during exhalation / inhalation is dynamically tracked by optical flow method (sensitivity ±0.05℃).

[0025] The radar signal and the nasal cavity infrared thermal imaging signal features are fused by establishing semantic associations between the radar and thermal imaging features to suppress redundant information between modes.

[0026] In the abnormal breathing recognition model, features are first extracted from the fused breathing signal, including time-domain and frequency-domain features. Then, a bidirectional LSTM network is used to recognize these features, outputting recognition results for three breathing modes: normal, Cheyne-Stokes, and apnea. Finally, the model is trained and optimized using cross-entropy loss and the Adam optimizer to improve its recognition accuracy. The LSTM network architecture includes: an input layer containing fused breathing signals at 32 time steps (sampling rate 50Hz), a hidden layer containing 64-node LSTM units with a dropout rate of 0.3 to prevent overfitting, and an output layer where Softmax identifies the three modes as normal, Cheyne-Stokes, or apnea.

[0027] Furthermore, the improved ResNet-18 model employs the following mechanism: Enhanced Attention Mechanism: Improved CBAM Module: The channel attention layer of the original CBAM (Convolutional Block Attention Module) was changed from "dimensionality reduction followed by dimensionality increase" to "dimensionality increase followed by dimensionality reduction." The MLP dimension was adjusted from C / r → C to C → C / r → C, amplifying key feature details such as nasal temperature distribution. Channel attention (CAM) and spatial attention (SAM) were changed from cascaded to parallel, avoiding mutual interference during computation and improving the ability to focus on key body position areas (such as the back, chest, and nasal airflow).

[0028] Global Context Fusion: GCNet (Global Context Network) is introduced, and a global context module is added to the end of the residual block. The channel weights are dynamically adjusted through a compression-excitation mechanism to enhance the perception of the overall thermal distribution of body position.

[0029] Residual block structure optimization: Dilated convolution expands the receptive field: In the first residual block, dilated convolution is used instead of standard 3×3 convolution to expand the receptive field to 7×7 pixels, capturing the subtle displacement features of the back and thorax when lying on one's side.

[0030] Multi-scale feature extraction: Drawing inspiration from the Inception structure, 3×3 and 5×5 convolution kernels are used in parallel in the second residual block to fuse postural features at different scales (such as overall chest movement in supine position and local muscle tremors in prone position).

[0031] Activation functions and regularization: Leaky ReLU replaces ReLU: It solves the gradient vanishing problem of ReLU in the negative region and improves the adaptability to low-contrast thermal imaging data (such as slight temperature changes in the occluded area of ​​the face when prone).

[0032] Local Importance Pooling (LIP): A LIP module is added before the pooling layer. The feature map is weighted by learnable weights to preserve detailed information of key regions of body position (such as nasal airflow dynamics).

[0033] Multi-source data feature fusion: Infrared thermal imaging features (channel attention weighting) and radar displacement features (spatial attention weighting) are fused through a cross-modal stitching layer, and then input into a fully connected layer. Dynamic weight feedback: If the LSTM anomaly detection module detects a sleep apnea event, it automatically increases the infrared sensor sampling rate to 30Hz and triggers CBAM channel weight recalibration. The final output feature map is weighted according to the channel weights: F out =F M c , in To perform channel-by-channel multiplication, important channels (such as respiratory rhythm characteristics) are strengthened, while irrelevant channels (such as noise from clothing friction) are weakened.

[0034] M c =σ(MLP(AvgPool(F))+MLP(MaxPool(F))) Where, σ( Let F be the activation function, and F be the original feature map, where F ∈ R. H×W×C H: Feature map height, representing the vertical spatial dimension of the feature map; W: Feature map width, representing the horizontal spatial dimension of the feature map; C: Number of channels, representing the depth of the feature map; AvgPool(F), representing the global average response intensity of each channel; MaxPool(F), representing the local salient feature intensity of each channel; MLP( ) is the weight vector for the nonlinear relationship between the output encoding channels of the multilayer perceptron.

[0035] Through M c By suppressing channels affected by environmental interference, the respiratory rate detection error is stabilized within ±0.3 breaths / minute. This invention dynamically allocates channel weights through a three-step process: dual-channel pooling (global average + maximum) → compressed-expanded MLP → Sigmoid normalization, thus solving the feature attenuation problem of traditional respiratory monitoring methods under changes in body position and environmental interference.

[0036] In the multi-source data fusion algorithm, millimeter-wave radar signals and infrared thermal imaging signals are first preprocessed to extract the thoracic displacement waveform and temperature changes in the nasal cavity region. Then, the fusion weights are adjusted in real time based on the signal-to-noise ratio, prioritizing the mode with higher current signal quality. Finally, outliers are removed using the 3σ principle, retaining 99.7% of the valid data within the confidence interval.

[0037] The signal preprocessing includes: millimeter-wave radar signal: the thoracic displacement waveform is extracted using Fast Fourier Transform (FFT) with a frequency domain resolution of 0.1 Hz, and the respiratory frequency band is preserved by bandpass filtering (0.1-0.5 Hz).

[0038] Infrared thermal imaging signal: The nasal cavity region is located based on the Otsu threshold segmentation algorithm, and the temperature change during exhalation / inhalation is dynamically tracked by optical flow method (sensitivity ±0.05℃).

[0039] The radar signal and the nasal cavity infrared thermal imaging signal features are fused by establishing semantic associations between the radar and thermal imaging features to suppress redundant information between modes.

[0040] Feature fusion is performed in the following manner:

[0041] Among them, A cross For the characteristics after fusion, W q To query the projection matrix, W k W is the key projection matrix. v Let F be the value projection matrix, d: feature dimension scaling factor, and F. radar : Millimeter-wave radar characteristic matrix, F cam : Thermal imaging feature matrix. For example, when radar detects the frequency f of the thoracic cavity micromotion. radar At 1.2Hz, the fused feature A cross This will enhance the optical flow characteristics of the facial region in the camera (corresponding to heart rate f). cam =1.15Hz), suppressing background interference.

[0042] In the abnormal breathing recognition model, features are first extracted from the fused breathing signal, including time-domain and frequency-domain features. Then, a bidirectional LSTM network is used to recognize these features, outputting recognition results for three breathing modes: normal, Cheyne-Stokes, and apnea. Finally, the model is trained and optimized using cross-entropy loss and the Adam optimizer to improve its recognition accuracy. The LSTM network architecture includes: an input layer containing fused breathing signals at 32 time steps (sampling rate 50Hz), a hidden layer containing 64-node LSTM units with a dropout rate of 0.3 to prevent overfitting, and an output layer where Softmax identifies the three modes as normal, Cheyne-Stokes, or apnea.

[0043] The specific application scenarios of this invention are as follows: Example 1: Nighttime ICU rounds The non-contact respiratory rate monitoring system of this invention has been effectively applied in nighttime ICU rounds. In ICU wards, patients are usually in a critical condition, and medical staff need to monitor their vital signs, including respiratory rate, in real time. Traditional contact monitoring devices, such as chest strap sensors, may cause discomfort to patients and affect their rest, while non-contact monitoring methods can avoid these problems.

[0044] The system employs a multi-source data fusion module to preprocess and fuse millimeter-wave radar signals and infrared thermal imaging signals. The signal preprocessing unit first performs a Fast Fourier Transform (FFT) on the millimeter-wave radar signal to extract the chest displacement waveform with a frequency domain resolution of 0.1 Hz, and then uses bandpass filtering (0.1-0.5 Hz) to preserve the respiratory frequency band. Simultaneously, the infrared thermal imaging signal is processed using the Otsu threshold segmentation algorithm to locate the nasal cavity region, combined with optical flow to dynamically track temperature changes during exhalation / inhalation with a sensitivity of ±0.05℃. The dynamic weight allocation unit adjusts the fusion weights in real time based on the signal-to-noise ratio, prioritizing modes with higher current signal quality. The anomaly filtering unit uses the 3σ principle to remove outliers, retaining 99.7% of the valid data within the confidence interval.

[0045] The adaptive positioning calibration module uses an improved ResNet-18 model to identify the patient's position, outputting recognition results for three positions: supine, lateral, and prone. Based on the position recognition results, the radar beam dynamic adjustment unit dynamically adjusts the scanning angle of the radar beam to ensure that the radar signal can accurately capture the human's respiratory signals. For example, if the patient is identified as supine, the radar beam covers the central chest area (scanning angle ±5°); if the patient is identified as lateral, the beam is adjusted to the back and chest (scanning angle ±15°), achieving beam deflection through the phase array antenna. The real-time calibration unit updates the position recognition results every 30 seconds, triggering radar parameter adjustments with a response time of less than 0.5 seconds.

[0046] The abnormal breathing recognition module extracts features from the fused breathing signal, including time-domain and frequency-domain features. Time-domain features include the respiratory cycle and coefficient of variation (CV), while frequency-domain features include the dominant frequency energy percentage (calculated via FFT). A bidirectional LSTM network is used to recognize these features, outputting recognition results for three breathing modes: normal, Cheyne-Stokes, and apnea. The model is trained and optimized using cross-entropy loss and the Adam optimizer to improve its recognition accuracy.

[0047] During night rounds, nurses wear smart glasses integrated with this system to check patients' respiratory rates without having to lift their blankets. When a patient's respiratory rate suddenly rises to 28 breaths per minute, an orange warning light illuminates at the edge of the glasses, triggering a vibration alert. The nurse immediately checks the monitor and finds that the blood oxygen saturation has dropped to 90%, promptly initiating oxygen intervention. The system's application in the ICU not only improves the efficiency of medical staff but also ensures patient safety, avoiding the discomfort and interference associated with traditional monitoring equipment.

[0048] Example 2: Postoperative recovery monitoring The monitoring system of this invention also plays a crucial role in postoperative recovery monitoring. During the recovery phase from general anesthesia, the breathing pattern of patients may be affected by the medication, resulting in shallow, slow, or abnormal breathing rates. Medical staff need to closely monitor the patient's respiratory status to promptly detect and manage any potential complications such as respiratory depression.

[0049] When acquiring respiratory signals, the system utilizes millimeter-wave radar and infrared thermal imaging technology in tandem to ensure signal accuracy and integrity. The signal preprocessing unit processes the signals from both modalities separately, extracting useful respiratory-related information. The dynamic weight allocation unit adjusts the fusion weights of the millimeter-wave radar and infrared thermal imaging signals based on the real-time calculated signal-to-noise ratio to achieve optimal signal quality. For example, in situations involving changes in patient position or significant environmental interference, the system can automatically adjust the weights, prioritizing the modality less affected by interference.

[0050] The adaptive positioning calibration module monitors the patient's positional changes in real time and adjusts the radar beam scanning angle accordingly. In the postoperative recovery room, patients may frequently change positions, from supine to lateral, and then to semi-recumbent. The system can quickly identify these positional changes and dynamically adjust the radar monitoring area to ensure stable acquisition of respiratory signals. The positional recognition results are updated every 30 seconds, triggering radar parameter adjustments to ensure real-time monitoring of the patient's respiratory status.

[0051] The abnormal breathing recognition module performs in-depth analysis of the fused respiratory signals to identify different breathing patterns. During postoperative recovery monitoring, the system pays particular attention to abnormal patterns such as shallow and slow breathing and apnea. When shallow and slow breathing (6 breaths / min) accompanied by fluctuations in blood oxygen saturation is detected, the system extracts relevant features through a feature extraction unit and uses an LSTM network for identification. The LSTM network can capture the temporal characteristics of respiratory signals, accurately identify abnormal breathing patterns, and trigger timely warnings. Based on the system's prompts, medical staff can adjust the analgesia regimen promptly, preventing the patient's respiratory depression caused by opioid suppression from worsening.

[0052] Example 3: Monitoring in an Infectious Disease Ward In infectious disease ward monitoring scenarios, the non-contact respiratory rate monitoring system of this invention demonstrates unique advantages. Patients in infectious disease wards require strict isolation, and medical staff must ensure data accuracy while avoiding the risk of cross-infection when monitoring vital signs. Traditional contact monitoring equipment may increase the risk of infection during use, while non-contact monitoring methods can effectively solve this problem.

[0053] The system collects patients' respiratory signals using millimeter-wave radar and infrared thermal imaging technology, eliminating the need for direct contact. Nurses can obtain patients' vital signs data in real time from outside the ward by wearing smart glasses integrated with the system. The data is automatically entered into electronic medical records, reducing manual recording work for medical staff and improving work efficiency.

[0054] In infectious disease wards, patients may frequently change positions due to changes in their condition. The adaptive positioning calibration module uses an improved ResNet-18 model to identify patient positions and dynamically adjust the scanning angle of the radar beam. For example, when a patient changes from a supine to a lateral decubitus position, the system can quickly identify and adjust the radar beam to cover the back and chest area, ensuring accurate acquisition of respiratory signals. The positioning recognition results are updated every 30 seconds, triggering radar parameter adjustments to ensure the system's monitoring effectiveness under different positions.

[0055] The abnormal breathing recognition module analyzes the fused respiratory signals in real time to identify abnormal breathing patterns. In infectious disease wards, patients may experience abnormally high respiratory rates or difficulty breathing due to infection. The system can detect these abnormalities promptly and trigger alerts. Medical staff can then take appropriate treatment measures based on the system's prompts, improving the success rate of patient treatment.

[0056] Furthermore, the system's ultraviolet disinfection chamber on the temples automatically sterilizes three times daily, completely eliminating the risk of cross-infection. The contactless design and automatic disinfection function of the smart glasses not only protect the safety of medical staff but also provide a safer environment for patient treatment.

[0057] In summary, the respiratory rate monitoring system and method based on multi-source data fusion of this invention have been effectively applied in various clinical scenarios, including nighttime ICU rounds, postoperative recovery monitoring, and infectious disease ward care. Through core technologies such as adaptive body position calibration, multi-source data fusion, and abnormal breathing identification, the system achieves high-precision, highly adaptable, and real-time respiratory rate monitoring, providing medical staff with a reliable auxiliary tool and ensuring patient safety.

Claims

1. A non-contact respiratory monitoring method based on multi-source data fusion, characterized in that, include: Human infrared images are acquired to obtain nasal cavity region temperature imaging and body position imaging. An improved ResNet-18 model is used to identify human body position, outputting the identification results for three positions: supine, lateral, and prone. A body position adaptive calibration algorithm is adopted. The millimeter-wave radar monitoring area is dynamically adjusted according to the body position recognition results to ensure signal integrity and accuracy. The millimeter-wave radar is used to monitor changes in the patient's chest position. The millimeter-wave radar signal was acquired from the adjusted monitoring area. The millimeter-wave radar signal and the nasal cavity infrared thermal imaging signal were preprocessed to extract the thoracic displacement waveform and the temperature change of the nasal cavity area. The radar signal and the nasal cavity infrared thermal imaging signal features are fused by establishing semantic association between the radar and thermal imaging features to generate a high-precision breathing waveform. The fusion weights are adjusted in real time based on the signal-to-noise ratio, prioritizing the mode with higher current signal quality. The fused waveform is analyzed in real time to identify abnormal breathing patterns and trigger early warnings.

2. The method according to claim 1, characterized in that, The preprocessing of the millimeter-wave radar signal includes: extracting the thoracic displacement waveform using Fast Fourier Transform (FFT) with a frequency domain resolution of 0.1 Hz; and preserving the respiratory frequency band by bandpass filtering (0.1-0.5 Hz).

3. The method according to claim 1, characterized in that, The preprocessing of the infrared thermal imaging signal includes: locating the nasal cavity region based on the Otsu threshold segmentation algorithm; and dynamically tracking temperature changes during exhalation / inhalation using optical flow method, with a sensitivity of ±0.05℃.

4. The method according to claim 1, characterized in that, The fusion of radar signal and infrared thermal imaging signal features, achieved by establishing semantic associations between radar and thermal imaging features, employs the following method: Feature fusion is performed in the following manner: Among them, A cross For the characteristics after fusion, W q To query the projection matrix, W k W is the key projection matrix. v Let F be the value projection matrix, d: feature dimension scaling factor, and F. radar : Millimeter-wave radar feature matrix, F cam : Thermal imaging feature matrix.

5. The method according to claim 1, characterized in that, The step of dynamically adjusting the radar monitoring area based on the body position recognition results to ensure signal integrity and accuracy includes: The improved ResNet-18 model is used to identify body positions, and the recognition results of three body positions are output: supine, lateral, and prone. The scanning angle of the radar beam is dynamically adjusted based on the body position recognition results. When lying supine, the radar beam covers the central area of ​​the chest with a scanning angle of ±5°. When lying on one's side, the beam is adjusted to the back with a scanning angle of ±15°. The beam deflection is achieved through a phase array antenna. The body position recognition results are updated every 30 seconds, triggering the adjustment of radar parameters.

6. The method according to claim 5, characterized in that, The method of recognizing body positions using the improved ResNet-18 model includes: The model's ability to focus on key feature regions is enhanced through attention mechanisms. These enhancements include: improving the CBAM module: changing the channel attention layer of the original CBAM (Convolutional Block Attention Module) from "dimensionality reduction followed by dimensionality increase" to "dimensionality increase followed by dimensionality reduction," adjusting the MLP dimension from C / r → C to C → C / r → C, amplifying key feature details of nasal temperature distribution, and changing the cascaded and parallel operation of channel attention (CAM) and spatial attention (SAM) to avoid mutual interference during computation, thus improving the ability to focus on key body position regions. A global context fusion mechanism is introduced to enhance the perception of the overall thermal distribution of body position. Global context fusion includes the introduction of GCNet, adding a global context module at the end of the residual block, and dynamically adjusting the channel weights through a compression-excitation mechanism to enhance the perception of the overall thermal distribution of body position. Optimize the residual block structure to expand the receptive field and extract multi-scale positional features.

7. The method according to claim 6, characterized in that, The dynamic adjustment of channel weights is as follows: If the LSTM anomaly detection module detects a sleep apnea event, it automatically increases the infrared sensor sampling rate to 30Hz and triggers CBAM channel weight recalibration. The final output feature map is weighted according to the channel weights: F out =F M c , in To perform channel-by-channel multiplication, important channels are emphasized, including respiratory rhythm characteristics, while irrelevant channels, including clothing friction noise, are weakened. c It is the channel weight, M c =σ(MLP(AvgPool(F))+MLP(MaxPool(F))),σ( Let F be the activation function, and F be the original feature map, where F ∈ R. H×W×C H: Feature map height, representing the vertical spatial dimension of the feature map; W: Feature map width, representing the horizontal spatial dimension of the feature map; C: Number of channels, representing the depth of the feature map; AvgPool(F), representing the global average response intensity of each channel; MaxPool(F), representing the local salient feature intensity of each channel; MLP( ) is the weight vector for the nonlinear relationship between the output encoding channels of the multilayer perceptron.

8. The method according to claim 5, characterized in that, The dynamic adjustment of the radar beam scanning angle based on the body position recognition result includes: If the patient is identified as lying supine, the radar beam will cover the central area of ​​the chest, with a scanning angle of ±5°. If the patient is identified as being in a lateral decubitus position, the beam is adjusted to the back and chest, with a scanning angle of ±15°. Beam deflection is achieved using a phased array antenna.

9. The method according to claim 1, characterized in that, The real-time analysis of the fused waveform to identify abnormal breathing patterns and trigger early warnings includes: Extract the time-domain and frequency-domain features of the fused respiratory signal; A bidirectional LSTM network is used to identify features and output the identification results of three breathing modes: normal, Cheyne-Stokes, and sleep apnea. An alert is triggered based on the identification results.

10. A non-contact respiratory monitoring system based on multi-source data fusion, used to implement the method according to any one of claims 1-9, characterized in that, include: The infrared image acquisition module is used to acquire infrared images of the human body, obtaining temperature imaging and positional imaging of the nasal cavity area. The body position recognition module is used to recognize human body positions using an improved ResNet-18 model, and outputs the recognition results for three body positions: supine, lateral, and prone. The millimeter-wave radar monitoring module is used to dynamically adjust the monitoring area of ​​the millimeter-wave radar based on the body position recognition results, ensuring the integrity and accuracy of the signal; the millimeter-wave radar is used to monitor changes in the patient's chest position. The multi-source data fusion module is used to preprocess and fuse millimeter-wave radar signals and nasal infrared thermal imaging signals to generate high-precision respiratory waveforms. The body position adaptive calibration module is used to dynamically adjust the radar monitoring area based on the body position recognition results to ensure signal integrity; The abnormal breathing recognition module is used to analyze the fused waveform in real time, identify abnormal breathing patterns, and trigger an early warning.

Images