Obstructive sleep apnea syndrome detection method, device and program product based on multi-modal data

Abstract

The invention relates to the field of intelligent medical treatment, in particular to an obstructive sleep apnea syndrome detection method and device based on multi-modal data and a program product. Comprising the following steps: S1, acquiring time sequence frame data of infrared thermal imaging of a tester in real time; s2, extracting chest and abdomen position data and air temperature data of a nose and mouth area based on the infrared thermal imaging time sequence frame data; s3, calculating to obtain breathing waveform data through thoracic and abdominal position data in the time sequence frame data; s4, air temperature change curve data are obtained through air temperature data calculation in the time sequence frame data; s5, performing phase correlation on the breathing waveform data and the airflow temperature change curve data to calculate a breathing ratio; comparing the respiration ratio with a preset threshold value, and judging that the obstructive sleep apnea syndrome occurs when the respiration ratio is greater than the preset threshold value. The method can effectively identify the obstructive sleep apnea syndrome, and has a good clinical value.

Description

Technical Field

[0001] The present application relates to the field of intelligent medical care, and specifically to a method, device, program product, and computer-readable storage medium for detecting obstructive sleep apnea syndrome based on multimodal data. Background Art

[0002] Obstructive sleep apnea (OSA) is a common sleep-related breathing disorder with a high prevalence and potentially fatal complications. The pathogenesis of OSA is a clinical syndrome characterized by chronic intermittent hypoxia and sleep fragmentation, accompanied by a series of physiological changes such as hypoxemia and hypercapnia. This syndrome can easily cause systemic damage to multiple systems, leading to multi-organ and multi-system complications such as coronary artery disease, hypertension, and diabetes. Diagnostic technology for OSA relies on traditional polysomnography (PSG). PSG requires the attachment of multiple electrodes and sensors to the body surface, making it complex, uncomfortable, and reliant on a specialized medical environment, making it difficult to meet the needs of home and general medical settings. Single-sensor devices (such as oximeters and respiratory belts) rely solely on a single signal, such as blood oxygen or respiratory airflow, which is susceptible to interference, has a high rate of missed detection, and cannot distinguish between OSA and central apnea (CSA), leading to the risk of misdiagnosis. Summary of the Invention

[0003] While existing wearable devices on the market (such as wrist-worn oximeters) offer improved portability, they are limited to single blood oxygen data and lack the ability to correlate analysis with respiration, body position, and other factors. They are unable to dynamically capture OSA-specific micro-arousal signals through pulse waveforms to provide precise intervention recommendations. Publicly available technologies for respiratory monitoring based on infrared thermal imaging do not combine pulse signals with body position analysis, making them unable to identify paradoxical respiratory characteristics of OSA. They also do not analyze the temporal correlation between respiratory airflow interruptions and pulse signals, resulting in a high misjudgment rate and difficulty meeting the needs of clinical precision monitoring.

[0004] To address the above issues, the present invention provides a method for detecting obstructive sleep apnea syndrome based on multimodal data, which specifically includes: S1. Real-time acquisition of time-series frame data of infrared thermal imaging of the tester; S2. Extracting chest and abdomen position data and air temperature data of the nose and mouth area based on the infrared thermal imaging time series frame data; S3, obtaining respiratory waveform data by calculating the chest and abdomen position data in the time series frame data; S4, obtaining temperature change curve data by calculating the temperature data in the time series frame data; S5, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to calculate the respiratory ratio; The respiratory ratio is compared with a preset threshold, and when the respiratory ratio is greater than the preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0005] The chest and abdomen position data are calculated by extracting the displacement of the isotherms in the time-series chest and abdomen thermal map, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data. The calculation process of the breathing ratio includes: The phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data is performed to obtain the relevant respiratory data. The relevant respiratory data is the change of respiratory displacement and airflow temperature in the same time period. Based on the airflow temperature change curve data, the temperature peak and valley values are identified, and the inhalation and exhalation time intervals are divided based on the temperature peak and valley values; Based on the correlation respiratory data, the respiratory displacement change in the inhalation time interval and the respiratory displacement change in the exhalation time interval are obtained, and respiratory verification is performed to obtain the verification respiratory time interval and the inhalation time interval; The temperature peak duration and trough duration are obtained based on the verified breathing time interval and the inspiratory time interval; The respiratory ratio was calculated based on the duration of the temperature peak and the duration of the trough; Optionally, the breathing verification is that the breathing displacement increases during the inhalation time interval and decreases during the exhalation time interval. The method further includes a pulsation feature, and determining whether obstructive sleep apnea syndrome occurs by using the pulsation feature and the respiratory ratio; the pulsation feature includes one or more of the following: heart rate, blood oxygen saturation, and pulse wave amplitude; Optionally, whether obstructive sleep apnea syndrome occurs is determined by heart rate and respiratory ratio, and the heart rate values in a continuous time are obtained. In a time axis, when the respiratory ratio is greater than a preset threshold and the heart rate increase is greater than the preset heart rate threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by blood oxygen saturation and respiratory ratio, blood oxygen saturation is obtained over a continuous period of time, and when, on a time axis, the respiratory ratio is greater than a preset threshold and the amplitude of the decrease in blood oxygen saturation is greater than a preset threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by pulse wave amplitude and respiratory ratio, the pulse wave transmission time and pulse waveform are obtained in a continuous time, and the pulse wave amplitude change is calculated by the pulse wave transmission time and pulse waveform. When the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than the pulse preset threshold on a time axis, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by heart rate, blood oxygen saturation and respiratory ratio, blood oxygen saturation and heart rate are obtained over a continuous period of time, and when, on a time axis, the respiratory ratio is greater than a preset threshold, the heart rate increase is greater than a preset heart rate threshold, and the blood oxygen saturation decrease is greater than a preset blood oxygen threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, the obstructive sleep apnea syndrome also includes dynamic allocation of body posture weights, and the tester's contour is extracted from the infrared imaging time-series frame data to obtain a thermal imaging contour; based on the thermal imaging contour classification, a classification result of supine or side-lying is obtained, and when the sleeping position is supine, the weight of the decrease in blood oxygen saturation is increased; when the sleeping position is side-lying, the weight of the chest and abdomen position data is increased. The method also includes grading obstructive sleep apnea syndrome, which is graded into mild, moderate, and severe levels by using the blood oxygen saturation decrease slope and the apnea-hypopnea index; Optionally, the blood oxygen saturation decrease slope is obtained by calculating the change in blood oxygen saturation over a continuous period of time; Optionally, the apnea-hypopnea index is calculated based on airflow temperature change curve data and blood oxygen saturation decrease amplitude; Optionally, the apnea-hypopnea index calculation process is: obtaining the total sleep time; extracting the number of apneas through the airflow temperature change curve data; obtaining the number of hypoventilation through the airflow temperature change curve data and the amplitude of the decrease in blood oxygen saturation; and calculating the apnea-hypopnea index through the total sleep time, the number of apneas, and the number of hypoventilation. S5 is replaced with: determining whether obstructive sleep apnea syndrome occurs based on the respiratory waveform data and the airflow temperature change curve data, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to obtain relevant respiratory data, and determining that obstructive sleep apnea syndrome occurs when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen perform paradoxical movements; Optionally, the judgment further includes one or more of the following: heart rate, blood oxygen saturation; Optionally, the judgment is performed using respiratory waveform data, airflow temperature change curve data, heart rate, and blood oxygen saturation. The heart rate increase amplitude is calculated for the heart rate over a continuous period of time, and the blood oxygen saturation decrease amplitude is calculated for the blood oxygen saturation over a continuous period of time. When the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold, the chest and abdomen perform contradictory movements, the heart rate increase amplitude is greater than a preset heart rate threshold, and the blood oxygen saturation decrease amplitude is greater than a preset blood oxygen threshold, it is determined that obstructive sleep apnea syndrome has occurred. Optionally, the judgment is based on the respiratory airflow, chest and abdominal movements, heart rate increase, and blood oxygen decrease within the same time period; Optionally, the judgment also includes dynamic weight allocation of body position, extracting thermal imaging contours to determine sleeping position, when the sleeping position is lying on the back, increasing the weight of the decrease in blood oxygen saturation; when the sleeping position is lying on the side, increasing the weight of the chest and abdomen position data. The method further includes CSA differentiation, where CSA is determined to have occurred when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen stop moving during the interruption duration. An object of the present invention is to provide a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the above-mentioned obstructive sleep apnea syndrome detection method based on multimodal data.

[0006] The present invention provides a computer program product, comprising: Infrared thermal imaging module, used to obtain real-time infrared thermal imaging data; Pulse sensor module, used to obtain pulse data; The data processing module is used to process real-time infrared thermal imaging data and / or pulse data; infrared thermal imaging processing includes: Extracting chest and abdomen position data and air temperature data of the nose and mouth area based on the infrared thermal imaging time series frame data; Respiratory waveform data is obtained by calculating the chest and abdomen position data in the time series frame data; The temperature change curve data is obtained by calculating the temperature data in the time series frame data; Calculating the respiratory ratio by performing phase correlation on the respiratory waveform data and the airflow temperature change curve data; Pulse data processing includes: obtaining pulse data in a continuous time, and calculating the change amplitude or change slope of the pulse data; A determination module, which determines whether obstructive sleep apnea syndrome occurs by receiving data from the data processing module; when the breathing ratio is greater than a preset threshold, it is determined that obstructive sleep apnea syndrome occurs; Optionally, the pulse data obtained includes one or more of the following: heart rate, blood oxygen saturation.

[0007] An object of the present invention is to provide a computer device comprising a memory, a processor, and a computer program or instructions stored on the memory, wherein the computer program or instructions are executed by the processor to implement the above-mentioned obstructive sleep apnea syndrome detection method based on multimodal data.

[0008] An object of the present invention is to provide a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the above-mentioned obstructive sleep apnea syndrome detection method based on multimodal data.

[0009] Advantages of the present invention: 1. The present invention uses infrared thermal imaging time series data (real time) to determine whether a test subject has obstructive sleep apnea syndrome. Specifically, it uses respiratory waveform data and air temperature change data to perform phase correlation calculations to verify respiratory movement. The respiratory ratio is calculated based on the verified respiratory data, and then OSA is identified based on the respiratory ratio data. By verifying lung and abdominal movement and airflow changes in the nose and mouth area on the same timeline, motion artifact interference is eliminated, time segmentation consistency is ensured, and the accuracy of OSA identification is improved.

[0010] 2. To reduce the false positive rate, the present invention simultaneously collects the subject's pulse data, including but not limited to heart rate, blood oxygen saturation, and pulse wave transit time, in addition to infrared thermal imaging time-series data. Data processing (feature extraction) of this pulse data reveals the amplitude of heart rate increases, amplitude of blood oxygen saturation decreases, and pulse wave amplitude attenuation. This multimodal data is then used to identify OSA. Time-series correlation analysis is performed between the pulse data and infrared thermal imaging time-series data, with changes in these indicators within the same time period used to determine whether OSA has occurred.

[0011] 3. Different sleeping positions have an impact on OSA detection. Therefore, during the OSA assessment process, the OSA assessment indicators are dynamically weighted by detecting the tester's sleeping position. When lying on your back, the focus is on blood oxygen saturation, and when lying on your side, the focus is on changes in chest and abdominal movement, further improving the accuracy of OSA assessment.

[0012] 4. This invention prioritizes severity classification during OSA testing. The apnea-hypopnea index and blood oxygen decline slope are further calculated based on OSA assessment indicators. These are then used to grade severity. Furthermore, the invention distinguishes between OSA and CSA through precise identification of chest and abdominal movements and airflow changes. When chest and abdominal movements and respiratory airflow disappear synchronously within the same time period, the condition is identified as CSA, eliminating the problem of misdiagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 A schematic flow chart of a method for detecting obstructive sleep apnea syndrome based on multimodal data provided by an embodiment of the present invention; Figure 2 A schematic diagram of an obstructive sleep apnea syndrome detection system based on multimodal data provided by an embodiment of the present invention; Figure 3 Schematic diagram of an obstructive sleep apnea syndrome detection device based on multimodal data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0016] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as S101, S102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0017] Figure 1 A schematic diagram of a method for detecting obstructive sleep apnea syndrome based on multimodal data provided by an embodiment of the present invention specifically includes: The test subject wears a wrist or earlobe device, including a pulse sensor, to obtain pulse data, including any one or more of the following: pulse signal, heart rate, pulse transit time, and blood oxygen saturation. Infrared thermal imaging equipment is set up to obtain infrared thermal imaging time series data from the test subject.

[0018] S1: Real-time acquisition of the tester's infrared thermal imaging time-series frame data; In one embodiment, infrared thermal imaging equipment is used to capture thermal images of the tester in real time, and time series frame data is obtained within a continuous period of time.

[0019] S2: extracting chest and abdomen position data and air temperature data of the nose and mouth area based on the infrared thermal imaging time series frame data; In one embodiment, the chest and abdomen and closed mouth regions are identified on infrared thermal imaging time-series frame data to obtain the chest and abdomen regions and the nose and mouth regions, and the chest and abdomen position data and the nose and mouth region temperature data in the time-series frame data are extracted.

[0020] In one embodiment, region identification is performed by an object detection algorithm.

[0021] S3: Calculate respiratory waveform data through chest and abdomen position data in time series frame data; In one embodiment, the chest and abdomen position data are calculated by extracting the displacement of the isotherms in the time-series chest and abdomen thermal map, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data.

[0022] S4, obtaining temperature change curve data by calculating the temperature data in the time series frame data; In one embodiment, there is a temperature difference between the respiratory image temperature peak and the inhalation phase temperature peak, and air temperature change curve data is obtained by statistically analyzing the air temperature change data.

[0023] S5, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to calculate the respiratory ratio; The respiratory ratio is compared with a preset threshold, and when the respiratory ratio is greater than the preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0024] In one embodiment, the calculation process of the breathing ratio includes: The phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data is performed to obtain the relevant respiratory data. The relevant respiratory data is the change of respiratory displacement and airflow temperature in the same time period. Based on the airflow temperature change curve data, the temperature peak and valley values are identified, and the inhalation and exhalation time intervals are divided based on the temperature peak and valley values; Based on the correlation respiratory data, the respiratory displacement change in the inhalation time interval and the respiratory displacement change in the exhalation time interval are obtained, and respiratory verification is performed to obtain the verification respiratory time interval and the inhalation time interval; The temperature peak duration and trough duration are obtained based on the verified breathing time interval and the inspiratory time interval; The respiration ratio was calculated based on the duration of the temperature peak and the duration of the trough.

[0025] In one embodiment, the respiratory verification is that the respiratory displacement increases during the inspiration time interval and decreases during the expiration time interval. Motion artifacts are eliminated through cross-validation to ensure consistency of time segmentation.

[0026] In one embodiment, the method further includes a pulsation feature, and whether obstructive sleep apnea syndrome occurs is determined by using the pulsation feature and the respiratory ratio; the pulsation feature includes one or more of the following: heart rate, blood oxygen saturation, and pulse wave amplitude.

[0027] In one embodiment, whether obstructive sleep apnea syndrome occurs is determined by heart rate and respiratory ratio, and the heart rate values in continuous time are obtained. In a time axis, when the respiratory ratio is greater than a preset threshold and the heart rate increase is greater than the preset heart rate threshold, obstructive sleep apnea syndrome is determined to occur.

[0028] In one embodiment, whether obstructive sleep apnea syndrome occurs is determined by blood oxygen saturation and respiratory ratio, and blood oxygen saturation is obtained over a continuous period of time. In a time axis, when the respiratory ratio is greater than a preset threshold and the decrease in blood oxygen saturation is greater than the preset threshold of blood oxygen, obstructive sleep apnea syndrome is determined to occur.

[0029] In one embodiment, whether obstructive sleep apnea syndrome occurs is determined by pulse wave amplitude and respiratory ratio. The pulse wave transmission time and pulse waveform are obtained in a continuous time. The pulse wave amplitude change is calculated based on the pulse wave transmission time and pulse waveform. On a time axis, when the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than the pulse preset threshold, obstructive sleep apnea syndrome is determined to occur.

[0030] In one embodiment, whether obstructive sleep apnea syndrome occurs is determined by heart rate, blood oxygen saturation and respiratory ratio. Blood oxygen saturation and heart rate are obtained over a continuous period of time. In a time axis, when the respiratory ratio is greater than a preset threshold, the heart rate increase is greater than the heart rate preset threshold, and the blood oxygen saturation decrease is greater than the blood oxygen preset threshold, obstructive sleep apnea syndrome is determined to have occurred.

[0031] In one embodiment, the preset threshold values for respiratory ratio, heart rate, blood oxygen saturation, and pulse are values for normal breathing.

[0032] In one embodiment, the obstructive sleep apnea syndrome also includes dynamic allocation of body posture weights, and the tester's contour is extracted from the infrared imaging time-series frame data to obtain a thermal imaging contour; based on the thermal imaging contour classification, a classification result of supine or side-lying is obtained. When the sleeping position is supine, the weight of the blood oxygen saturation decrease amplitude is increased; when the sleeping position is side-lying, the weight of the chest and abdominal position data is increased.

[0033] In one embodiment, the wrist or earlobe device worn by the tester further includes an inertial sensor for acquiring the tester's three-axis acceleration and three-axis angular velocity signals, and calculating the tester's sleeping position through the acceleration.

[0034] Optionally, the sleeping position is obtained by performing sleeping position recognition through an inertial sensor and infrared thermal imaging; Furthermore, coarse-grained body position classification is performed using inertial sensors, and then fine-grained body position verification is performed using infrared thermal imaging to obtain the final sleeping position result.

[0035] In a specific embodiment, infrared thermal imaging data: continuously collects the thermal radiation distribution of the chest and abdomen; MPU6050 sensor data: integrated into a wrist-worn device or earlobe clip, outputs three-axis acceleration (±16g) and three-axis angular velocity (±2000° / s) signals in real time, and is synchronously transmitted to the main control unit via low-power Bluetooth (BLE 5.0) or WiFi.

[0036] Data processing: Coarse-grained body position classification (MPU6050-driven). Accelerometer static posture calculation: Calculate the gravity direction vector and determine the general body position through pitch and roll angles.

[0037] Supine: Pitch angle close to 0° (±5°), Roll angle close to 0° (±5°); Side lying: The pitch angle is close to 90° (±15°), and the roll angle is different depending on the left or right side (left: +90°~+120°, right: -90°~-120°).

[0038] Fine-grained body position verification (dominated by infrared thermal imaging): thermal imaging contour matching.

[0039] Verification in supine position: chest hot zone symmetry > 85% and area ratio > 60%; Verification during side-lying: significant changes in unilateral intercostal thermal gradient (ΔT>1.5℃ / cm).

[0040] The system performs data fusion judgment based on the MPU6050 sensor data and infrared thermal imaging data, and outputs body posture parameters, such as supine / left side / right side / prone.

[0041] In a specific embodiment, a red light and infrared light source LED is integrated into an integrated pulse sensor with a wide spectrum and high sensitivity for measuring pulse waveform, heart rate value, and blood oxygen value.

[0042] Light source: integrated dual-wavelength LED (red light 660nm ±5nm, infrared light 940nm ±10nm), using time-division multiplexing technology to emit light alternately to avoid spectral interference.

[0043] Photodetector: High-sensitivity silicon photodiode (response wavelength range 500-1100nm), supporting reflective signal acquisition; Data interface: Output digital signals to the main control unit through the UART interface.

[0044] Heart rate calculation: By analyzing the data of the optical channel, the peak interval in each heartbeat cycle is identified, and the heart rate (unit: beats / minute) is calculated.

[0045] Blood oxygen saturation calculation: Using red light and infrared light data, blood oxygen saturation is estimated by comparing the ratio of light absorption at two different wavelengths.

[0046] In one embodiment, the method further comprises grading obstructive sleep apnea syndrome, wherein the obstructive sleep apnea syndrome is graded into mild, moderate, and severe by using the blood oxygen saturation decrease slope and the apnea-hypopnea index.

[0047] In one embodiment, the blood oxygen saturation decrease slope is obtained by calculating the change in blood oxygen saturation over a continuous period of time.

[0048] In one embodiment, the apnea-hypopnea index is calculated based on airflow temperature change curve data and blood oxygen saturation decrease amplitude; Optionally, the apnea-hypopnea index calculation process is: obtaining the total sleep time; extracting the number of apneas through the airflow temperature change curve data; obtaining the number of hypoventilation through the airflow temperature change curve data and the amplitude of the decrease in blood oxygen saturation; and calculating the apnea-hypopnea index through the total sleep time, the number of apneas, and the number of hypoventilation.

[0049] In one embodiment, S5 is replaced by: judging whether obstructive sleep apnea syndrome occurs based on the respiratory waveform data and the airflow temperature change curve data, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to obtain relevant respiratory data, and when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen perform contradictory movements, it is judged that obstructive sleep apnea syndrome occurs.

[0050] In one embodiment, the judgment also includes one or more of the following: heart rate, blood oxygen saturation.

[0051] In one embodiment, the judgment is completed through respiratory waveform data, airflow temperature change curve data, heart rate, and blood oxygen saturation. The heart rate increase amplitude is calculated for the heart rate in a continuous time, and the blood oxygen saturation decrease amplitude is calculated for the blood oxygen saturation in a continuous time. When the duration of respiratory airflow interruption in the relevant respiratory data is greater than the preset respiratory threshold, the chest and abdomen perform contradictory movements, the heart rate increase amplitude is greater than the preset heart rate threshold, and the blood oxygen saturation decrease amplitude is greater than the preset blood oxygen threshold, it is judged that obstructive sleep apnea syndrome has occurred.

[0052] In one embodiment, the judgment is based on the respiratory airflow, chest and abdominal movements, heart rate increase, and blood oxygen decrease within the same time period.

[0053] Optionally, the judgment also includes dynamic weight allocation of body position, extracting thermal imaging contours to determine sleeping position, when the sleeping position is lying on the back, increasing the weight of the decrease in blood oxygen saturation; when the sleeping position is lying on the side, increasing the weight of the chest and abdomen position data.

[0054] In one embodiment, the method further includes CSA differentiation, where CSA is determined to have occurred when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen stop moving during the interruption duration.

[0055] In a specific embodiment, an infrared thermal imaging camera is used to collect real-time thermal images of chest and abdominal respiratory movements and nasal and oral airflow, and phase difference analysis (chest and abdominal movements and airflow signals) is used to determine contradictory breathing and identify OSA events.

[0056] Heart rate (HRV), blood oxygen saturation (SpO2), and pulse wave transit time (PTT) are detected through wrist artery / earlobe blood vessel signals; OSA-related features are extracted: blood oxygen decline slope, sudden increase in heart rate, pulse wave amplitude attenuation, etc.

[0057] Multimodal data fusion is performed on the above indicators (features), and the indicator features are input into the neural network model to perform classification prediction after data fusion to obtain the prediction results of whether OSA occurs or not.

[0058] In another embodiment, the above indicators (features) are subjected to a time series correlation analysis, and the changes in various indicators within the same time period are detected. When the respiratory airflow is interrupted for more than 10 seconds, the chest and abdomen make paradoxical movements, the wrist / earlobe SpO2 decreases by ≥4%, and the heart rate suddenly increases by ≥10%, it is determined that OSA has occurred.

[0059] In one embodiment, when performing temporal correlation analysis on indicators (features), sleep position recognition is performed, the decision weight of the wrist SpO2 decrease slope is increased (60%) in the supine position, and the earlobe PTT and / or chest and abdominal movement analysis are emphasized in the lateral position.

[0060] In a specific embodiment, the method also includes a voice alarm. When an OSA event is detected, the system determines the user's sleeping position (supine / side-lying) in real time through infrared thermal imaging time-series frames, and integrates the OSA severity grade to push position adjustment suggestions to trigger the device voice reminder (such as "Abnormal breathing detected, it is recommended to switch to left side lying position").

[0061] In one embodiment, the method further includes generating a grading report, generating an OSA severity grading report based on the data of the nocturnal blood oxygen decrement slope (ODI) and the apnea-hypopnea index (AHI), In a specific embodiment, the technical solution of the present invention has OSA-specific detection: accurate classification is achieved through joint analysis of multimodal signals, solving the problem that existing wrist-worn / earlobe devices cannot distinguish between OSA and CSA; compared with traditional detection methods, a non-invasive wrist / earlobe wearing solution is provided to avoid the discomfort of traditional surface electrodes; and it has dynamic intervention capabilities, and analyzes micro-arousals and hemodynamic changes in real time based on pulse signals to provide OSA intervention recommendations.

[0062] In a specific embodiment, respiratory movement and body position are monitored by infrared thermal imaging, and blood oxygen, heart rate and pulse wave signals are synchronously collected through wrist / earlobe sensors; the duration of respiratory airflow interruption and the phase difference of chest and abdominal movement are extracted from the thermal imaging time series data, and the SpO2 decline slope / amplitude, heart rate oscillation amplitude, and pulse wave attenuation index are extracted from the pulse signal; if the respiratory interruption is greater than 10 seconds and there is contradictory breathing, combined with a decrease in wrist / earlobe SpO2 of ≥4%, it is determined to be an OSA event; if the chest and abdominal movement and airflow disappear synchronously, it is determined to be a CSA event.

[0063] The cloud collects and analyzes relevant data in real time, calculates the apnea-hypopnea index (AHI), and generates an OSA severity grading report based on the blood oxygen decline slope (ODI).

[0064] The diagnostic criteria for OSA grades include: Mild: AHI 5-14 times / hour, accompanied by a decrease in blood oxygen saturation of 85%-89%; Moderate: AHI 15-30 times / hour, accompanied by a decrease in blood oxygen saturation of 80%-84%; Severe: AHI ≥ 30 times / hour, accompanied by a decrease in blood oxygen saturation ≤ 80%.

[0065] In a specific embodiment, infrared thermal imaging is used to monitor breathing and posture, and combined with chest-abdominal paradoxical breathing phase difference analysis and dynamic changes in pulse wave transmission time (PTT), accurate identification of OSA and CSA can be achieved, effectively distinguishing OSA from positional respiratory disorders, and significantly reducing the risk of misdiagnosis with a single sensor.

[0066] Non-contact infrared monitoring combined with wrist / earlobe pulse sensors avoids the discomfort of traditional surface electrodes, allowing for both home and general medical use, meeting the need for non-invasive nighttime monitoring. Cloud-based analysis of the nocturnal oxygen depletion slope (ODI) and apnea-hypopnea index (AHI) generates an OSA severity grading report (mild / moderate / severe).

[0067] In the present invention, compared with the prior art that uses multiple sensors to identify OSA, the present invention can obtain chest and abdominal movement / respiration waveforms, airflow temperature (change) data only through the time-series frame data of infrared thermal imaging, and calculate the respiratory ratio to complete OSA detection and judgment. In addition, the sleeping position can be verified through infrared thermal imaging sleep position data, and weights can be dynamically allocated when assisting other detection indicators such as blood oxygen saturation to improve the detection rate.

[0068] The disclosed embodiments of the present invention further provide a computer program product or system, including a computer program, which implements the above method steps when executed by a processor.

[0069] Figure 2 The schematic diagram of the obstructive sleep apnea syndrome detection system based on multimodal data provided by an embodiment of the present invention specifically includes: Acquisition unit: acquires the time-series frame data of the tester's infrared thermal imaging in real time; Extraction unit: extracts chest and abdomen position data and air temperature data of nose and mouth area based on the infrared thermal imaging time series frame data; The first data unit: respiratory waveform data is calculated by chest and abdomen position data in the time series frame data; The second data unit: obtains the temperature change curve data by calculating the temperature data in the time series frame data; The judgment unit is configured to calculate the respiratory ratio by performing phase correlation on the respiratory waveform data and the airflow temperature change curve data; compare the respiratory ratio with a preset threshold, and determine that obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than the preset threshold.

[0070] An embodiment of the present invention provides a computer program product, including: Infrared thermal imaging module, used to obtain real-time infrared thermal imaging data; Pulse sensor module, used to obtain pulse data; The data processing module is used to process real-time infrared thermal imaging data and / or pulse data; infrared thermal imaging processing includes: Extracting chest and abdomen position data and air temperature data of the nose and mouth area based on the infrared thermal imaging time series frame data; Respiratory waveform data is obtained by calculating the chest and abdomen position data in the time series frame data; The temperature change curve data is obtained by calculating the temperature data in the time series frame data; Calculating the respiratory ratio by performing phase correlation on the respiratory waveform data and the airflow temperature change curve data; Pulse data processing includes: obtaining pulse data in a continuous time, and calculating the change amplitude or change slope of the pulse data; The determination module determines whether obstructive sleep apnea syndrome occurs by receiving data from the data processing module; when the breathing ratio is greater than a preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0071] In one embodiment, the pulse data obtained includes one or more of the following: heart rate, blood oxygen saturation, pulse wave transit time, and pulse waveform.

[0072] In one embodiment, the determination module determines that obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than a preset threshold and the decrease in blood oxygen saturation is greater than a preset threshold in a time axis.

[0073] In another embodiment, the determination module determines that obstructive sleep apnea syndrome occurs when the breathing ratio is greater than a preset threshold and the heart rate increase amplitude is greater than a preset heart rate threshold in a time axis; In another embodiment, the determination module determines that obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than a preset pulse threshold in a time axis by calculating the change in pulse wave amplitude through pulse wave transmission time and pulse waveform.

[0074] In another embodiment, the determination module determines that obstructive sleep apnea syndrome occurs when, in a time axis, the respiratory ratio is greater than a preset threshold, the heart rate increase is greater than the heart rate preset threshold, and the blood oxygen saturation decrease is greater than the blood oxygen preset threshold.

[0075] In one embodiment, the data processing module extracts the test subject's profile from the infrared imaging time-series frame data to obtain a thermal imaging profile. Based on this thermal imaging profile, the module classifies the subject as either supine or side-lying. When the sleep position is supine, the weight of the blood oxygen saturation decrease is increased; when the sleep position is side-lying, the weight of the chest and abdomen position data is increased. The weighted data is then fed into the decision module for determination.

[0076] In one embodiment, the program product further includes a voice alarm module for alarming an OSA occurrence event.

[0077] When an OSA event is detected, the system uses the infrared thermal imaging module to determine the user's sleeping position (supine / side) in real time, and integrates the OSA severity level to push position adjustment suggestions to trigger the device voice reminder (such as "Abnormal breathing detected, it is recommended to switch to the left side lying position").

[0078] In one embodiment, the program product further includes a cloud-based health management platform for generating an OSA severity grading report.

[0079] An OSA severity grading report is generated based on the nighttime oxygen decrement slope (ODI) and apnea-hypopnea index (AHI) data.

[0080] Figure 3 A schematic diagram of a computer device provided by an embodiment of the present invention specifically includes: A memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions, and when the program instructions are executed, any one of the above-mentioned obstructive sleep apnea syndrome detection methods based on multimodal data is executed.

[0081] The disclosed embodiments of the present invention further provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs any of the above-mentioned methods for detecting obstructive sleep apnea syndrome based on multimodal data.

[0082] The validation results of this validation example demonstrate that assigning inherent weights to indications can improve the performance of the present method compared to the default settings. Those skilled in the art will readily appreciate that, for ease of description and brevity, the specific operating processes of the systems, devices, and units described above can be referenced to the corresponding processes in the aforementioned method embodiments and will not be further elaborated upon here. It should be understood that the disclosed systems, devices, and methods can be implemented in other ways within the several embodiments provided herein. For example, the device embodiments described above are merely illustrative. For example, the division of units described is merely a logical functional division. In actual implementation, other divisions may be employed, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, the coupling, direct coupling, or communication connection shown or discussed may be through interfaces, indirect coupling, or communication connection between devices or units, and may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the units may be selected to achieve the objectives of the present embodiment as needed. In addition, the functional units in the various embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units. Those skilled in the art will understand that all or part of the steps in the various methods of the above-mentioned embodiments may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0083] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned medium storage can be a read-only memory, a disk or an optical disk, etc.

[0084] The above is a detailed introduction to a computer device provided by the present invention. For those skilled in the art, according to the concept of the embodiments of the present invention, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for detecting obstructive sleep apnea syndrome based on multimodal data, characterized in that: include: S1. Real-time acquisition of time-series frame data of infrared thermal imaging of the tester; S2. Extracting chest and abdomen position data and air temperature data of the nose and mouth area based on the infrared thermal imaging time series frame data; S3, obtaining respiratory waveform data by calculating the chest and abdomen position data in the time series frame data; S4. Calculate the temperature change curve data by using the temperature data in the time series frame data; S5, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to calculate the respiratory ratio; The respiratory ratio is compared with a preset threshold, and when the respiratory ratio is greater than the preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

2. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 1, characterized in that: The chest and abdomen position data are calculated by extracting the displacement of the isotherms in the time-series chest and abdomen thermal map, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data.

3. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 1, characterized in that: The calculation process of the breathing ratio includes: The phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data is performed to obtain the relevant respiratory data. The relevant respiratory data is the change of respiratory displacement and airflow temperature in the same time period. Based on the airflow temperature change curve data, the temperature peak and valley values are identified, and the inhalation and exhalation time intervals are divided based on the temperature peak and valley values; Based on the correlation respiratory data, the respiratory displacement change in the inhalation time interval and the respiratory displacement change in the exhalation time interval are obtained, and respiratory verification is performed to obtain the verification respiratory time interval and the inhalation time interval; The temperature peak duration and trough duration are obtained based on the verified breathing time interval and the inspiratory time interval; The respiratory ratio was calculated based on the duration of the temperature peak and the duration of the trough; Optionally, the breathing verification is that the breathing displacement increases during the inhalation time interval and decreases during the exhalation time interval.

4. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 1, characterized in that: The method further includes a pulsation feature, and determining whether obstructive sleep apnea syndrome occurs by using the pulsation feature and the respiratory ratio; the pulsation feature includes one or more of the following: heart rate, blood oxygen saturation, and pulse wave amplitude; Optionally, whether obstructive sleep apnea syndrome occurs is determined by heart rate and respiratory ratio, and the heart rate values in a continuous time are obtained. In a time axis, when the respiratory ratio is greater than a preset threshold and the heart rate increase is greater than the preset heart rate threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by blood oxygen saturation and respiratory ratio, blood oxygen saturation is obtained over a continuous period of time, and when, on a time axis, the respiratory ratio is greater than a preset threshold and the amplitude of the decrease in blood oxygen saturation is greater than a preset threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by pulse wave amplitude and respiratory ratio, the pulse wave transmission time and pulse waveform are obtained in a continuous time, and the pulse wave amplitude change is calculated by the pulse wave transmission time and pulse waveform. When the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than the pulse preset threshold on a time axis, obstructive sleep apnea syndrome is determined to occur; Optionally, whether obstructive sleep apnea syndrome occurs is determined by heart rate, blood oxygen saturation and respiratory ratio, blood oxygen saturation and heart rate are obtained over a continuous period of time, and when, on a time axis, the respiratory ratio is greater than a preset threshold, the heart rate increase is greater than a preset heart rate threshold, and the blood oxygen saturation decrease is greater than a preset blood oxygen threshold, obstructive sleep apnea syndrome is determined to occur; Optionally, the obstructive sleep apnea syndrome further includes dynamic allocation of body posture weights, performing tester contour extraction on infrared imaging time series frame data to obtain a thermal imaging contour; based on the thermal imaging contour classification, a supine or side-lying classification result is obtained, and when the sleeping position is supine, the weight of the decrease in blood oxygen saturation is increased; When the sleeping position is lying on the side, the weight of the chest and abdomen position data is increased; based on the data after the posture weight is dynamically allocated, it is determined whether obstructive sleep apnea syndrome occurs.

5. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 4, characterized in that: The method further includes grading obstructive sleep apnea syndrome. When obstructive sleep apnea syndrome is determined to occur, the obstructive sleep apnea syndrome is graded into mild, moderate, or severe levels by using the blood oxygen saturation decrease slope and the apnea-hypopnea index. Optionally, the blood oxygen saturation decrease slope is obtained by calculating the change in blood oxygen saturation over a continuous period of time; Optionally, the apnea-hypopnea index is calculated based on airflow temperature change curve data and blood oxygen saturation decrease amplitude; Optionally, the apnea-hypopnea index calculation process is: obtaining the total sleep time; extracting the number of apneas through the airflow temperature change curve data; obtaining the number of hypoventilation through the airflow temperature change curve data and the amplitude of the decrease in blood oxygen saturation; and calculating the apnea-hypopnea index through the total sleep time, the number of apneas, and the number of hypoventilation.

6. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 1, characterized in that: S5 is replaced with: determining whether obstructive sleep apnea syndrome occurs based on the respiratory waveform data and the airflow temperature change curve data, performing phase correlation calculation on the respiratory waveform data and the airflow temperature change curve data to obtain relevant respiratory data, and determining that obstructive sleep apnea syndrome occurs when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen perform paradoxical movements; Optionally, the judgment further includes one or more of the following: heart rate, blood oxygen saturation; Optionally, the judgment is performed using respiratory waveform data, airflow temperature change curve data, heart rate, and blood oxygen saturation. The heart rate increase amplitude is calculated for the heart rate over a continuous period of time, and the blood oxygen saturation decrease amplitude is calculated for the blood oxygen saturation over a continuous period of time. When the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold, the chest and abdomen perform contradictory movements, the heart rate increase amplitude is greater than a preset heart rate threshold, and the blood oxygen saturation decrease amplitude is greater than a preset blood oxygen threshold, it is determined that obstructive sleep apnea syndrome has occurred. Optionally, the judgment is based on the respiratory airflow, chest and abdominal movements, heart rate increase, and blood oxygen decrease within the same time period; Optionally, the judgment also includes dynamic weight allocation of body position, extracting thermal imaging contours to determine sleeping position, when the sleeping position is lying on the back, increasing the weight of the decrease in blood oxygen saturation; when the sleeping position is lying on the side, increasing the weight of the chest and abdomen position data; and making judgments based on the data after dynamic weight allocation of body position.

7. The obstructive sleep apnea syndrome detection method based on multimodal data according to claim 6, characterized in that: The method further includes CSA differentiation, where CSA is determined to have occurred when the duration of respiratory airflow interruption in the relevant respiratory data is greater than a preset respiratory threshold and the chest and abdomen stop moving during the interruption duration.

8. A computer program product comprising a computer program or instructions, characterized in that: The computer program or instructions are executed by a processor to implement the obstructive sleep apnea syndrome detection method based on multimodal data according to any one of claims 1 to 7; Optionally, the computer program product includes: Infrared thermal imaging module, used to obtain real-time infrared thermal imaging data; Pulse sensor module, used to obtain pulse data; A data processing module, configured to process real-time infrared thermal imaging data and / or pulse data; The determination module determines whether obstructive sleep apnea syndrome occurs by receiving data from the data processing module.

9. A computer device comprising a memory, a processor, and a computer program or instruction stored in the memory, characterized in that: The computer program or instructions are executed by a processor to implement the obstructive sleep apnea syndrome detection method based on multimodal data according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instructions are executed by a processor to implement the obstructive sleep apnea syndrome detection method based on multimodal data according to any one of claims 1 to 7.

Images