A method, device, and program product for detecting obstructive sleep apnea syndrome based on multi-modal data

Abstract

The application relates to the field of intelligent medical treatment, in particular to a method, equipment and program product for detecting obstructive sleep apnea syndrome based on multi-modal data. The method comprises the following steps: S1, acquiring time sequence frame data of infrared thermal imaging of a tester in real time; S2, extracting chest and abdomen position data and air temperature data of a nose and mouth area based on the time sequence frame data of the infrared thermal imaging; S3, calculating breathing waveform data by using the chest and abdomen position data in the time sequence frame data; S4, calculating air temperature change curve data by using the air temperature data in the time sequence frame data; S5, calculating a breathing ratio by performing phase correlation calculation on the breathing waveform data and the air temperature change curve data; and comparing the breathing ratio with a preset threshold value, and determining that obstructive sleep apnea syndrome occurs when the breathing ratio is greater than the preset threshold value. The application can effectively identify obstructive sleep apnea syndrome and has good clinical value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent medical treatment, in particular to a method, device, program product and computer readable storage medium for detecting obstructive sleep apnea syndrome based on multi-modal data. BACKGROUND

[0002] Obstructive sleep apnea (OSA) is a common sleep breathing disorder with a high prevalence rate and is a condition with potential fatal complications. The pathogenesis of OSA is a clinical syndrome of chronic intermittent hypoxia and sleep fragmentation, accompanied by hypoxemia, hypercapnia and a series of physiological changes, which is easy to cause damage to multiple systems of the whole body, leading to multiple organ and system complications such as coronary heart disease, hypertension and diabetes. Traditional polysomnography (PSG) is used for detecting obstructive sleep apnea. PSG requires multiple electrodes and sensors to be attached to the body surface, which is complex to operate, has poor comfort, and relies on a professional medical environment, making it difficult to meet the needs of family and general medical settings. Single sensor devices (such as oximeters and respiratory bands) rely only on a single signal such as blood oxygen or respiratory airflow, are easily disturbed, have a high rate of missed detection, and cannot distinguish between obstructive apnea (OSA) and central apnea (CSA), resulting in a risk of misdiagnosis. SUMMARY

[0003] Although wearable devices on the market (such as wrist-mounted oximeters) have improved portability, they are limited to single blood oxygen data and lack the ability to analyze correlations with breathing, body position, etc., and cannot provide precise intervention recommendations by dynamically capturing OSA-specific micro-awakening signals from pulse waveforms. In the disclosed related technology, respiratory monitoring based on infrared thermal imaging does not combine pulse signal and body position analysis, cannot identify OSA paradoxical breathing characteristics, and does not involve time sequence correlation analysis of respiratory airflow interruption and pulse signals, resulting in a high rate of misjudgment and making it difficult to meet the needs of clinical precise monitoring.

[0004] To solve the above problems, the present application provides a method for detecting obstructive sleep apnea syndrome based on multi-modal data, which specifically comprises:

[0005] S1, real-time acquisition of time sequence frame data of infrared thermal imaging of a testee;

[0006] S2, extraction of chest and abdominal position data and air temperature data of the nose and mouth region based on the time sequence frame data of the infrared thermal imaging;

[0007] S3, calculation of respiratory waveform data from the chest and abdominal position data in the time sequence frame data;

[0008] S4, calculation of air temperature change curve data from the air temperature data in the time sequence frame data;

[0009] S5, performing phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data to obtain a respiratory ratio;

[0010] Comparing the respiratory ratio with a preset threshold value, and determining that the obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than the preset threshold value.

[0011] The thoracico-abdominal position data calculation is performed by extracting the displacement of the isotherm in the time-series thoracico-abdominal thermogram, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data.

[0012] The respiratory ratio calculation process includes:

[0013] The phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data obtains correlation respiratory data, which is the respiratory displacement change and the airflow temperature change in the same time period.

[0014] The temperature peak and valley are identified based on the airflow temperature change curve data, and the inspiration and expiration time intervals are divided based on the temperature peak and valley.

[0015] The respiratory displacement change in the inspiration time interval and the respiratory displacement change in the expiration time interval are obtained based on the correlation respiratory data, and the verification respiratory time interval and the inspiration time interval are obtained by performing respiratory verification.

[0016] The temperature peak duration and the valley duration are obtained based on the verification respiratory time interval and the inspiration time interval.

[0017] The respiratory ratio is calculated based on the temperature peak duration and the valley duration.

[0018] Optionally, the respiratory verification is that the respiratory displacement is rising in the inspiration time interval, and the respiratory displacement is falling in the expiration time interval.

[0019] The method further includes a pulsation feature, and whether the obstructive sleep apnea syndrome occurs is determined based on the pulsation feature and the respiratory ratio; the pulsation feature includes one or more of the following: heart rate, blood oxygen saturation, and pulse amplitude.

[0020] Optionally, whether the obstructive sleep apnea syndrome occurs is determined based on the heart rate and the respiratory ratio, the heart rate values in a continuous time are obtained, and in a time axis, when the respiratory ratio is greater than a preset threshold value and the heart rate rising amplitude is greater than a preset threshold value of the heart rate, it is determined that the obstructive sleep apnea syndrome occurs.

[0021] Optionally, the occurrence of obstructive sleep apnea syndrome is determined by blood oxygen saturation and respiratory ratio, blood oxygen saturation in continuous time is obtained, and in a time axis, when the respiratory ratio is greater than a preset threshold and the blood oxygen saturation decreases by more than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0022] Optionally, the occurrence of obstructive sleep apnea syndrome is determined by pulse wave amplitude and respiratory ratio, pulse wave transit time and pulse wave form in continuous time are obtained, pulse wave amplitude change is calculated from the pulse wave transit time and the pulse wave form, and in a time axis, when the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than a pulse preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0023] Optionally, the occurrence of obstructive sleep apnea syndrome is determined by heart rate, blood oxygen saturation and respiratory ratio, blood oxygen saturation and heart rate in continuous time are obtained, and in a time axis, when the respiratory ratio is greater than a preset threshold, the heart rate increases by more than a heart rate preset threshold, and the blood oxygen saturation decreases by more than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0024] Optionally, the obstructive sleep apnea syndrome further comprises dynamic distribution of body position weight, tester contour is extracted from infrared imaging time sequence frame data to obtain thermal imaging contour; the classification result of supine or lateral lying is obtained based on the thermal imaging contour classification judgment; when the sleep position is supine, the weight of the blood oxygen saturation decrease amplitude is increased; when the sleep position is lateral lying, the weight of the chest and abdomen position data is increased.

[0025] The method further comprises obstructive sleep apnea syndrome grading, and the obstructive sleep apnea syndrome is graded as mild, moderate or severe by blood oxygen saturation decrease slope and apnea hypopnea index;

[0026] Optionally, the blood oxygen saturation decrease slope is calculated from the change of blood oxygen saturation in continuous time;

[0027] Optionally, the apnea hypopnea index is calculated from the airflow temperature change curve data and the blood oxygen saturation decrease amplitude;

[0028] Optionally, the apnea hypopnea index calculation process is: obtaining total sleep time; extracting the number of apneas from the airflow temperature change curve data; discriminating the number of hypopneas from the airflow temperature change curve data and the blood oxygen saturation decrease amplitude; calculating the apnea hypopnea index from the total sleep time, the number of apneas and the number of hypopneas.

[0029] S5 is replaced by: judging whether obstructive sleep apnea syndrome occurs based on the respiratory waveform data and the airflow temperature change curve data, calculating the correlation respiratory data by phase correlation of the respiratory waveform data and the airflow temperature change curve data, and when the duration of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold and the chest and abdomen perform contradictory movements, it is judged that obstructive sleep apnea syndrome occurs;

[0030] Optionally, the judgment further includes one or more of the following: heart rate, blood oxygen saturation;

[0031] Optionally, the judgment is completed by the respiratory waveform data and the airflow temperature change curve data, the heart rate, and the blood oxygen saturation, the heart rate rising amplitude is calculated for the heart rate in a continuous time, the blood oxygen saturation falling amplitude is calculated for the blood oxygen saturation in a continuous time, when the duration of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold, the chest and abdomen perform contradictory movements, the heart rate rising amplitude is greater than a preset heart rate threshold, and the blood oxygen saturation falling amplitude is greater than a preset blood oxygen threshold, it is judged that obstructive sleep apnea syndrome occurs;

[0032] Optionally, the judgment is made on the respiratory airflow, the chest and abdomen movement, the heart rate rising amplitude, and the blood oxygen falling amplitude in the same time period.

[0033] Optionally, the judgment further includes body position dynamic weight distribution, the sleep position is judged by extracting the thermal imaging profile, when the sleep position is supine, the weight of the blood oxygen saturation falling amplitude is increased; and when the sleep position is lateral, the weight of the chest and abdomen position data is increased.

[0034] The method further includes CSA differentiation, when the duration of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold and the chest and abdomen stop moving in the duration, it is judged that CSA occurs.

[0035] The purpose of the present application is to provide a computer program product, which includes a computer program or instructions thereon, the computer program or instructions are executed by a processor to realize the above-mentioned obstructive sleep apnea syndrome detection method based on multi-modal data.

[0036] The present application provides a computer program product, which includes:

[0037] An infrared thermal imaging module is configured to acquire real-time infrared thermal imaging data.

[0038] A pulse sensor module is configured to acquire pulse data.

[0039] A data processing module is configured to process the real-time infrared thermal imaging data and / or the pulse data.

[0040] Extracting chest and abdomen position data and air temperature data of nose and mouth region based on the infrared thermal imaging time sequence frame data;

[0041] Calculating breathing waveform data through chest and abdomen position data in time sequence frame data;

[0042] Calculating air temperature change curve data through air temperature data in time sequence frame data;

[0043] Calculating respiratory ratio through phase correlation of the breathing waveform data and air flow temperature change curve data;

[0044] Pulse data processing includes: obtaining pulse data in continuous time, and calculating change amplitude or change slope of the pulse data;

[0045] A determination module determines whether obstructive sleep apnea syndrome occurs by receiving data processed by the data processing module; when the respiratory ratio is greater than a preset threshold, it is determined that obstructive sleep apnea syndrome occurs;

[0046] Optionally, the obtained pulse data includes one or more of the following: heart rate, blood oxygen saturation.

[0047] The purpose of the present application is to provide a computer device, which includes a memory, a processor and a computer program or instructions stored on the memory, the computer program or instructions being executed by the processor to realize the above-mentioned obstructive sleep apnea syndrome detection method based on multi-modal data.

[0048] The purpose of the present application is to provide a computer readable storage medium, which stores a computer program or instructions, the computer program or instructions being executed by the processor to realize the above-mentioned obstructive sleep apnea syndrome detection method based on multi-modal data.

[0049] Advantages of the present application:

[0050] 1. The present application tests whether the subject has obstructive sleep apnea syndrome through infrared thermal imaging time sequence data (real-time), specifically uses breathing waveform data and air temperature change data to calculate and verify respiratory movement, calculates respiratory ratio through verified breathing data, discriminates OSA through respiratory ratio data, verifies lung and abdomen movement and air flow change of nose and mouth region under the same time axis, eliminates motion artifact interference, ensures time segmentation consistency, and improves OSA discrimination accuracy.

[0051] 2. In order to reduce the misjudgment rate, the pulse data of the tester, including but not limited to heart rate, blood oxygen saturation, pulse wave transmission time, are collected at the same time when the infrared thermal imaging time sequence data are collected. The heart rate rise amplitude, the blood oxygen saturation drop amplitude, and the pulse wave amplitude attenuation are obtained by data processing (feature extraction) of the pulse data. The OSA is discriminated by the above-mentioned multi-modal data. The pulse data and the infrared thermal imaging time sequence data are analyzed for time sequence correlation, and the changes of various indexes in the same time period are used to determine whether OSA occurs.

[0052] 3. Different sleep positions have an impact on the detection of OSA, so the sleep position of the tester is detected during the OSA determination process to dynamically allocate the determination indexes of OSA. The blood oxygen saturation is emphasized when lying on the back, and the chest and abdominal movement changes are emphasized when lying on the side, thereby further improving the accuracy of OSA determination.

[0053] 4. The present application classifies the severity of OSA detection. The apnea hypopnea index and the blood oxygen drop slope are calculated by the determination indexes of OSA, and the severity classification is performed by using the apnea hypopnea index and the blood oxygen drop slope. In addition, the present application also distinguishes OSA from CSA. The OSA and CSA are identified by accurate chest and abdominal movement and airflow changes. When the chest and abdominal movement and the respiratory airflow disappear synchronously in the same time period, it is determined that CSA occurs, thereby solving the misdiagnosis problem of OSA and CSA. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0055] Figure 1 The flowchart of the OSA detection method based on multi-modal data provided by the embodiment of the present application is shown.

[0056] Figure 2 The schematic diagram of the OSA detection system based on multi-modal data provided by the embodiment of the present application is shown.

[0057] Figure 3 The schematic diagram of the OSA detection device based on multi-modal data provided by the embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] In order for those skilled in the art to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0059] In some of the processes described in this specification and in the accompanying drawings, multiple operations are described in a specific order. However, it should be understood that unless otherwise specifically stated, these operations can be performed in any order, or in parallel, and that the sequence of operations can be changed. The sequence of operations, such as S101, S102, etc., is merely an example for distinguishing between different operations, and the sequence itself is not a limitation of the order of execution. In addition, these processes can include more or fewer operations, and the operations can be performed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this paper are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence or limit the types of "first" and "second".

[0060] Figure 1 The method for detecting obstructive sleep apnea syndrome based on multi-modal data provided by the embodiments of the present application includes the following steps:

[0061] The tester wears a wrist or earlobe device, including a pulse sensor, for acquiring pulse data, including any one or several of the following pulse signals, heart rate, pulse transmission time, and blood oxygen saturation. The tester's infrared thermal imaging time series data is obtained by setting an infrared thermal imaging device.

[0062] S1: Real-time acquisition of time series frame data of the tester's infrared thermal imaging;

[0063] In one embodiment, the tester's thermal imaging is extracted in real time by the infrared thermal imaging device, and time series frame data is obtained within a continuous period of time.

[0064] S2: Extracting chest and abdomen position data and nasal and oral region air temperature data based on the infrared thermal imaging time series frame data;

[0065] In one embodiment, the chest and abdomen and closed mouth region of the infrared thermal imaging time series frame data are identified to obtain the chest and abdomen region and the nasal and oral region, and the chest and abdomen position data and the nasal and oral region air temperature data in the time series frame data are extracted.

[0066] In one embodiment, the region identification is identified by a target detection algorithm.

[0067] S3: Calculating the respiratory waveform data from the chest and abdomen position data in the time series frame data;

[0068] In one embodiment, the chest and abdomen position data calculation is performed by extracting the displacement of the isotherm in the time series chest and abdomen thermal map, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data.

[0069] S4, calculating the air temperature change curve data from the air temperature data in the time sequence frame data;

[0070] In one embodiment, the temperature difference exists between the temperature peak of the respiratory waveform data and the temperature peak of the inhaled phase, and the air temperature change curve data is obtained by statistical analysis of the air temperature change data.

[0071] S5, calculating the respiratory ratio by phase correlation of the respiratory waveform data and the air temperature change curve data;

[0072] Comparing the respiratory ratio with a preset threshold, and determining that obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than the preset threshold.

[0073] In one embodiment, the process of calculating the respiratory ratio comprises:

[0074] The correlation respiratory data is obtained by phase correlation of the respiratory waveform data and the air temperature change curve data, and the correlation respiratory data is the respiratory displacement change and the air temperature change in the same time period.

[0075] The temperature peak and the valley are identified based on the air temperature change curve data, and the inhaled and exhaled time intervals are divided based on the temperature peak and the valley.

[0076] The respiratory displacement change in the inhaled time interval and the respiratory displacement change in the exhaled time interval are obtained based on the correlation respiratory data, and the verification respiratory time interval and the inhaled time interval are obtained by respiratory verification.

[0077] The temperature peak duration and the valley duration are obtained based on the verification respiratory time interval and the inhaled time interval.

[0078] The respiratory ratio is calculated based on the temperature peak duration and the valley duration.

[0079] In one embodiment, the respiratory verification is that the respiratory displacement is rising in the inhaled time interval, and the respiratory displacement is falling in the exhaled time interval. Motion artifact interference is eliminated by cross verification to ensure time segmentation consistency.

[0080] In one embodiment, the method further comprises a pulsation feature, and whether obstructive sleep apnea syndrome occurs is determined by the pulsation feature and the respiratory ratio; the pulsation feature comprises one or more of the following: heart rate, blood oxygen saturation, and pulse amplitude.

[0081] In one embodiment, the determination of whether obstructive sleep apnea syndrome occurs is made by heart rate and breath ratio, the heart rate values in continuous time are obtained, and in a time axis, when the breath ratio is greater than a preset threshold and the heart rate rising amplitude is greater than a heart rate preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0082] In one embodiment, the determination of whether obstructive sleep apnea syndrome occurs is made by blood oxygen saturation and breath ratio, the blood oxygen saturation in continuous time is obtained, and in a time axis, when the breath ratio is greater than a preset threshold and the blood oxygen saturation falling amplitude is greater than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0083] In one embodiment, the determination of whether obstructive sleep apnea syndrome occurs is made by pulse wave amplitude and breath ratio, the pulse wave transit time and pulse wave form in continuous time are obtained, the pulse wave amplitude change is calculated by the pulse wave transit time and the pulse wave form, and in a time axis, when the breath ratio is greater than a preset threshold and the pulse wave amplitude is greater than a pulse preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0084] In one embodiment, the determination of whether obstructive sleep apnea syndrome occurs is made by heart rate, blood oxygen saturation and breath ratio, the blood oxygen saturation and heart rate in continuous time are obtained, and in a time axis, when the breath ratio is greater than a preset threshold, the heart rate rising amplitude is greater than a heart rate preset threshold, and the blood oxygen saturation falling amplitude is greater than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0085] In one embodiment, the breath ratio preset threshold, the heart rate preset threshold, the blood oxygen saturation preset threshold, and the pulse preset threshold are the values when the respiration is normal.

[0086] In one embodiment, the obstructive sleep apnea syndrome further comprises dynamic allocation of body position weight, the tester contour is extracted from the infrared imaging time sequence frame data to obtain a thermal imaging contour, a classification result of supine or lateral lying is obtained based on the thermal imaging contour classification judgment, when the sleep body position is supine, the weight of the blood oxygen saturation falling amplitude is increased, and when the sleep body position is lateral lying, the weight of the chest and abdomen position data is increased.

[0087] In one embodiment, the tester wears a wrist or earlobe type device which further comprises an inertial sensor for obtaining the three-axis acceleration and three-axis angular velocity signals of the tester, and the sleep body position of the tester is calculated by the acceleration.

[0088] Optionally, the sleep body position is obtained by sleep body position recognition by the inertial sensor and infrared thermal imaging.

[0089] Further, the final sleep position result is obtained by coarse-grained body position classification through the inertial sensor and fine-grained body position verification through infrared thermal imaging.

[0090] In one specific embodiment, the infrared thermal imaging data: continuously collects the thermal radiation distribution of the chest and abdomen; the MPU6050 sensor data: integrated in a wrist-worn device or earlobe clip, outputs real-time three-axis acceleration (±16g) and three-axis angular velocity (±2000° / s) signals, and is transmitted to the main control unit through low-power Bluetooth (BLE 5.0) or wifi synchronization.

[0091] Data processing: coarse-grained body position classification (MPU6050 dominant). Accelerometer static posture solution: calculate the gravity direction vector, and determine the general body position through the pitch angle (Pitch) and roll angle (Roll).

[0092] Supine: Pitch angle close to 0° (±5°), Roll angle close to 0° (±5°);

[0093] Side lying: Pitch angle close to 90° (±15°), Roll angle according to left and right side lying (left: +90°~+120°, right: -90°~-120°).

[0094] Fine-grained body position verification (infrared thermal imaging dominant): thermal imaging contour matching.

[0095] Supine verification: chest thermal zone symmetry > 85% and area ratio > 60%;

[0096] Side lying verification: significant change in unilateral intercostal thermal gradient (ΔT > 1.5℃ / cm).

[0097] Based on MPU6050 sensor data and infrared thermal imaging data, data fusion decision is made, and body position state parameters such as supine, left side lying, right side lying, and prone are output.

[0098] In one specific embodiment, red light and infrared light source LEDs are integrated, and a pulse sensor with wide spectrum and high sensitivity is used for pulse waveform, heart rate value, and blood oxygen value measurement.

[0099] Light source: integrated dual-wavelength LED (red light 660nm ±5nm, infrared light 940nm ±10nm), using time division multiplexing technology to alternate light emission, avoiding spectral interference.

[0100] Photodetector: high-sensitivity silicon photodiode (response wavelength range 500-1100nm), supporting reflective signal acquisition;

[0101] Data interface: output digital signal to the main control unit through UART interface.

[0102] Heart rate calculation: By analyzing the data of the light channel, the peak interval in each heartbeat cycle is identified, and the heart rate (unit: times / min) is calculated.

[0103] Oxygen saturation calculation: Using the data of red light and infrared light, the oxygen saturation is estimated by comparing the proportion of light absorption at two different wavelengths.

[0104] In one embodiment, the method further comprises obstructive sleep apnea syndrome grading, mild, moderate, severe grading of obstructive sleep apnea syndrome by oxygen saturation drop slope and apnea hypopnea index.

[0105] In one embodiment, the oxygen saturation drop slope is calculated by the change of oxygen saturation in continuous time.

[0106] In one embodiment, the apnea hypopnea index is calculated by the airflow temperature change curve data and the oxygen saturation drop amplitude.

[0107] Optionally, the apnea hypopnea index calculation process is: obtaining the total sleep time; extracting the apnea frequency from the airflow temperature change curve data; discriminating the hypopnea frequency from the airflow temperature change curve data and the oxygen saturation drop amplitude; calculating the apnea hypopnea index by the total sleep time, the apnea frequency and the hypopnea frequency.

[0108] In one embodiment, S5 is replaced by: based on the respiratory waveform data and the airflow temperature change curve data, whether obstructive sleep apnea syndrome occurs is judged, the phase correlation of the respiratory waveform data and the airflow temperature change curve data is calculated to obtain relevant respiratory data, when the duration of respiratory airflow interruption in the relevant respiratory data is greater than the respiratory preset threshold and the chest and abdomen perform contradictory movement, it is judged that obstructive sleep apnea syndrome occurs.

[0109] In one embodiment, the judgment further includes one or more of the following: heart rate, oxygen saturation.

[0110] In one embodiment, the judgment is completed by the respiratory waveform data and the airflow temperature change curve data, heart rate, oxygen saturation, the heart rate rise amplitude is calculated by the heart rate in continuous time, the oxygen saturation drop amplitude is calculated by the oxygen saturation in continuous time, when the duration of respiratory airflow interruption in the relevant respiratory data is greater than the respiratory preset threshold, the chest and abdomen perform contradictory movement, the heart rate rise amplitude is greater than the heart rate preset threshold, and the oxygen saturation drop amplitude is greater than the oxygen saturation preset threshold, it is judged that obstructive sleep apnea syndrome occurs.

[0111] In one embodiment, the judgment is made on respiratory airflow, chest and abdominal movement, heart rate rise amplitude, and blood oxygen drop amplitude within the same time period.

[0112] Optionally, the judgment further includes body position dynamic weight distribution, extracting thermal imaging profile to judge sleep position, increasing the weight of blood oxygen saturation drop amplitude when the sleep position is supine, and increasing the weight of chest and abdominal position data when the sleep position is lateral.

[0113] In one embodiment, the method further includes CSA differentiation, when the duration of respiratory airflow interruption in the relevant respiratory data is greater than the respiratory preset threshold and the chest and abdomen stop moving within the duration of interruption, it is judged that CSA occurs.

[0114] In one specific embodiment, the chest and abdominal respiratory movement and the nose and mouth airflow thermal imaging are collected in real time by an infrared thermal imaging camera, and the contradictory breathing is determined by phase difference analysis (chest and abdominal movement and airflow signal), and OSA events are identified.

[0115] Through the wrist artery / earlobe blood vessel signal, heart rate (HRV), blood oxygen saturation (SpO2), and pulse wave transmission time (PTT) are detected; OSA-related features are extracted: blood oxygen drop slope, heart rate sudden rise, pulse wave amplitude attenuation, etc.

[0116] The above indicators (features) are subjected to multi-modal data fusion, and the indicator features are input into a neural network model to obtain the prediction result of OSA occurrence or non-OSA occurrence after data fusion and classification prediction.

[0117] In another embodiment, the above indicators (features) are subjected to time series correlation analysis, and the changes of each indicator within the same time period are detected, when the respiratory airflow interruption is >10 seconds, the chest and abdomen make contradictory movement, the wrist / earlobe SpO2 decreases ≥4%, and the heart rate suddenly rises ≥10%, it is determined that OSA occurs.

[0118] In one embodiment, the sleep position is identified when the indicators (features) are subjected to time series correlation analysis, the decision weight of the wrist SpO2 drop slope is increased (60%) when the supine position is improved, and the earlobe PTT and / or chest and abdominal movement analysis is focused on when the lateral position is improved.

[0119] In one specific embodiment, the method further includes voice alarm, when the OSA event is detected, the system determines the user's sleep position (supine / lateral) in real time through the infrared thermal imaging time series frame, and integrates the OSA severity grading, pushes the body position adjustment suggestion to trigger the device voice reminder (such as "respiratory abnormalities are detected, it is recommended to turn to left lateral position").

[0120] In one embodiment, the method further comprises hierarchical report generation, generating an OSA severity hierarchical report based on data of a nocturnal blood oxygen decline slope (ODI) and an apnea hypopnea index (AHI),

[0121] In one specific embodiment, the technical solution of the present application has OSA specific detection: precise typing is achieved through joint analysis of multi-modal signals, solving the problem that existing wrist / earlobe devices cannot distinguish between OSA and CSA; compared with traditional detection methods, a non-invasive wrist / earlobe wearing solution is provided to avoid the discomfort of traditional body surface electrodes; and has dynamic intervention capability, based on real-time analysis of micro-waking and blood oxygen dynamics based on pulse signals, OSA intervention suggestions are provided.

[0122] In one specific embodiment, respiratory movement and body position are monitored by infrared thermal imaging, and blood oxygen, heart rate and pulse wave signals are synchronously collected by wrist / earlobe sensors; the duration of respiratory airflow interruption and the phase difference between chest and abdominal movement are extracted from thermal imaging time series data, and the SpO2 decline slope / amplitude, heart rate oscillation amplitude and pulse wave attenuation index are extracted from pulse signals; if the respiratory interruption is >10 seconds and there is contradictory breathing, combined with wrist / earlobe SpO2 drop ≥4% to determine as OSA event; if the chest and abdominal movement disappears synchronously with the airflow, it is determined as CSA event.

[0123] The cloud end collects relevant data in real time and performs analysis, calculates the apnea hypopnea index (AHI), and generates an OSA severity hierarchical report in combination with the blood oxygen decline slope (ODI).

[0124] The OSA level diagnostic criteria include:

[0125] Mild: AHI 5-14 times / hour, accompanied by a decrease in blood oxygen saturation of 85%-89%;

[0126] Moderate: AHI 15-30 times / hour, accompanied by a decrease in blood oxygen saturation of 80%-84%;

[0127] Severe: AHI ≥ 30 times / hour, accompanied by a decrease in blood oxygen saturation of ≤80%.

[0128] In one specific embodiment, respiratory movement and posture are monitored by infrared thermal imaging, and combined with chest and abdominal contradictory breathing phase difference analysis and pulse wave transit time (PTT) dynamic changes, precise differentiation of OSA and CSA is achieved, effectively distinguishing OSA from postural respiratory disorders, and significantly reducing the risk of misdiagnosis by a single sensor.

[0129] The non-contact infrared monitoring is combined with the wrist-worn / earlobe pulse sensor, the discomfort caused by the traditional body surface electrode attachment is avoided, the use in the family and general medical scenarios is supported, and the night non-invasive monitoring demand is met. The cloud analyzes the oxygen desaturation index (ODI) and the apnea hypopnea index (AHI) in the night, and generates an OSA severity grading report (mild / medium / severe).

[0130] In the present application, compared with the OSA identification by multiple sensors in the prior art, the present application can obtain the chest and abdomen movement / respiration waveform, airflow temperature (change) data through only the time sequence frame data of infrared thermal imaging, and calculate the respiratory ratio to complete the OSA detection and judgment, in addition, the sleep position can be verified through the infrared thermal imaging sleep position data, and the weight dynamic distribution is performed in the auxiliary detection index such as blood oxygen saturation, so as to improve the detection rate.

[0131] The present application discloses an embodiment further provides a computer program product or system, comprising a computer program, which is executed by a processor to realize the method steps described above.

[0132] Figure 2 The present application embodiment provides a schematic diagram of the obstructive sleep apnea syndrome detection system based on multi-modal data, which specifically comprises:

[0133] The acquisition unit acquires the time sequence frame data of infrared thermal imaging of the tester in real time;

[0134] The extraction unit extracts the chest and abdomen position data and the air temperature data of the nose and mouth region based on the infrared thermal imaging time sequence frame data;

[0135] The first data unit calculates the respiration waveform data through the chest and abdomen position data in the time sequence frame data;

[0136] The second data unit calculates the air temperature change curve data through the air temperature data in the time sequence frame data;

[0137] The judgment unit calculates the respiratory ratio by performing phase correlation on the respiration waveform data and the airflow temperature change curve data, and compares the respiratory ratio with a preset threshold value, and determines that the obstructive sleep apnea syndrome occurs when the respiratory ratio is greater than the preset threshold value.

[0138] The present application embodiment provides a computer program product, comprising:

[0139] The infrared thermal imaging module is used for acquiring real-time infrared thermal imaging data;

[0140] The pulse sensor module is used for acquiring pulse data;

[0141] The data processing module is used for processing the real-time infrared thermal imaging data and / or the pulse data; the infrared thermal imaging processing comprises:

[0142] extracting chest and abdomen position data and air temperature data of nose and mouth region based on the infrared thermal imaging time sequence frame data;

[0143] calculating breathing waveform data through chest and abdomen position data in time sequence frame data;

[0144] calculating air temperature change curve data through air temperature data in time sequence frame data;

[0145] calculating breathing ratio through phase correlation of the breathing waveform data and air flow temperature change curve data;

[0146] pulse data processing includes: obtaining pulse data in continuous time, and calculating change amplitude or change slope of the pulse data;

[0147] a determination module determines whether obstructive sleep apnea syndrome occurs by receiving data processed by the data processing module; when the breathing ratio is greater than a preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0148] In one embodiment, the pulse data includes one or more of the following: heart rate, blood oxygen saturation, pulse wave transmission time, and pulse waveform.

[0149] In one embodiment, the determination of the determination module is that, in a time axis, when the breathing ratio is greater than a preset threshold and the blood oxygen saturation decreases by more than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0150] In another embodiment, the determination of the determination module is that, in a time axis, when the breathing ratio is greater than a preset threshold and the heart rate increases by more than a heart rate preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0151] In another embodiment, the determination of the determination module is that, in a time axis, the pulse wave amplitude change is calculated through the pulse wave transmission time and the pulse waveform, and in a time axis, when the breathing ratio is greater than a preset threshold and the pulse wave amplitude is greater than a pulse preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0152] In another embodiment, the determination of the determination module is that, in a time axis, when the breathing ratio is greater than a preset threshold, the heart rate increases by more than a heart rate preset threshold, and the blood oxygen saturation decreases by more than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

[0153] In one embodiment, the data processing module extracts a thermal imaging contour from the infrared imaging time sequence frame data; a classification result of lying on the back or lying on the side is obtained based on the thermal imaging contour classification judgment, when the sleep position is lying on the back, the weight of the oxygen saturation decrease amplitude is increased; when the sleep position is lying on the side, the weight of the chest and abdomen position data is increased. The data with the changed weight is input into the judgment module for judgment.

[0154] In one embodiment, the program product further comprises a voice alarm module for alarming OSA occurrence events.

[0155] When the OSA event is detected, the system determines the user's sleep position (lying on the back / lying on the side) in real time through the infrared thermal imaging module, and fuses the OSA severity classification to push the position adjustment suggestion to trigger the device voice reminder (such as "detecting abnormal breathing, suggesting turning to left lateral position").

[0156] In one embodiment, the program product further comprises a cloud health management platform for generating an OSA severity classification report.

[0157] Based on the data of the night oxygen desaturation slope (ODI) and the apnea hypopnea index (AHI), the OSA severity classification report is generated.

[0158] Figure 3 The computer device provided by the embodiment of the present application specifically comprises:

[0159] The memory is used for storing program instructions, and the processor is used for calling the program instructions, so as to execute any one of the above-mentioned obstructive sleep apnea syndrome detection methods based on multi-modal data.

[0160] The present application discloses an embodiment of a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to execute any one of the above-mentioned obstructive sleep apnea syndrome detection methods based on multi-modal data.

[0161] The verification result of the verification embodiment shows that assigning inherent weights to the indications can improve the performance of the method compared with the default setting. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme. In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units. Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, which can include read only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0162] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium, and the above-mentioned medium storage can be read only memory, magnetic disk or optical disk, etc.

[0163] The computer device provided by the present application has been described in detail above. For those skilled in the art, according to the idea of the embodiment of the present application, there will be changes in specific implementation and application range. In view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A system for detecting obstructive sleep apnea syndrome based on multi-modal data, the system comprising: The application relates to a method for detecting obstructive sleep apnea syndrome, comprising the following steps: an acquisition unit: acquiring time sequence frame data of infrared thermal imaging of a tester in real time; an extraction unit: extracting chest and abdomen position data and airflow temperature data of a nose and mouth area based on the time sequence frame data of the infrared thermal imaging; a first data unit: calculating breathing waveform data through the chest and abdomen position data in the time sequence frame data; the chest and abdomen position data calculation is carried out by extracting displacement of isotherms in a time sequence chest and abdomen thermal image, calculating breathing frequency and amplitude, and obtaining breathing waveform data; a second data unit: calculating airflow temperature change curve data through the airflow temperature data in the time sequence frame data; a judgment unit: performing phase correlation calculation of breathing ratio on the breathing waveform data and the airflow temperature change curve data; the calculation process of the breathing ratio comprises the following steps: performing phase correlation calculation on the breathing waveform data and the airflow temperature change curve data to obtain correlation breathing data, wherein the correlation breathing data is breathing displacement change and airflow temperature change in the same time period; identifying temperature peak value and valley value based on the airflow temperature change curve data, and dividing inhalation and exhalation time intervals based on the temperature peak value and the valley value; obtaining breathing displacement change of the inhalation time interval and breathing displacement change of the exhalation time interval based on the correlation breathing data, and performing breathing verification to obtain verification exhalation time interval and verification inhalation time interval; obtaining temperature peak value duration and valley value duration based on the verification exhalation time interval and the verification inhalation time interval; calculating the breathing ratio based on the temperature peak value duration and the valley value duration; comparing the breathing ratio with a preset threshold value, and determining that obstructive sleep apnea syndrome occurs when the breathing ratio is greater than the preset threshold value; the method further comprises a pulsation feature, and whether the obstructive sleep apnea syndrome occurs is determined by the pulsation feature and the breathing ratio.

2. The system for detection of obstructive sleep apnea syndrome based on multi-modal data according to claim 1, characterized in that, The breathing verification is that the breathing displacement is rising in the inhalation time interval, and the breathing displacement is falling in the exhalation time interval.

3. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 1 wherein, The pulsation feature comprises one or more of the following: heart rate, blood oxygen saturation and pulse wave amplitude.

4. The system for detection of obstructive sleep apnea syndrome based on multi-modal data according to claim 3, characterized in that, Whether the obstructive sleep apnea syndrome occurs is determined by the heart rate and the breathing ratio, heart rate values in continuous time are acquired, and in a time axis, when the breathing ratio is greater than a preset threshold value and the heart rate rising amplitude is greater than a heart rate preset threshold value, it is determined that the obstructive sleep apnea syndrome occurs.

5. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 3 wherein, Whether the obstructive sleep apnea syndrome occurs is determined by the blood oxygen saturation and the breathing ratio, blood oxygen saturation in continuous time is acquired, and in a time axis, when the breathing ratio is greater than a preset threshold value and the blood oxygen saturation falling amplitude is greater than a blood oxygen preset threshold value, it is determined that the obstructive sleep apnea syndrome occurs.

6. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 3 wherein, Whether the obstructive sleep apnea syndrome occurs is determined by the pulse wave amplitude and the breathing ratio, pulse wave conduction time and pulse wave form in continuous time are acquired, pulse wave amplitude change is calculated through the pulse wave conduction time and the pulse wave form, and in a time axis, when the breathing ratio is greater than a preset threshold value and the pulse wave amplitude is greater than a pulse preset threshold value, it is determined that the obstructive sleep apnea syndrome occurs.

7. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 3 wherein, The determination of whether the obstructive sleep apnea syndrome occurs is performed by heart rate, blood oxygen saturation and respiratory ratio. The blood oxygen saturation and heart rate in continuous time are obtained. In a time axis, when the respiratory ratio is greater than a preset threshold, the heart rate rising amplitude is greater than a preset threshold of heart rate, and the blood oxygen saturation falling amplitude is greater than a preset threshold of blood oxygen, it is determined that the obstructive sleep apnea syndrome occurs.

8. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 1 wherein, The obstructive sleep apnea syndrome further comprises body position weight dynamic distribution. The tester contour is extracted from infrared imaging time sequence frame data to obtain thermal imaging contour. The classification result of lying on back or lying on side is obtained based on the thermal imaging contour classification judgment. When the sleep body position is lying on back, the weight of blood oxygen saturation falling amplitude is increased. When the sleep body position is lying on side, the weight of chest and abdomen position data is increased. The data after the body position weight dynamic distribution is used to determine whether the obstructive sleep apnea syndrome occurs.

9. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 1 wherein, The system further comprises obstructive sleep apnea syndrome grading. When it is determined that the obstructive sleep apnea syndrome occurs, the mild, moderate and severe grading of the obstructive sleep apnea syndrome is performed by blood oxygen saturation falling slope and apnea hypopnea index.

10. The system for detection of obstructive sleep apnea syndrome based on multi-modal data according to claim 9, characterized in that, The blood oxygen saturation falling slope is calculated by the blood oxygen saturation change in continuous time.

11. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 9 wherein, The apnea hypopnea index is calculated by airflow temperature change curve data and blood oxygen saturation falling amplitude.

12. The system for detection of obstructive sleep apnea syndrome based on multi-modal data according to claim 11, characterized in that, The apnea hypopnea index calculation process is as follows: the total sleep time is obtained; the apnea frequency is extracted from the airflow temperature change curve data; the low ventilation frequency is obtained by discriminating the airflow temperature change curve data and the blood oxygen saturation falling amplitude; the apnea hypopnea index is calculated by the total sleep time, the apnea frequency and the low ventilation frequency.

13. The system for detection of obstructive sleep apnea syndrome based on multi-modal data of claim 1, wherein, The judgment unit is replaced by: the determination of whether the obstructive sleep apnea syndrome occurs is performed based on the respiratory waveform data and the airflow temperature change curve data. The correlation respiratory data is obtained by phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data. When the respiratory airflow interruption duration in the correlation respiratory data is greater than a preset threshold of respiration and the chest and abdomen perform contradictory movement, it is determined that the obstructive sleep apnea syndrome occurs.

14. The system for detection of obstructive sleep apnea syndrome based on multi-modal data according to claim 13, characterized in that, The judgment further comprises one or more of the following: heart rate, blood oxygen saturation.

15. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 13 wherein, The judgment is completed by the respiratory waveform data and the airflow temperature change curve data, heart rate and blood oxygen saturation. The heart rate rising amplitude is calculated by the heart rate in continuous time. The falling amplitude is calculated by the blood oxygen saturation in continuous time. When the respiratory airflow interruption duration in the correlation respiratory data is greater than a preset threshold of respiration, the chest and abdomen perform contradictory movement, the heart rate rising amplitude is greater than a preset threshold of heart rate, and the blood oxygen saturation falling amplitude is greater than a preset threshold of blood oxygen, it is determined that the obstructive sleep apnea syndrome occurs.

16. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 14 wherein, The judgment is performed on the respiratory airflow, chest and abdomen movement, heart rate rising amplitude and blood oxygen falling amplitude in the same time period.

17. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 13 wherein, The judgment also includes body position dynamic weight distribution, extracting thermal imaging profile to judge sleep body position, when the sleep body position is supine, the weight of the blood oxygen saturation drop range is increased; when the sleep body position is lateral, the weight of the chest and abdominal position data is increased; the data after the body position dynamic weight distribution is judged.

18. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 13 wherein, The system also includes central sleep apnea (CSA) differentiation, when the relevant respiratory data is interrupted for longer than the preset threshold of respiratory and the chest and abdomen stop moving within the interruption time, it is judged that CSA occurs.

19. The system for detection of obstructive sleep apnea syndrome based on multi-modal data as claimed in claim 1 wherein, The system also includes: An infrared thermal imaging module for acquiring real-time infrared thermal imaging data; A pulse sensor module for acquiring pulse data.

20. A computer device comprising a memory, a processor, and a computer program or instructions stored on the memory, wherein, The computer program or instructions are executed by the processor to realize the detection method of obstructive sleep apnea syndrome based on multi-modal data, including: S1, real-time acquisition of time sequence frame data of tester infrared thermal imaging; S2, extracting chest and abdominal position data and airflow temperature data of nose and mouth area based on the infrared thermal imaging time sequence frame data; S3, calculating the respiratory waveform data through the chest and abdominal position data in the time sequence frame data; the chest and abdominal position data calculation is to extract the displacement of the isotherm in the time sequence chest and abdominal thermal image, calculate the respiratory frequency and amplitude, and obtain the respiratory waveform data; S4, calculating the airflow temperature change curve data through the airflow temperature data in the time sequence frame data; S5, calculating the respiratory ratio by phase correlation of the respiratory waveform data and the airflow temperature change curve data; the calculation process of the respiratory ratio includes: Calculating the relevant respiratory data by phase correlation of the respiratory waveform data and the airflow temperature change curve data, which is the respiratory displacement change and airflow temperature change in the same time period; Identifying the temperature peak and valley based on the airflow temperature change curve data, and dividing the inspiration and expiration time interval based on the temperature peak and valley; Obtaining the respiratory displacement change of the inspiration time interval and the respiratory displacement change of the expiration time interval based on the relevant respiratory data, and performing respiratory verification to obtain the verification expiration time interval and the inspiration time interval; Obtaining the temperature peak duration and the valley duration based on the verification expiration time interval and the inspiration time interval; Calculating the respiratory ratio based on the temperature peak duration and the valley duration; Comparing the respiratory ratio with the preset threshold, when the respiratory ratio is greater than the preset threshold, it is determined that obstructive sleep apnea syndrome occurs; further including pulsation characteristics, determining whether obstructive sleep apnea syndrome occurs by the pulsation characteristics and the respiratory ratio.

21. The computer device of claim 20, wherein, Including: The respiratory verification is that the respiratory displacement is rising in the inspiration time interval, and the respiratory displacement is falling in the expiration time interval.

22. The computer device of claim 20, wherein, The pulsation characteristics include one or more of the following: heart rate, blood oxygen saturation, and pulse amplitude.

23. The computer device of claim 22, wherein, Determine whether obstructive sleep apnea syndrome occurs by heart rate and respiratory ratio, obtain heart rate values in continuous time, and in one time axis, when the respiratory ratio is greater than the preset threshold and the heart rate rising amplitude is greater than the preset threshold of heart rate, it is determined that obstructive sleep apnea syndrome occurs.

24. The computer device of claim 22, wherein, The determination of whether the obstructive sleep apnea syndrome occurs is made by blood oxygen saturation and respiratory ratio, the blood oxygen saturation in continuous time is obtained, and in a time axis, when the respiratory ratio is greater than a preset threshold and the blood oxygen saturation decreases by a blood oxygen preset threshold, it is determined that the obstructive sleep apnea syndrome occurs.

25. The computer device of claim 22, wherein, The determination of whether the obstructive sleep apnea syndrome occurs is made by pulse wave amplitude and respiratory ratio, the pulse wave transmission time and the pulse waveform in continuous time are obtained, the pulse wave amplitude change is calculated by the pulse wave transmission time and the pulse waveform, and in a time axis, when the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than a pulse preset threshold, it is determined that the obstructive sleep apnea syndrome occurs.

26. The computer device of claim 22, wherein, The determination of whether the obstructive sleep apnea syndrome occurs is made by heart rate, blood oxygen saturation and respiratory ratio, the blood oxygen saturation and heart rate in continuous time are obtained, and in a time axis, when the respiratory ratio is greater than a preset threshold, the heart rate increases by a heart rate preset threshold, and the blood oxygen saturation decreases by a blood oxygen preset threshold, it is determined that the obstructive sleep apnea syndrome occurs.

27. The computer device of claim 20, wherein, The obstructive sleep apnea syndrome further comprises body position weight dynamic allocation, the tester contour is extracted from the infrared imaging time sequence frame data to obtain a thermal imaging contour, a classification result of lying on the back or lying on the side is obtained based on the thermal imaging contour classification judgment, when the sleep body position is lying on the back, the weight of the blood oxygen saturation decrease amplitude is increased; When the sleep body position is lying on the side, the weight of the chest and abdominal position data is increased; whether the obstructive sleep apnea syndrome occurs is determined based on the data after the body position weight dynamic allocation.

28. The computer device of claim 20, wherein, The method further comprises obstructive sleep apnea syndrome grading, when it is determined that the obstructive sleep apnea syndrome occurs, the mild, moderate and severe grading of the obstructive sleep apnea syndrome is made by the blood oxygen saturation decrease slope and the apnea hypopnea index.

29. The computer device of claim 28, wherein, The blood oxygen saturation decrease slope is calculated by the blood oxygen saturation change in continuous time.

30. The computer device of claim 28, wherein, The apnea hypopnea index is calculated by the airflow temperature change curve data and the blood oxygen saturation decrease amplitude.

31. The computer device of claim 30, wherein, The apnea hypopnea index calculation process is: obtaining the total sleep time; the apnea frequency is extracted from the airflow temperature change curve data; the low ventilation frequency is discriminated by the airflow temperature change curve data and the blood oxygen saturation decrease amplitude; the apnea hypopnea index is calculated by the total sleep time, the apnea frequency and the low ventilation frequency.

32. The computer device of claim 20, wherein, S5 is replaced by: whether the obstructive sleep apnea syndrome occurs is judged based on the respiratory waveform data and the airflow temperature change curve data, the correlation respiratory data is calculated by the phase correlation of the respiratory waveform data and the airflow temperature change curve data, when the respiratory airflow interruption time in the correlation respiratory data is greater than a respiratory preset threshold and the chest and abdominal contradictory movement occurs, it is determined that the obstructive sleep apnea syndrome occurs.

33. The computer device of claim 32, wherein, The judgment further comprises one or more of the following: heart rate, blood oxygen saturation.

34. The computer device of claim 32, wherein, The judgment is completed by the respiratory waveform data and the airflow temperature change curve data, the heart rate, and the blood oxygen saturation. The heart rate rising amplitude is calculated for the heart rate in a continuous time. The falling amplitude is calculated for the blood oxygen saturation in a continuous time. When the length of the respiratory airflow interruption in the related respiratory data is greater than a preset threshold of the respiration, the chest and abdominal movements are contradictory, the heart rate rising amplitude is greater than a preset threshold of the heart rate, and the blood oxygen saturation falling amplitude is greater than a preset threshold of the blood oxygen, it is judged that the obstructive sleep apnea syndrome occurs.

35. The computer device of claim 34, wherein, The judgment is performed on the respiratory airflow, the chest and abdominal movements, the heart rate rising amplitude, and the blood oxygen falling amplitude in the same time period.

36. The computer device of claim 32, wherein, The judgment further includes dynamic weight distribution of the body position. The sleep body position is judged by extracting the thermal imaging profile. When the sleep body position is lying on the back, the weight of the blood oxygen saturation falling amplitude is increased. When the sleep body position is lying on the side, the weight of the chest and abdominal position data is increased. The data after the dynamic weight distribution of the body position is used for judgment.

37. The computer device of claim 32, wherein, The method further includes central sleep apnea (CSA) differentiation. When the length of the respiratory airflow interruption in the related respiratory data is greater than a preset threshold of the respiration and the chest and abdominal movements stop in the interruption length, it is judged that the CSA occurs.

38. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to realize the obstructive sleep apnea syndrome detection method based on the multi-modal data, which includes: S1, real-time acquisition of time sequence frame data of infrared thermal imaging of a tester; S2, extraction of chest and abdominal position data and airflow temperature data of a nose and mouth area based on the infrared thermal imaging time sequence frame data; S3, calculation of respiratory waveform data by the chest and abdominal position data in the time sequence frame data; the chest and abdominal position data calculation is performed by extracting the displacement of the isotherm in the time sequence chest and abdominal thermal image, calculating the respiratory frequency and amplitude, and obtaining the respiratory waveform data; S4, calculation of airflow temperature change curve data by the airflow temperature data in the time sequence frame data; S5, phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data to obtain a respiratory ratio; the calculation process of the respiratory ratio includes: phase correlation calculation of the respiratory waveform data and the airflow temperature change curve data to obtain related respiratory data; the related respiratory data is the respiratory displacement change and the airflow temperature change in the same time period; identification of temperature peak and valley values based on the airflow temperature change curve data; the inspiration and expiration time intervals are divided based on the temperature peak and valley values; obtaining of the respiratory displacement change in the inspiration time interval and the respiratory displacement change in the expiration time interval based on the related respiratory data; respiratory verification is performed to obtain verified inspiration and expiration time intervals; obtaining of temperature peak duration and valley duration based on the verified inspiration and expiration time intervals; calculation of the respiratory ratio based on the temperature peak duration and the valley duration; comparison of the respiratory ratio with a preset threshold; when the respiratory ratio is greater than the preset threshold, it is determined that the obstructive sleep apnea syndrome occurs; further including a pulsation feature; whether the obstructive sleep apnea syndrome occurs is determined by the pulsation feature and the respiratory ratio.

39. The computer-readable storage medium of claim 38, wherein, including: the respiratory verification is that the respiratory displacement is rising in the inspiration time interval and the respiratory displacement is falling in the expiration time interval.

40. The computer-readable storage medium of claim 38, wherein, The pulsation features include one or more of the following: heart rate, blood oxygen saturation, pulse wave amplitude.

41. The computer-readable storage medium of claim 40, wherein, The occurrence of obstructive sleep apnea syndrome is determined by heart rate and respiratory ratio, heart rate values in continuous time are obtained, and when the respiratory ratio is greater than a preset threshold and the heart rate rising amplitude is greater than a heart rate preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

42. The computer-readable storage medium of claim 40, wherein, The occurrence of obstructive sleep apnea syndrome is determined by blood oxygen saturation and respiratory ratio, blood oxygen saturation in continuous time is obtained, and when the respiratory ratio is greater than a preset threshold and the blood oxygen saturation falling amplitude is greater than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

43. The computer-readable storage medium of claim 40, wherein, The occurrence of obstructive sleep apnea syndrome is determined by pulse wave amplitude and respiratory ratio, pulse wave transmission time and pulse wave form in continuous time are obtained, the pulse wave amplitude change is calculated by the pulse wave transmission time and the pulse wave form, and when the respiratory ratio is greater than a preset threshold and the pulse wave amplitude is greater than a pulse preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

44. The computer-readable storage medium of claim 40, wherein, The occurrence of obstructive sleep apnea syndrome is determined by heart rate, blood oxygen saturation and respiratory ratio, blood oxygen saturation and heart rate in continuous time are obtained, and when the respiratory ratio is greater than a preset threshold, the heart rate rising amplitude is greater than a heart rate preset threshold, and the blood oxygen saturation falling amplitude is greater than a blood oxygen preset threshold, it is determined that obstructive sleep apnea syndrome occurs.

45. The computer-readable storage medium of claim 38, wherein, The obstructive sleep apnea syndrome further includes dynamic allocation of body position weight, tester contour extraction is performed on infrared imaging time sequence frame data to obtain thermal imaging contour; a classification result of supine or lateral lying is obtained based on the thermal imaging contour classification judgment, when the sleep body position is supine, the weight of the blood oxygen saturation falling amplitude is increased; When the sleep body position is lateral lying, the weight of the chest and abdomen position data is increased; whether obstructive sleep apnea syndrome occurs is determined based on the data after dynamic allocation of body position weight.

46. The computer-readable storage medium of claim 38, wherein, The method further includes obstructive sleep apnea syndrome grading, when it is determined that obstructive sleep apnea syndrome occurs, the mild, moderate and severe grading of obstructive sleep apnea syndrome is performed by blood oxygen saturation falling slope and apnea hypopnea index.

47. The computer-readable storage medium of claim 46, wherein, The blood oxygen saturation falling slope is calculated by the change of blood oxygen saturation in continuous time.

48. The computer-readable storage medium of claim 46, wherein, The apnea hypopnea index is calculated by the airflow temperature change curve data and the blood oxygen saturation falling amplitude.

49. The computer-readable storage medium of claim 48, wherein, The apnea hypopnea index calculation process is: obtaining total sleep time; the apnea frequency is extracted by the airflow temperature change curve data; the low ventilation frequency is discriminated by the airflow temperature change curve data and the blood oxygen saturation falling amplitude; the apnea hypopnea index is calculated by the total sleep time, the apnea frequency and the low ventilation frequency.

50. The computer-readable storage medium of claim 38, wherein, S5 is replaced by: judging whether obstructive sleep apnea syndrome occurs based on the respiratory waveform data and the airflow temperature change curve data, calculating the correlation respiratory data by phase correlation of the respiratory waveform data and the airflow temperature change curve data, and when the length of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold and the chest and abdomen perform contradictory movements, judging that obstructive sleep apnea syndrome occurs.

51. The computer-readable storage medium of claim 50, wherein, The judging further includes one or more of the following: heart rate, blood oxygen saturation.

52. The computer-readable storage medium of claim 50, wherein, The judging is completed by the respiratory waveform data and the airflow temperature change curve data, the heart rate, and the blood oxygen saturation, calculating the heart rate rising amplitude of the heart rate in continuous time, calculating the blood oxygen saturation falling amplitude of the blood oxygen saturation in continuous time, when the length of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold, the chest and abdomen perform contradictory movements, the heart rate rising amplitude is greater than a preset heart rate threshold, and the blood oxygen saturation falling amplitude is greater than a preset blood oxygen threshold, judging that obstructive sleep apnea syndrome occurs.

53. The computer-readable storage medium of claim 52, wherein, The judging is performed on the respiratory airflow, the chest and abdomen movements, the heart rate rising amplitude, and the blood oxygen falling amplitude in the same time period.

54. The computer-readable storage medium of claim 50, wherein, The judging further includes body position dynamic weight distribution, extracting a thermal imaging contour to judge a sleep body position, increasing the weight of the blood oxygen saturation falling amplitude when the sleep body position is supine, increasing the weight of the chest and abdomen position data when the sleep body position is lateral, and judging by the data after the body position dynamic weight distribution.

55. The computer-readable storage medium of claim 50, wherein, The method further includes central sleep apnea (CSA) differentiation, and when the length of respiratory airflow interruption in the correlation respiratory data is greater than a preset respiratory threshold and the chest and abdomen stop moving in the length of interruption, judging that CSA occurs.

Images