Multimodal sleep apnea detection method, apparatus, device, and medium

Abstract

This invention relates to the field of intelligent decision-making technology, and discloses a multimodal sleep apnea detection method, device, equipment, and medium. The method includes: acquiring radar signals and the target user's blood oxygenation signal and sleep posture signal; fusing the radar signal, blood oxygenation signal, and sleep posture signal into a multimodal continuous signal; segmenting the multimodal continuous signal window to obtain signal segments; extracting multimodal physiological fusion features from the signal segments; classifying the signal segments into respiratory event types based on the multimodal physiological fusion features; applying rule constraints to correct the respiratory event types to obtain corrected respiratory event types; and using the corrected respiratory event types as the sleep apnea detection result. By classifying the signal segments into respiratory event types, the objectivity and accuracy of the detection results are improved; and by applying rule constraints to correct the respiratory event types, the detection results are made more in line with clinical standards, improving the reliability and practicality of the results.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to a multimodal sleep apnea detection method, device, equipment, and medium. Background Technology

[0002] Sleep apnea syndrome is a common sleep-related breathing disorder, mainly including obstructive sleep apnea, central sleep apnea, and mixed sleep apnea. Long-term untreated sleep apnea will significantly increase the risk of cardiovascular and cerebrovascular diseases such as hypertension, coronary heart disease, and stroke.

[0003] Currently, traditional sleep apnea detection methods mostly rely on environmental sensing devices such as radio frequency sensors. Radio frequency sensing solutions often use a single-channel signal processing mode, but this method can only achieve preliminary monitoring of respiratory rhythm and lacks joint discrimination of respiratory changes, blood oxygen changes and body position information. It also has limited ability to classify different types of apnea, resulting in insufficient detection accuracy.

[0004] Therefore, in the face of the increasing demand for sleep apnea detection, current sleep apnea detection methods urgently need to be improved to address the problem of insufficient accuracy in existing methods. Summary of the Invention

[0005] This invention provides a multimodal sleep apnea detection method, device, equipment, and medium, the main purpose of which is to solve the problem of insufficient accuracy in sleep apnea detection results.

[0006] To achieve the above objectives, the present invention provides a multimodal sleep apnea detection method, comprising: The radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, are collected and fused into a multimodal continuous signal. The multimodal continuous signal is divided into time windows to obtain the signal segment corresponding to each window; Extracting multimodal physiological fusion features from signal fragments; Based on the multimodal physiological fusion characteristics, the respiratory event type of the signal segment is classified to obtain the respiratory event type corresponding to the signal segment; The breathing event types are modified by applying rule constraints to obtain modified breathing event types, which are then used as the sleep apnea detection results.

[0007] The present invention also provides a multimodal sleep apnea detection device, the device comprising: The multimodal continuous signal fusion module is used to collect the radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, and fuse the radar signal, blood oxygen signal and sleep position signal into a multimodal continuous signal. The signal segmentation module is used to segment multimodal continuous signals into time windows to obtain signal segments corresponding to each window. A multimodal physiological fusion feature extraction module is used to extract multimodal physiological fusion features from signal segments; The respiratory event type classification module is used to classify the respiratory event type of signal segments based on multimodal physiological fusion features, and obtain the respiratory event type corresponding to the signal segment; The revised breathing event type module is used to modify the breathing event type according to rules, obtain the revised breathing event type, and use the revised breathing event type as the sleep apnea detection result.

[0008] The present invention also provides an electronic device, the electronic device comprising: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory stores a computer program that can be executed by at least one processor, such that the at least one processor can perform the multimodal sleep apnea detection method described above.

[0009] The present invention also provides a computer-readable medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described multimodal sleep apnea detection method.

[0010] In this invention, radar signals, blood oxygen signals, and sleep posture signals are fused into a multimodal continuous signal, achieving complementary multimodal physiological data and providing a comprehensive and reliable data foundation for accurate detection. By segmenting the multimodal continuous signal into time windows, long-sequence signals are broken down into standardized short units, facilitating segment-by-segment analysis of respiratory events and improving detection efficiency and accuracy. Extracting multimodal physiological fusion features from signal segments overcomes the limitation of traditional single-channel signals that can only monitor respiratory rhythm, improving the ability to differentiate between different types of apnea. Classifying signal segments by respiratory event type avoids the one-sidedness of traditional single-channel signal discrimination, reduces misclassification, and improves the objectivity and accuracy of detection results. By applying rule constraints to the respiratory event types, misclassifications are eliminated and classifications are corrected, making the detection results more consistent with clinical standards and improving the reliability and practicality of the results. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a multimodal sleep apnea detection method according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a process for extracting multimodal physiological fusion features from a signal segment according to an embodiment of the present invention; Figure 3 This is a functional block diagram of a multimodal sleep apnea detection device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of an electronic device for implementing a multimodal sleep apnea detection method according to an embodiment of the present invention.

[0012] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0014] To address the shortcomings of existing multimodal sleep apnea detection methods, which only provide preliminary monitoring of respiratory rhythm and lack joint discrimination of respiratory changes, blood oxygenation changes, and body position information, as well as limited ability to classify different types of apnea, resulting in insufficient detection accuracy, this application provides a multimodal sleep apnea detection method. This method improves the objectivity and accuracy of detection results by classifying signal segments into respiratory event types and applies rule constraints to correct respiratory event types, making the detection results more in line with clinical standards and improving the reliability and practicality of the results.

[0015] Reference Figure 1 The diagram shown is a flowchart illustrating a multimodal sleep apnea detection method according to an embodiment of this application. In this embodiment, the multimodal sleep apnea detection method includes: S1. Collect the radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, and fuse the radar signal, blood oxygen signal and sleep position signal into a multimodal continuous signal.

[0016] In this embodiment of the invention, the radar signal refers to the millimeter-wave radar continuously transmitting signals to the user's chest cavity, receiving echoes, and obtaining an intermediate frequency signal containing information on chest cavity micro-movements; the blood oxygen signal refers to the physiological signal reflecting the saturation of oxyhemoglobin in the blood collected by a pulse oximeter; and the sleep posture signal refers to the three-dimensional posture data reflecting the user's sleep posture collected by a posture sensor (such as an accelerometer).

[0017] In this embodiment of the invention, the radar signal may be chest micro-movement, respiratory waveform, heartbeat waveform, body movement index, etc., the blood oxygen signal may be blood oxygen saturation, pulse rate, perfusion index, etc., and the sleep position signal may include supine, left lateral, right lateral, prone, sitting up / awakening, etc.

[0018] In this embodiment of the invention, radar signals, blood oxygen signals, and sleep position signals are fused into a multimodal continuous signal, including: timestamping the radar signals, blood oxygen signals, and sleep position signals respectively to obtain a radar signal sequence, a blood oxygen signal sequence, and a sleep position signal sequence with timestamps; resampling the blood oxygen signal sequence and the sleep position signal sequence according to the sampling frequency of the radar signal sequence to obtain a sampled blood oxygen signal sequence and a sampled sleep position signal sequence with a unified sampling frequency; aligning the radar signal sequence, the sampled blood oxygen signal sequence, and the sampled sleep position signal sequence by time points to generate a time-synchronized multimodal signal sequence; and generating a multimodal continuous signal according to the multimodal signal sequence in chronological order.

[0019] In this embodiment of the invention, when radar signals are collected by a radar sensor, the current system time is recorded for each frame of echo data received; when blood oxygen signals are collected by a pulse oximeter, the corresponding time is recorded for each blood oxygen saturation value measured; and when sleep position signals are collected by a body position sensor, the current timestamp is recorded for each triaxial acceleration sample collected, thus obtaining a radar signal sequence, a blood oxygen signal sequence, and a sleep position signal sequence with timestamps.

[0020] Furthermore, using the sampling frequency of the radar signal sequence as a benchmark, for blood oxygen signal sequences or sleep position signal sequences with sampling rates lower than those of the radar signal sequence, an interpolation method is used to generate new data points between the blood oxygen signal sequences or sleep position signal sequences, thereby increasing their sampling frequency to the sampling frequency of the radar signal sequence; for blood oxygen signal sequences or sleep position signal sequences with sampling rates higher than those of the radar signal sequence, a decimation method is used to reduce their sampling frequency to the sampling frequency of the radar signal sequence, resulting in a sampled blood oxygen signal sequence and a sampled sleep position signal sequence with the same sampling rate as the radar signal sequence.

[0021] Furthermore, during the time point alignment of the radar signal sequence, the sampled blood oxygen signal sequence, and the sampled sleep position signal sequence, each timestamp of the radar signal sequence is traversed, and the sampled blood oxygen signal or sampled sleep position signal that is closest to that timestamp is found in the sampled blood oxygen signal sequence and the sampled sleep position signal sequence. The time difference between the timestamp of the found sampled blood oxygen signal or sampled sleep position signal and the timestamp of its corresponding radar signal sequence is calculated.

[0022] Furthermore, if the time difference does not exceed the preset time difference threshold, the sampled blood oxygen signal or sampled sleep position signal is used as the time-aligned blood oxygen signal value or position signal value. If the time difference exceeds the preset time difference threshold, several sampling points before and after the timestamp of the radar signal sequence are selected, centered on the timestamp of the radar signal sequence or the sleep position signal sequence. The blood oxygen signal value or sleep position signal value corresponding to the timestamp of the center is calculated by linear interpolation, forming a time-synchronized multimodal signal sequence, that is, each time point simultaneously contains data in three dimensions: radar signal, blood oxygen signal and position signal.

[0023] Furthermore, the radar signal value, blood oxygen signal value, and body position signal value at each time point are used as the three channel data for that time point. The three channel data are arranged sequentially in chronological order to form a multimodal continuous signal with a total length equal to the number of time points, and each point contains three channel data.

[0024] In this embodiment of the invention, by marking the timestamps of radar signals, blood oxygen signals, and sleep position signals respectively, a benchmark is provided for subsequent time sequence alignment, avoiding time sequence chaos of multimodal signals; the blood oxygen signal sequence and sleep position signal sequence are resampled according to the sampling frequency of the radar signal sequence to eliminate the difference in sampling rate between different sensors, which facilitates subsequent point-by-point matching and fusion; by aligning the radar signal sequence, the sampled blood oxygen signal sequence, and the sampled sleep position signal sequence at the same time point, the radar, blood oxygen, and position data corresponding to the same moment are truly synchronized, solving the problem of acquisition start deviation and improving the accuracy of physiological feature correlation.

[0025] S2. Divide the multimodal continuous signal into time windows to obtain the signal segment corresponding to each window.

[0026] In this embodiment of the invention, a signal segment refers to multiple independent time-series signal units obtained by dividing a multimodal continuous signal into segments of a fixed time length.

[0027] In this embodiment of the invention, a multimodal continuous signal is divided into time windows to obtain a signal segment corresponding to each window. This includes: determining the starting time point of the window according to a preset window length and sliding step size, and collecting the starting time points of the window into a cutting position set; extracting signal segments from the multimodal continuous signal according to the cutting position set to obtain preliminary signal segments; and verifying the validity of the preliminary signal segments according to a preset duration threshold to obtain the signal segment corresponding to each window.

[0028] In this embodiment of the invention, the window length refers to the fixed time length of each segmented signal (e.g., 10 seconds), the sliding step size refers to the time interval between the starting points of two adjacent windows (e.g., 2 seconds), and the duration threshold refers to a preset value, usually set as a preset proportion of the window length. For example, if the window length is 30 seconds and the proportion is 80%, then the duration threshold is 24 seconds.

[0029] In this embodiment of the invention, starting from the starting time point of the multimodal continuous signal, the window start time point of each window is calculated sequentially with a sliding step size as the interval. All window start time points are collected into an ordered set in chronological order, and this ordered set is used as the cutting position set.

[0030] For example, the window start time points are 0 seconds, 5 seconds, 10 seconds, 15 seconds... and so on, until the last window start time point is obtained.

[0031] Furthermore, the starting time point of each window in the cutting position set is traversed, and the starting time point of the window is taken as the starting point of the window. Combined with the preset window length, the signal data within the corresponding time range is extracted from the multimodal continuous signal. The above extraction operation is performed on all the starting time points of the windows in the cutting position set in sequence to obtain the preliminary signal segment.

[0032] For example, if the window starts at 0 seconds and the window length is 30 seconds, then the signal segment from 0 seconds to 30 seconds is captured; if the window starts at 5 seconds, then the signal segment from 5 seconds to 35 seconds is captured.

[0033] Furthermore, the actual effective duration of each preliminary signal segment is checked against a preset duration threshold. If the actual effective duration of the preliminary signal segment is lower than the preset duration threshold, it is determined to be an invalid segment and is removed. If it reaches or exceeds the duration threshold, it is determined to be a valid segment and is retained. The retained valid segments are used as the signal segments corresponding to each window.

[0034] In this embodiment of the invention, the starting time point of the window is determined by the window length and the sliding step size to ensure uniform segmentation and traceable timing; the validity of the initial signal segments is verified according to the preset duration threshold, and segments that are insufficient in length or incomplete or abnormal are removed to ensure that each signal segment has a uniform format and reliable quality, thereby improving the stability of subsequent classification.

[0035] S3. Extract multimodal physiological fusion features from signal fragments.

[0036] like Figure 2As shown, in this embodiment of the invention, extracting multimodal physiological fusion features from signal segments includes: S21, performing channel separation on the signal segments to obtain radar signal sub-segments, blood oxygen signal sub-segments, and body position signal sub-segments; S22, extracting respiratory features, chest and abdominal movement features, heart rate features, and body movement index from the radar signal sub-segments; S23, combining the respiratory features, chest and abdominal movement features, heart rate features, and body movement index into a radar feature vector; S24, extracting static blood oxygen features and dynamic blood oxygen features from the blood oxygen signal sub-segments, and combining the static blood oxygen features and dynamic blood oxygen features into a blood oxygen statistical feature vector; S25, extracting sleep posture features from the body position signal sub-segments, and converting the sleep posture features into a body position feature vector; S26, performing correlation analysis on the radar signal sub-segments and blood oxygen signal sub-segments to generate a respiratory-blood oxygen correlation feature vector; S27, concatenating the radar feature vector, blood oxygen statistical feature vector, body position feature vector, and respiratory-blood oxygen correlation feature vector into a multimodal physiological fusion feature.

[0037] In this embodiment of the invention, respiratory characteristics include mean respiratory amplitude, variance of respiratory amplitude, and respiratory rate; chest and abdominal movement characteristics include chest and abdominal movement amplitude ratio and chest and abdominal movement phase difference; heart rate characteristics include heart rate, heart rate variability, and mean heart rate amplitude; static blood oxygenation characteristics include mean blood oxygenation and minimum blood oxygenation; dynamic blood oxygenation characteristics include blood oxygenation depth, duration of blood oxygenation decline, and slope of blood oxygenation decline; and sleep posture characteristics include dominant posture, proportion of supine position, and markers of posture change.

[0038] In this embodiment of the invention, each sampling time in the signal segment is traversed, and the values ​​of the first channel corresponding to each sampling time are arranged in chronological order to form a radar signal sub-segment; the values ​​of the second channel are arranged in chronological order to form a blood oxygen signal sub-segment; and the values ​​of the third channel are arranged in chronological order to form a body position signal sub-segment.

[0039] Furthermore, a range-dimensional FFT (Fast Fourier Transform) is performed on the radar signal sub-segment to obtain a range-time spectrum. The signal energy or phase change amplitude of all range units on the range-time spectrum is analyzed, and the range unit with the most significant change in signal energy or phase change amplitude is extracted. The extracted range unit is used as the range unit where the chest cavity is located (i.e., the range sampling point corresponding to the position of the human chest cavity). The phase signal of the range unit is extracted, and the phase signal is de-wound. According to the preset radar signal wavelength, the de-wound phase signal is converted into a displacement signal proportional to the displacement of the chest cavity (i.e., the displacement signal of the chest cavity region).

[0040] Among them, the distance-time spectrum refers to a two-dimensional time-series spectrum formed by using time as the horizontal axis and distance units (distance) as the vertical axis, where the amplitude / energy of each pixel represents the echo intensity of the target at that time and distance.

[0041] Furthermore, the distance cells within a preset range of the distance cell containing the thoracic cavity are searched on the distance-time spectrum. The signal energy or phase change amplitude of the searched distance cells is determined, and the distance cell with the most significant signal energy or phase change amplitude is extracted as the distance cell containing the abdominal cavity (i.e., the distance sampling point corresponding to the location of the human abdominal cavity). The phase signal of the distance cell containing the abdominal cavity is extracted, and the phase signal of the distance cell containing the abdominal cavity is de-wound. Based on the radar signal wavelength, the phase signal of the de-wound distance cell containing the abdominal cavity is converted into a displacement signal proportional to the displacement of the abdominal cavity (i.e., the displacement signal of the abdominal cavity region).

[0042] Furthermore, a bandpass filter with a passband range of 0.1Hz to 0.5Hz is used to filter the displacement signal in the thoracic region, removing high-frequency heartbeat components and low-frequency body motion drift, and the output is the respiratory waveform; a bandpass filter with a passband range of 0.8Hz to 2.0Hz is used to filter the displacement signal in the thoracic region, removing respiratory signals and noise, and the output is the heartbeat waveform.

[0043] Furthermore, the peak and trough amplitudes of the respiratory waveform are detected according to the preset respiratory cycle. The peak amplitude is subtracted from the trough amplitude to obtain the respiratory amplitude for each respiratory cycle. The average of the respiratory amplitudes for all respiratory cycles is calculated to obtain the mean respiratory amplitude. Then, the sum of squares of the differences between the respiratory amplitude and the mean respiratory amplitude for each respiratory cycle is calculated and averaged to obtain the variance of the respiratory amplitude.

[0044] Furthermore, the total number of respiratory cycles is counted, and the total number of respiratory cycles is divided by the window duration of the signal segment corresponding to the respiratory signal to obtain the respiratory rate.

[0045] Furthermore, the ratio of chest-abdominal motion amplitude is calculated based on the displacement signals of the thoracic and abdominal regions. This ratio can be expressed by the following formula:

[0046] in, Indicates in The ratio of chest and abdominal movement amplitude at any given moment, Indicates in Displacement signal of the thoracic region at any given time. Indicates in Displacement signal of the abdominal region at time t, This represents the root mean square operation. Indicates the sampling time.

[0047] Furthermore, the phase difference of chest and abdominal movements can be expressed by the following formula:

[0048] in, Indicates in Phase difference of chest and abdominal movements at any given moment Represents the Hilbert transform. This represents the arctangent function.

[0049] Furthermore, the displacement signal of the chest cavity region is decomposed to obtain sinusoidal components of different frequencies and their corresponding energy values. The energy values ​​of all sinusoidal components with frequencies higher than a preset threshold are integrated, and the sum of the integrated energy values ​​is used as the body movement index. Optionally, when the body movement index exceeds a preset body movement threshold, the target user is determined to be in a state of large-scale body rolling. When the body movement index does not exceed the preset body movement threshold, the target user is determined to be in a resting state.

[0050] Furthermore, peak detection is performed on the heartbeat waveform. Based on the detection results, the peak value of each heartbeat in the heartbeat waveform is located, the time interval between adjacent peak values ​​is calculated, and a heartbeat interval sequence is obtained. The average heartbeat interval is calculated from the heartbeat interval sequence, and then 60 is divided by the average heartbeat interval to obtain the heart rate (unit: beats / minute).

[0051] Furthermore, heartbeat intervals below a preset heartbeat interval threshold in the heartbeat interval sequence are removed to obtain a valid heartbeat interval sequence. The average valid heartbeat interval of the valid heartbeat interval sequence is calculated, and the standard deviation and variance of the valid heartbeat interval are calculated based on the average valid heartbeat interval. The standard deviation and variance of the valid heartbeat interval are combined to form heart rate variability. The peak amplitude and trough amplitude of the heartbeat waveform are located according to the preset heartbeat cycle, and the difference between the peak amplitude and the trough amplitude is calculated. The average value of the difference for all heartbeat cycles is analyzed to obtain the mean heartbeat amplitude.

[0052] Furthermore, the average value of blood oxygen saturation within the blood oxygen signal sub-segment is analyzed to obtain the average blood oxygen value; the lowest blood oxygen saturation within the blood oxygen signal sub-segment is taken as the minimum blood oxygen value.

[0053] Furthermore, median filtering is applied to the blood oxygen signal segments to obtain the filtered blood oxygen signal. Then, the dynamic baseline of the filtered blood oxygen signal is extracted using the moving average method. Based on the dynamic baseline, the blood oxygen saturation depth at each sampling time is calculated. The blood oxygen saturation depth can be expressed by the following formula:

[0054] in, Indicates in The depth of blood oxygenation at any given moment. Indicates in Dynamic baseline at any given time, Indicates in Filtered blood oxygen signal at any given time; when The event was determined to be a decrease in blood oxygenation.

[0055] Furthermore, the maximum value of the blood oxygen descent depth within the blood oxygen signal sub-segment is taken as the blood oxygen descent depth corresponding to the blood oxygen signal sub-segment.

[0056] Furthermore, the start and end times of the blood oxygenation decline event are identified based on the depth of blood oxygenation decline, i.e., when... The change from less than 3% to greater than or equal to 3% is marked as the start time of the event. When the blood oxygen level changes from greater than or equal to 3% to less than 3%, it is marked as the end time of the event. Then, the end time of the event is subtracted from the start time of the event to obtain the duration of blood oxygen decrease corresponding to that event. The duration of blood oxygen decrease for each blood oxygen decrease event is calculated and the average value is taken to obtain the total duration of blood oxygen decrease.

[0057] Furthermore, in the case of a decrease in blood oxygenation, the rate of decrease in blood oxygenation between the start of the event and the time of the lowest point in blood oxygenation is calculated, and this rate is used as the slope of the decrease in blood oxygenation.

[0058] Furthermore, in the process of extracting sleep position features from position signal sub-segments, the body movement index is used to determine whether the target user is in a resting state. If the target user is in a resting state, the triaxial acceleration data at each sampling moment is extracted from the position signal sub-segment. The gravity vector at that sampling moment is constructed based on the triaxial acceleration data. The gravity vector is normalized and projected onto discrete position categories (including supine, left lateral, right lateral, prone, and sit-up / wake-up) according to a preset threshold. The position category with the highest proportion in each window is counted as the dominant position of that window. The proportion of supine positions in the window and whether position changes (i.e., position change indicators) are analyzed based on the position categories. The dominant position, supine position proportion, and position change indicators are combined into sleep position features. The dominant position in the sleep position features is encoded into a dominant position code. The dominant position code, supine position proportion, and position change indicators are then combined into a position feature vector.

[0059] In addition, if the target user is not in a resting state, the dominant body position is marked as an invalid body position, the supine position percentage is set to 0, and the body position change flag is set to 0; the invalid body position is encoded to obtain the corresponding body position code value, and the body position code value, the supine position percentage and the body position change flag are combined to form a body position feature vector.

[0060] Furthermore, the time difference between the trough amplitude and the minimum blood oxygen value in the respiratory waveform is calculated, and the correlation between the respiratory waveform and the blood oxygen signal sub-segment is calculated based on the Pearson correlation coefficient. The time difference and correlation are combined into a respiratory blood oxygen correlation feature vector.

[0061] Furthermore, the radar feature vector and the blood oxygen statistical feature vector are concatenated end-to-end to form the first fused feature vector, and the body position feature vector is concatenated end-to-end with the first fused feature vector to form the second fused feature vector.

[0062] Furthermore, the respiratory blood oxygen correlation feature vector and the second fusion feature vector are concatenated end to end to form a multimodal physiological fusion feature.

[0063] In this embodiment of the invention, by separating signal segments into channels, radar, blood oxygen, and body position signals can be processed independently, avoiding mutual interference between different modal features; by filtering and decomposing radar signal sub-segments, noise and body motion interference are eliminated; by cross-analyzing respiratory waveforms, heartbeat waveforms, and blood oxygen signal sub-segments, the temporal correspondence between respiratory events and blood oxygen decline is explored, making up for the lack of correlation of single-modal features and improving the reliability of hypoventilation and apnea discrimination.

[0064] S4. Classify the respiratory event types of the signal segments according to the multimodal physiological fusion characteristics to obtain the respiratory event types corresponding to the signal segments.

[0065] In this embodiment of the invention, the respiratory event type refers to the result category of the judgment of the physiological state corresponding to each signal segment, which is used to distinguish whether there is respiratory abnormality and its specific type within the time period, including normal events, hypoventilation, obstructive apnea, and central apnea.

[0066] In this embodiment of the invention, the respiratory event type classification of signal segments based on multimodal physiological fusion features is performed to obtain the respiratory event type corresponding to the signal segment. This includes: standardizing the multimodal physiological fusion features to obtain a standardized feature vector; calculating the probability that the signal segment belongs to each preset respiratory event type based on the standardized feature vector; determining the preliminary classification result based on the calculated probability of each preset respiratory event type; and using the preliminary classification result as the respiratory event type corresponding to the signal segment.

[0067] In this embodiment of the invention, during the standardization process of multimodal physiological fusion features, the mean and standard deviation of each feature vector in the multimodal physiological fusion features are calculated, and each feature vector is normalized according to the mean and standard deviation of each feature vector to obtain a standardized feature vector.

[0068] Furthermore, the standardized feature vector is input into a pre-trained machine learning classifier. The standardized feature vector is nonlinearly transformed and weighted through a multi-layer network inside the machine learning classifier to calculate the probability that the signal segment belongs to each of the N preset respiratory event types. From the probabilities of all respiratory event types, the respiratory event type with the highest probability is selected as the preliminary classification result of the signal segment, and the probability is used as the classification confidence of the corresponding respiratory event type.

[0069] In this embodiment of the invention, by standardizing the multimodal physiological fusion features, the numerical range and dimensions of features of different dimensions are unified, eliminating the interference of feature numerical differences on model classification and improving classification stability and convergence speed. The probability of a signal segment belonging to each preset respiratory event type is calculated by standardizing the feature vectors, providing an objective and comparable numerical basis for subsequent classification decisions. The preliminary classification results and classification confidence are determined based on the calculated probabilities of each preset respiratory event type, realizing the quality assessment of the classification results and providing a judgment standard for subsequent screening.

[0070] S5. Correct the breathing event types by applying rule constraints to obtain corrected breathing event types, and use the corrected breathing event types as the sleep apnea detection results.

[0071] In this embodiment of the invention, the method of performing rule-based constraint correction on respiratory event types to obtain corrected respiratory event types includes: performing time smoothing on respiratory event types to obtain optimized respiratory event types; acquiring the optimized blood oxygen saturation depth, optimized chest-abdominal motion phase difference, and optimized chest-abdominal motion amplitude ratio of the signal segment corresponding to the optimized respiratory event type, and determining whether the optimized respiratory event type is a hypoventilation type; if the optimized respiratory event type is a hypoventilation type and the optimized blood oxygen saturation depth is less than a preset blood oxygen threshold, then the optimized respiratory event type is corrected to a first corrected respiratory event type, and the first corrected respiratory event type is used as the corrected respiratory event type; if the optimized respiratory event type is not a hypoventilation type, then the method of correcting the optimized respiratory event type is ... then performing time smoothing on respiratory event types to obtain optimized respiratory event types; if the optimized respiratory event type is not a hypoventilation type, then the optimized respiratory event type is corrected to a first corrected respiratory event type, and the optimized respiratory event type is: performing time smoothing on respiratory event types to obtain optimized respiratory event types; acquiring the optimized blood oxygen saturation depth, optimized If the ventilation type is determined, it is determined whether the optimized breathing event type is an apnea type. If the optimized breathing event type is an apnea type and the duration of the optimized breathing event type is lower than the preset duration threshold, the optimized breathing event type is corrected to a second corrected breathing event type, and the second corrected breathing event type is used as the corrected breathing event type. If the optimized breathing event type is an apnea type and the duration of the optimized breathing event type is not lower than the preset duration threshold, the target breathing event type corresponding to the optimized breathing event type is determined based on the optimized chest and abdominal motion phase difference and the optimized chest and abdominal motion amplitude ratio, and the target breathing event type is used as the corrected breathing event type.

[0072] In this embodiment of the invention, the current respiratory event type is taken as the center, and the classification results of several adjacent signal segments are comprehensively considered. The optimized respiratory event type is determined based on the classification results of several adjacent signal segments.

[0073] Furthermore, it is determined whether the optimized respiratory event type is a hypoventilation type. If the optimized respiratory event type is a hypoventilation type, the optimized blood oxygen saturation drop depth corresponding to the optimized respiratory event type is compared with the preset blood oxygen threshold. If the optimized blood oxygen saturation drop depth is less than the preset blood oxygen threshold, the optimized respiratory event type is downgraded and corrected to a normal event type, i.e., the first corrected respiratory event type. If the optimized blood oxygen saturation drop depth reaches or exceeds the preset blood oxygen threshold, the optimized respiratory event type is kept as the original hypoventilation type.

[0074] Furthermore, if the optimized respiratory event type is not a hypoventilation type, it is then determined whether the optimized respiratory event type is an apnea type. If the optimized respiratory event type is an apnea type, the duration of the signal segment corresponding to the optimized respiratory event type is compared with a preset duration threshold. If the duration corresponding to the optimized respiratory event type is less than the preset duration threshold, the optimized respiratory event type is corrected to a normal event type, i.e., the second corrected respiratory event type.

[0075] Furthermore, if the optimized breathing event type is a sleep apnea type, and the duration corresponding to the optimized breathing event type is not less than a preset duration threshold, then it is further determined whether the optimized breathing event type is a central sleep apnea type. If the optimized breathing event type is a central sleep apnea type, it is further determined whether the target user is making a breathing effort. If the target user is making a breathing effort, then the optimized breathing event type is changed to an obstructive sleep apnea type. If the target user is not making a breathing effort, then the optimized breathing event type remains unchanged.

[0076] In detail, in determining whether the target user is making a respiratory effort, the mean of the optimized chest-abdominal motion phase difference and the standard deviation of the optimized chest-abdominal motion amplitude ratio of the signal segment corresponding to the optimized respiratory event type are calculated. If the mean of the optimized chest-abdominal motion phase difference is less than a preset inversion threshold and the standard deviation of the optimized chest-abdominal motion amplitude ratio is less than a preset amplitude ratio threshold, then it is confirmed that the target user is not making a respiratory effort. If the mean of the optimized chest-abdominal motion phase difference is not less than a preset inversion threshold and the standard deviation of the optimized chest-abdominal motion amplitude ratio is not less than a preset amplitude ratio threshold, then it is confirmed that the target user is making a respiratory effort.

[0077] Furthermore, if the optimized respiratory event type is not a central sleep apnea type, it is then determined whether the target user is making a breathing effort. If the target user is making a breathing effort, the optimized respiratory event type is kept as the original type. If the target user is not making a breathing effort, the optimized respiratory event type is changed to a central sleep apnea type.

[0078] Furthermore, if the optimized respiratory event type is not a sleep apnea type, then the optimized respiratory event type is defined as a normal event type, i.e., a modified respiratory event type.

[0079] Furthermore, if the optimized respiratory event type is an apnea type, the confidence difference between the classification confidence of the optimized respiratory event type belonging to the central apnea type and the classification confidence of the central apnea type is calculated. If the confidence difference is less than a preset confidence difference threshold, and the body position category in the body position feature vector corresponding to the optimized respiratory event type is supine, then the optimized respiratory event type is determined to be obstructive apnea. If the confidence difference is not less than the preset confidence difference threshold, then the optimized respiratory event type is kept as the original type.

[0080] In one embodiment, the total number of events, average duration, and longest duration of each respiratory event type in the multimodal continuous signal can also be statistically analyzed based on the modified respiratory event type.

[0081] Furthermore, the AHI index (the average number of apnea and hypopnea events per hour during sleep) can be expressed by the following formula:

[0082] in, This indicates the average number of apnea and hypopnea events per hour during sleep. This indicates the total number of different types of sleep apnea events that occurred throughout the night. This indicates the total number of hypoventilation events that occurred during the entire night's sleep. This indicates the total sleep duration.

[0083] Specifically, the system determines whether a user is in a sleep state by using body movement index and respiratory waveform, combined with sleep posture signals (e.g., no significant body movement for a period of time, regular breathing), and accumulates the total duration of all windows that are determined to be in a sleep state, using this total duration as the total sleep duration.

[0084] Furthermore, the supine AHI (i.e., the average number of apnea and hypoventilation events per hour during supine sleep) can be calculated using the following formula:

[0085] in, This indicates the supine AHI position. This indicates the total number of different types of apnea and hypoventilation events that occurred while the patient was in the supine position. This indicates the total sleep duration in the supine position.

[0086] In addition to determining that the user is in a sleep state, the system further filters out time windows with the supine position as the body position category, and sums up the duration of the filtered time windows to obtain the total sleep time in the supine position.

[0087] Furthermore, the lateral decubitus position AHI (i.e., the average number of apnea and hypoventilation events per hour during lateral decubitus sleep) can be calculated using the following formula:

[0088] in, This indicates AHI in the lateral decubitus position. This indicates the total number of different types of apnea and hypoventilation events that occurred while in the lateral decubitus position. This indicates the total sleep duration in the side-lying position.

[0089] In addition to determining that the user is in a sleep state, the system further filters out time windows with the side-lying position category, and sums up the duration of the filtered time windows to obtain the total sleep time in the supine position.

[0090] Furthermore, the oxygen depletion index, which is the average number of times blood oxygen levels drop per hour, can be calculated using the following formula:

[0091] in, Indicates the oxygen reduction index. This indicates the total number of events that cause a drop in blood oxygenation during the entire night's sleep (a drop in blood oxygenation ≥3% and a duration of blood oxygenation ≥10 seconds). This indicates the total sleep duration.

[0092] Furthermore, a sleep report is generated based on the AHI index, supine AHI, lateral AHI, and oxygen depletion index.

[0093] In this embodiment of the invention, temporal smoothing of respiratory event types to obtain optimized respiratory event types includes: selecting multiple respiratory event types adjacent to the respiratory event type according to a preset domain range, and combining the selected respiratory event types into a local event type set; performing frequency statistics on each respiratory event type in the local event type set to obtain the frequency statistics result for each respiratory event type; selecting the respiratory event type with the highest proportion in the local event type set based on the frequency statistics result, and using the respiratory event type with the highest proportion as a candidate smoothing event type; performing duration verification on the candidate smoothing event type according to a preset duration threshold to obtain the smoothing event type that passes the verification; and performing temporal continuity correction on the smoothing event type to obtain the optimized respiratory event type.

[0094] In this embodiment of the invention, based on a preset neighborhood range (e.g., taking two segments before and after), with the signal segment corresponding to the current respiratory event type as the center, several respiratory event types located before and after the signal segment are selected from all respiratory event types; the selected respiratory event types are combined with the current respiratory event type to form a local event type set.

[0095] Furthermore, iterate through all respiratory event types in the local event type set and count the number of times each respiratory event type appears in the local event type set. For example, if the local event type set contains [normal, normal, hypoventilation, normal, normal], then the frequency of normal is 4 and the frequency of hypoventilation is 1.

[0096] Furthermore, based on the frequency statistics, the respiratory event type with the highest frequency (i.e., the most occurrences) is identified. If there is only one respiratory event type corresponding to the highest frequency, it is directly used as a candidate smoothing event type. In addition, if there are multiple types with the highest frequency (e.g., normal and hypoventilation each occur twice), a preset tie-breaking rule is adopted to prioritize the respiratory event type that is closer to the current signal segment in time as the candidate smoothing event type.

[0097] Furthermore, it checks whether the total duration of the signal segments corresponding to the consecutively identified candidate smoothing event types reaches or exceeds a preset duration threshold. If the total duration reaches the duration threshold, the candidate smoothing event type is retained as a smoothing event type; if the total duration does not reach the duration threshold, the candidate smoothing event type is changed to a normal event type, i.e., a smoothing event type.

[0098] Furthermore, it is determined whether the respiratory event types adjacent to the smoothed event type are the same as the smoothed event type. If they are the same, the smoothed event type is used as the optimized respiratory event type. If they are different, the respiratory event type with the highest frequency among the respiratory event types adjacent to the smoothed event type is used as the optimized respiratory event type corresponding to the smoothed event type according to a preset range.

[0099] In this embodiment of the invention, by performing time smoothing on respiratory event types, misjudgment of adjacent events and fragmented labeling are eliminated, type jumps caused by short-term noise are corrected, and the consistency of overall labeling is improved. By obtaining the blood oxygen saturation drop depth, chest and abdominal motion phase difference and amplitude ratio of the signal segment corresponding to the optimized respiratory event type, and determining whether the optimized respiratory event type is a hypoventilation type, a quantitative judgment standard is provided for the correction of subsequent hypoventilation events, avoiding misjudgment based on single features.

[0100] like Figure 3 The diagram shown is a functional block diagram of a multimodal sleep apnea detection device provided in an embodiment of the present invention.

[0101] The multimodal sleep apnea detection device 300 of the present invention can be installed in an electronic device. Depending on the functions implemented, the multimodal sleep apnea detection device 300 may include a multimodal continuous signal fusion module 301, a signal segmentation module 302, a multimodal physiological fusion feature extraction module 303, a respiratory event type classification module 304, and a respiratory event type correction module 305. A module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0102] In this embodiment, the functions of each module / unit are as follows: The multimodal continuous signal fusion module 301 is used to collect the radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, and fuse the radar signal, blood oxygen signal and sleep position signal into a multimodal continuous signal. The signal segmentation module 302 is used to segment a multimodal continuous signal into time windows to obtain a signal segment corresponding to each window. The multimodal physiological fusion feature extraction module 303 is used to extract multimodal physiological fusion features from signal segments; The respiratory event type classification module 304 is used to classify the respiratory event type of the signal segment according to the multimodal physiological fusion features to obtain the respiratory event type corresponding to the signal segment; The breathing event type correction module 305 is used to correct the breathing event type by applying rule constraints, thereby obtaining the corrected breathing event type, which is then used as the sleep apnea detection result.

[0103] In one embodiment, the multimodal continuous signal fusion module 301 is specifically used to timestamp the radar signal, blood oxygen signal, and sleep position signal respectively to obtain a radar signal sequence, blood oxygen signal sequence, and sleep position signal sequence with timestamps; to resample the blood oxygen signal sequence and sleep position signal sequence according to the sampling frequency of the radar signal sequence to obtain a sampled blood oxygen signal sequence and a sampled sleep position signal sequence with a unified sampling frequency; to align the radar signal sequence, the sampled blood oxygen signal sequence, and the sampled sleep position signal sequence with time points to generate a time-synchronized multimodal signal sequence; and to generate a multimodal continuous signal according to the multimodal signal sequence in chronological order.

[0104] In one embodiment, the signal segmentation module 302 is specifically used to determine the window start time point according to the preset window length and sliding step size, and to collect the window start time points into a segmentation position set; to perform signal segmentation on the multimodal continuous signal according to the segmentation position set to obtain the segmented preliminary signal segments; and to perform validity verification on the preliminary signal segments according to the preset duration threshold to obtain the signal segments corresponding to each window.

[0105] In one embodiment, the multimodal physiological fusion feature extraction module 303 is specifically used to perform channel separation on signal segments to obtain radar signal sub-segments, blood oxygen signal sub-segments, and body position signal sub-segments; extract respiratory features, chest and abdominal movement features, heart rate features, and body movement index from the radar signal sub-segments; combine the respiratory features, chest and abdominal movement features, heart rate features, and body movement index into a radar feature vector; extract static blood oxygen features and dynamic blood oxygen features from the blood oxygen signal sub-segments, and combine the static blood oxygen features and dynamic blood oxygen features into a blood oxygen statistical feature vector; extract sleep position features from the body position signal sub-segments, and convert the sleep position features into a body position feature vector; perform correlation analysis on the radar signal sub-segments and blood oxygen signal sub-segments to generate a respiratory blood oxygen correlation feature vector; and concatenate the radar feature vector, blood oxygen statistical feature vector, body position feature vector, and respiratory blood oxygen correlation feature vector into a multimodal physiological fusion feature.

[0106] In one embodiment, the respiratory event type classification module 304 is specifically used to standardize the multimodal physiological fusion features to obtain a standardized feature vector; calculate the probability that the signal segment belongs to each preset respiratory event type based on the standardized feature vector; determine the preliminary classification result based on the calculated probability of each preset respiratory event type; and use the preliminary classification result as the respiratory event type corresponding to the signal segment.

[0107] In one embodiment, the respiratory event type correction module 305 is specifically used to perform time smoothing on the respiratory event type to obtain an optimized respiratory event type; acquire the optimized blood oxygen saturation depth, optimized chest and abdominal motion phase difference, and optimized chest and abdominal motion amplitude ratio of the signal segment corresponding to the optimized respiratory event type, and determine whether the optimized respiratory event type is a hypoventilation type; if the optimized respiratory event type is a hypoventilation type and the optimized blood oxygen saturation depth is less than a preset blood oxygen threshold, then the optimized respiratory event type is corrected to a first corrected respiratory event type, and the first corrected respiratory event type is used as the corrected respiratory event type; if the optimized respiratory event type is not a hypoventilation type, ... Then, determine whether the optimized breathing event type is a sleep apnea type; if the optimized breathing event type is a sleep apnea type and the duration corresponding to the optimized breathing event type is lower than the preset duration threshold, then the optimized breathing event type is corrected to the second corrected breathing event type, and the second corrected breathing event type is used as the corrected breathing event type; if the optimized breathing event type is a sleep apnea type and the duration of the optimized breathing event type is not lower than the preset duration threshold, then determine the target breathing event type corresponding to the optimized breathing event type based on the optimized chest and abdominal motion phase difference and the optimized chest and abdominal motion amplitude ratio, and use the target breathing event type as the corrected breathing event type.

[0108] In detail, each module in the multimodal sleep apnea detection device 300 of this embodiment of the invention uses the same technical means as the multimodal sleep apnea detection method in the accompanying drawings, and can produce the same technical effect, which will not be repeated here.

[0109] like Figure 4 The diagram shown is a structural schematic of an electronic device for implementing a multimodal sleep apnea detection method according to an embodiment of the present invention.

[0110] Electronic device 4 may include processor 40, memory 41, communication bus 42 and communication interface 43, and may also include computer programs stored in memory 41 and run on processor 40, such as multimodal sleep apnea detection program.

[0111] In some embodiments, the processor 40 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The memory 41 includes at least one type of readable medium, including flash memory, portable hard drives, multimedia cards, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disks, optical disks, etc. The communication bus 42 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication interface 43 is used for communication between the above-mentioned electronic device and other devices, including network interfaces and user interfaces.

[0112] Figure 4 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 4 The structure shown does not constitute a limitation on the electronic device 4, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0113] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 40 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power sources, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.

[0114] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0115] Specifically, the processor 40's specific implementation method of the above instructions can be found in the description of the relevant steps in the corresponding embodiments in the accompanying drawings, and will not be repeated here.

[0116] Furthermore, if the modules / units integrated in the electronic device 4 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable medium. The computer-readable medium can be volatile or non-volatile. For example, a computer-readable medium may include: any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0117] The present invention also provides a computer-readable medium storing a computer program, which, when executed by a processor, can implement a multimodal sleep apnea detection method according to any of the above embodiments.

[0118] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0119] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0120] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0121] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0122] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multimodal sleep apnea detection method, characterized in that, The method includes: The radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, are collected, and the radar signal, the blood oxygen signal and the sleep position signal are fused into a multimodal continuous signal. The multimodal continuous signal is divided into time windows to obtain signal segments corresponding to each window; Extract multimodal physiological fusion features from the signal segment; The respiratory event type is classified according to the multimodal physiological fusion features to obtain the respiratory event type corresponding to the signal segment; The breathing event type is modified by rule constraints to obtain a modified breathing event type, which is then used as the sleep apnea detection result.

2. The multimodal sleep apnea detection method as described in claim 1, characterized in that, The extraction of multimodal physiological fusion features from the signal segment includes: The signal segment is subjected to channel separation to obtain radar signal sub-segment, blood oxygen signal sub-segment, and body position signal sub-segment; Respiratory features, chest and abdominal movement features, heart rate features, and body movement index are extracted from the radar signal sub-segments; The respiratory features, chest and abdominal movement features, heart rate features, and body movement index are combined into a radar feature vector; Extract static and dynamic blood oxygenation features from the blood oxygenation signal sub-segments, and combine the static and dynamic blood oxygenation features into a blood oxygenation statistical feature vector. Extract sleep posture features from the posture signal sub-segment, and convert the sleep posture features into a posture feature vector; Correlation analysis is performed on the radar signal sub-segment and the blood oxygen signal sub-segment to generate a respiratory blood oxygen correlation feature vector; The radar feature vector, the blood oxygen statistical feature vector, the body position feature vector, and the respiratory blood oxygen correlation feature vector are concatenated into a multimodal physiological fusion feature.

3. The multimodal sleep apnea detection method as described in claim 1, characterized in that, The step of classifying the respiratory event type of the signal segment based on the multimodal physiological fusion features to obtain the respiratory event type corresponding to the signal segment includes: The multimodal physiological fusion features are standardized to obtain a standardized feature vector; Calculate the probability that the signal segment belongs to each preset respiratory event type based on the standardized feature vector; The preliminary classification result is determined based on the calculated probability of each preset respiratory event type, and the preliminary classification result is used as the respiratory event type corresponding to the signal segment.

4. The multimodal sleep apnea detection method as described in claim 1, characterized in that, The step of modifying the respiratory event type by applying rule constraints to obtain the modified respiratory event type includes: Time smoothing is performed on the respiratory event types to obtain optimized respiratory event types; The optimized blood oxygen saturation depth, optimized chest and abdominal motion phase difference, and optimized chest and abdominal motion amplitude ratio of the signal segment corresponding to the optimized respiratory event type are obtained, and it is determined whether the optimized respiratory event type is a hypoventilation type. If the optimized respiratory event type is a hypoventilation type and the optimized blood oxygen drop depth is less than a preset blood oxygen threshold, then the optimized respiratory event type is corrected to a first corrected respiratory event type, and the first corrected respiratory event type is used as the corrected respiratory event type. If the optimized respiratory event type is not a hypoventilation type, then determine whether the optimized respiratory event type is a sleep apnea type; If the optimized breathing event type is a sleep apnea type, and the duration corresponding to the optimized breathing event type is lower than a preset duration threshold, then the optimized breathing event type is corrected to a second corrected breathing event type, and the second corrected breathing event type is used as the corrected breathing event type. If the optimized breathing event type is a sleep apnea type, and the duration of the optimized breathing event type is not less than a preset duration threshold, then the target breathing event type corresponding to the optimized breathing event type is determined based on the optimized chest and abdominal motion phase difference and the optimized chest and abdominal motion amplitude ratio, and the target breathing event type is used as the corrected breathing event type.

5. The multimodal sleep apnea detection method as described in claim 4, characterized in that, The step of performing time smoothing on the respiratory event types to obtain optimized respiratory event types includes: According to a preset domain range, select multiple respiratory event types that are adjacent to the respiratory event type, and combine the selected respiratory event types into a local event type set; Frequency statistics are performed on each respiratory event type in the local event type set to obtain the frequency statistics results for each respiratory event type; Based on the frequency statistics results, the respiratory event type with the highest proportion in the local event type set is selected, and the respiratory event type with the highest proportion is used as the candidate smoothing event type; The candidate smoothing event types are validated for duration based on a preset duration threshold to obtain the smoothing event types that pass the validation. The smoothed event type is modified by temporal continuity to obtain an optimized respiratory event type.

6. The multimodal sleep apnea detection method as described in claim 1, characterized in that, The process of fusing the radar signal, the blood oxygen signal, and the sleep posture signal into a multimodal continuous signal includes: The radar signal, the blood oxygen signal, and the sleep position signal are timestamped to obtain a radar signal sequence, a blood oxygen signal sequence, and a sleep position signal sequence with timestamps. The blood oxygen signal sequence and the sleep position signal sequence are resampled according to the sampling frequency of the radar signal sequence to obtain a blood oxygen signal sequence and a sleep position signal sequence with a unified sampling frequency. The radar signal sequence, the sampled blood oxygen signal sequence, and the sampled sleep posture signal sequence are time-aligned to generate a time-synchronized multimodal signal sequence; A multimodal continuous signal is generated according to the multimodal signal sequence in chronological order.

7. The multimodal sleep apnea detection method as described in claim 1, characterized in that, The step of dividing the multimodal continuous signal into time windows to obtain signal segments corresponding to each window includes: The window start time point is determined based on the preset window length and sliding step size, and the window start time points are collected into a cutting position set. Based on the set of cutting positions, the multimodal continuous signal is segmented to obtain preliminary segmented signal segments; The validity of the initial signal segments is verified according to a preset duration threshold to obtain the signal segments corresponding to each window.

8. A multimodal sleep apnea detection device, characterized in that, The device includes: The multimodal continuous signal fusion module is used to collect the radar signal returned after the millimeter-wave radar transmits electromagnetic waves to the chest area of ​​the target user, as well as the target user's blood oxygen signal and sleep position signal, and fuse the radar signal, the blood oxygen signal and the sleep position signal into a multimodal continuous signal. The signal segmentation module is used to segment the multimodal continuous signal into time windows to obtain signal segments corresponding to each window. A multimodal physiological fusion feature extraction module is used to extract multimodal physiological fusion features from the signal segment; The respiratory event type classification module is used to classify the signal segment into respiratory event types based on the multimodal physiological fusion features, and obtain the respiratory event type corresponding to the signal segment; The breathing event type correction module is used to perform rule constraint correction on the breathing event type to obtain a corrected breathing event type, and use the corrected breathing event type as the sleep apnea detection result.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multimodal sleep apnea detection method as described in any one of claims 1 to 7.

10. A computer-readable medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the multimodal sleep apnea detection method as described in any one of claims 1 to 7.

Images