Sleep breathing state monitoring method, sleep monitoring system, equipment and medium

Abstract

The invention provides a sleep breathing state monitoring method, a sleep monitoring system, equipment and a medium. The method comprises the following steps: generating a time-frequency spectrogram of a normal mixed heart impact signal; screening out an area associated with the heart rate in the time-frequency spectrogram based on the mask matrix to obtain time-frequency spectrum information; decomposing an intrinsic mode function of a normal mixed heart impact signal according to the information of the trained mask; time domain waveform information in the intrinsic mode function is extracted; and performing fusion analysis on the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result. According to the sleep breathing state monitoring method provided by the invention, the time-frequency spectrum information with high quality and extremely low noise is obtained, the time-domain waveform information without distortion is also obtained, and further through fusion analysis of the time-frequency spectrum information and the time-domain waveform information, comprehensive analysis of sleep apnea events is realized, so that the sleep apnea state monitoring accuracy is improved. An accurate sleep apnea classification result is generated, and powerful technical support is provided for diagnosis and treatment of sleep disorder diseases.

Description

Technical Field

[0001] The present invention relates to the technical field of sleep monitoring, and in particular to a sleep respiratory state monitoring method, a sleep monitoring system, a device and a medium. Background Art

[0002] Obstructive sleep apnea (OSA) is a common sleep disorder characterized by upper airway obstruction or narrowing during sleep. OSA can lead to daytime sleepiness, cognitive impairment, cardiovascular disease, and, in severe cases, even fatal sleep-related events. Therefore, continuous monitoring and analysis of individual respiratory movements during sleep, particularly sleep apnea, can help prevent health risks associated with disorders like OSA.

[0003] Portable sleep monitoring devices offer an economical and efficient solution for monitoring sleep apnea status in the home. These devices typically use mattress sensors to collect heart rate information, offering excellent adaptability and convenience. While these sensors can capture heart rate information, their signal quality is poor. Subsequent signal processing filters can weaken or offset signal peaks, even generating spurious peaks. This severely impacts true signal recognition, limiting the application of these devices in sleep apnea monitoring. Summary of the Invention

[0004] The present invention provides a sleep respiratory state monitoring method, a sleep monitoring system, a device and a medium, which are used to solve the problem of inaccurate sleep apnea monitoring results.

[0005] In a first aspect, the present invention provides a method for monitoring sleep respiratory status, comprising: Generate a time-frequency spectrum of a normal mixed ballistocardial signal; Based on the mask matrix, filtering out the region associated with the heart rate in the time-spectrum graph to obtain time-spectrum information; Decomposing the intrinsic mode function of the normal mixed ballistocardia signal; Extracting time domain waveform information from the intrinsic mode function; The time-frequency spectrum information and the time-domain waveform information are fused and analyzed to generate a sleep apnea classification result.

[0006] In one embodiment, the normal mixed ballistocardiographic signal is obtained by: Collect physiological signals from the chest and abdomen through piezoelectric ceramic sensors; extracting an original cardiac signal from the physiological signal; The normal mixed cardiac ballistic signal is obtained by detecting the nonlinear characteristics of the original cardiac ballistic signal and filtering out the normal signal in the original cardiac ballistic signal.

[0007] In one embodiment, generating a time-frequency spectrum of a normal mixed ballistocardiographic signal includes: Determine multiple different window lengths based on the resolution requirements in the time domain and frequency domain; Based on each of the window lengths, a discrete short-time Fourier transform is performed on the normal mixed cardiac ballistic signal to obtain time-frequency spectrograms of multiple scales.

[0008] In one embodiment, when the regions associated with the heart rate in the time-spectrogram are screened out based on the mask matrix to obtain the time-spectrogram information, the following steps are performed for each time-spectrogram: Inputting the time-spectrogram into the ViT model to obtain a probability matrix output by the ViT model; each probability value in the probability matrix is ​​used to reflect the confidence level that the image block at the corresponding position of the probability value in the time-spectrogram is associated with the heart rate; Determine the mask of the image block corresponding to the probability value greater than or equal to the preset threshold in the probability matrix as 1; Determine the mask of the image block corresponding to the probability value less than the preset threshold in the probability matrix as 0; generating a mask matrix according to the mask of each image block in the time-spectrogram; combining the mask matrix and the time-spectrogram to generate a region associated with the heart rate; The region associated with the heart rate is determined as time-frequency spectrum information.

[0009] In one embodiment, decomposing the intrinsic mode function of the normal mixed ballistocardiographic signal includes: The optimization problem of the variational mode decomposition algorithm is defined as minimizing the sum of the bandwidths of all modes; Determining the constraints of the problem to be optimized based on the time-frequency spectrum information; Under the constraints, variational modal decomposition is performed on the normal mixed ballistocardial signal, and a set of intrinsic modal functions is obtained by solving the problem to be optimized.

[0010] In one embodiment, extracting the time domain waveform information from the intrinsic mode function includes: Filtering out a plurality of target modal signals close to the heart rate, respiratory rate, and IJKL wave from the set of intrinsic modal functions; Time domain waveform information is extracted from each of the target modal signals.

[0011] In one embodiment, the fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result includes: Inputting each time domain waveform information into a one-dimensional convolutional network to obtain a time domain feature of each time domain waveform information output by the one-dimensional convolutional network; Inputting each time-frequency spectrum information into a two-dimensional convolutional network to obtain time-frequency spectrum features of each time-frequency spectrum graph output by the two-dimensional convolutional network; Controlling each of the time domain features and each of the time-frequency spectrum features in the same time dimension to obtain a plurality of target time domain features and a plurality of target time-frequency spectrum features; Stacking the multiple target time domain features and the multiple target time-frequency spectrum features to obtain a fusion feature; A sleep apnea classification result is determined based on the fused features.

[0012] In a second aspect, the present invention further provides a sleep monitoring system, comprising: A time-frequency spectrum generating module, used for generating a time-frequency spectrum of a normal mixed heartbeat signal; a time-frequency spectrum information generation module, configured to filter out regions associated with the heart rate in the time-frequency spectrum graph based on a mask matrix to obtain time-frequency spectrum information; A modal decomposition module, configured to decompose the intrinsic modal function of the normal mixed ballistocardial signal; A time domain waveform information generating module, used for extracting the time domain waveform information from the intrinsic mode function; The sleep apnea analysis module is used to fuse and analyze the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0013] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described sleep respiratory status monitoring methods are implemented.

[0014] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the sleep respiratory state monitoring method as described in any one of the above are implemented.

[0015] The sleep respiratory state monitoring method, sleep monitoring system, device and medium provided by the present invention generate a time-frequency spectrum of a normal mixed cardiopulmonary bypass signal and use a mask matrix to screen out key areas associated with the heart rate in the time-frequency spectrum to ensure that the frequency bands directly related to heart rate fluctuations are highlighted in the time-frequency spectrum, thereby obtaining high-quality time-frequency spectrum information with minimal noise. In addition, considering that the resolution of the time-frequency spectrum information is limited and it is difficult to capture the peak amplitude and variability of the heartbeat, distortion can be avoided by decomposing the intrinsic mode function of the normal mixed cardiopulmonary bypass signal and extracting time-domain waveform information therefrom. Furthermore, through the fusion analysis of the time-frequency spectrum information and the time-domain waveform information, a comprehensive analysis of sleep apnea events is achieved, and accurate sleep apnea classification results are generated, providing strong technical support for the diagnosis and treatment of sleep disorders. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a flow chart of the sleep respiratory state monitoring method provided by the present invention.

[0018] Figure 2 This is an overall flow chart of the sleep respiratory status monitoring method provided by the present invention.

[0019] Figure 3 This is a diagram showing the DSTFT change process and target frequency band of a normal mixed ballistocardial signal provided by the present invention.

[0020] Figure 4 It is a schematic diagram of the image segmentation training of the ViT model provided by the present invention.

[0021] Figure 5 It is a schematic diagram of the mask generation process used in the model training provided by the present invention.

[0022] Figure 6 It is a schematic diagram of multiple modes obtained by the variational modal decomposition provided by the present invention.

[0023] Figure 7 This is an overall architecture diagram of the sleep respiratory status monitoring method provided by the present invention.

[0024] Figure 8 It is a structural schematic diagram of the sleep monitoring system provided by the present invention.

[0025] Figure 9It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0027] The terms "first," "second," and the like in the present invention are used to distinguish similar objects and are not used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention can be implemented in orders other than those illustrated or described herein.

[0028] The following combination Figures 1-9 The present invention describes the sleep respiratory state monitoring method, sleep monitoring system, device and medium provided by the present invention.

[0029] It should be noted that the sleep respiratory status monitoring method provided in the embodiments of the present invention is implemented based on a sleep monitoring system. The sleep respiratory status monitoring method provided in the embodiments of the present invention uses a pressure acquisition device to monitor the ballistocardiogram (BCG) signals of the chest and abdomen during sleep. To extract more accurate spectral information, a discrete short-time Fourier transform (DSTFT) is used to convert the one-dimensional mixed BCG signal into a high-resolution three-dimensional dynamic spectrogram. Combining the heart rate variability (HRV) spectrum with real-time heart rate information, a dynamic time-frequency spectrum mask is designed to highlight target features associated with heart rate. Image segmentation techniques such as ViT (Vision Transformer) and U-Net can then be used to process the complex dynamic spectrogram, filter out target regions, and enhance the time-frequency spectrum features. To further improve the performance of sleep respiratory status monitoring, variational mode decomposition (VDM) technology is applied based on key parameters extracted in the feature enhancement stage to extract time-domain features such as amplitude, interval, and kurtosis from the mixed BCG signal. These time-domain features, combined with time-spectral features, provide a more comprehensive description of signal characteristics. Multimodal feature fusion is achieved through a Multi-CNN architecture, which utilizes one-dimensional and two-dimensional convolutional neural networks to process temporal and spatial features, respectively. During processing, the time-spectral and time-domain features are processed in a unified dimension and fused within the Multi-CNN. The final model uses fully connected layers for sleep apnea classification, fully utilizing and integrating multimodal features and effectively improving monitoring performance.

[0030] In the embodiment of the present invention, a sleep monitoring system is used as an example to describe the sleep respiratory state monitoring method.

[0031] Combine Figure 1 , Figure 1 It is a flow chart of the sleep respiratory state monitoring method provided by the present invention.

[0032] like Figure 1 As shown, the method includes the following: Step 101: Generate a time-frequency spectrum of a normal mixed ballistocardial signal; Step 102: Based on the mask matrix, filter out the area associated with the heart rate in the time-frequency spectrum to obtain time-frequency spectrum information; Step 103: Decomposing the intrinsic mode function of the normal mixed ballistocardial signal; Step 104: extracting time domain waveform information from the intrinsic mode function; Step 105: Fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0033] The following content can be combined Figure 2 , Figure 2 This is an overall flow chart of the sleep respiratory status monitoring method provided by the present invention.

[0034] Specifically, the sleep monitoring system uses piezoelectric ceramic sensors to collect physiological signals from the human chest and abdomen. However, the physiological signals obtained by the piezoelectric ceramic sensors are actually mixed signals, which are mixed with multiple physiological signals such as breathing, heartbeat, and body movement. Therefore, it is necessary to screen out normal mixed BCG signals from the mixed signals.

[0035] Furthermore, the DSTFT algorithm is used to perform discrete short-time Fourier transform on the normal mixed BCG signal to obtain the time-frequency spectrum of the normal mixed BCG signal.

[0036] Furthermore, the time-frequency spectrum of the normal mixed BCG signal is obtained through the real heart rate and input into the ViT model. The ViT model performs image segmentation, feature encoding and feature decoding on it, and finally predicts the probability value associated with the heart rate for each image block. Therefore, ViT generates a probability matrix according to the position of the image block based on the probability values ​​associated with the heart rate of all image blocks and outputs it.

[0037] Furthermore, a mask corresponding to each image block is generated by comparing each probability value in the probability matrix with a preset threshold, and a mask matrix is ​​generated according to the mask corresponding to each image block.

[0038] Furthermore, based on the mask matrix, the area in the time-spectrum diagram associated with the heart rate is filtered out to obtain the time-spectrum information. It can be understood that the mask matrix and the time-spectrum diagram are combined to filter out the area in the time-spectrum diagram associated with the heart rate, and the area is determined as the time-spectrum information. The time-spectrum information can highlight the target area related to the heart rate and enhance the time-spectrum characteristics within the area.

[0039] In addition, due to the limited time and frequency resolution of the DSTFT method, it is difficult to accurately capture the peak amplitude of each heartbeat and the details of its heart rate variability. To solve this problem, the VDM technology is used to decompose the intrinsic mode function (IMF) of the normal mixed BCG signal. The IMF can be considered as the mode decomposed from the signal.

[0040] Furthermore, the time domain waveform information is extracted from the intrinsic mode function (IMF) to enhance the time domain features of the original BCG signal.

[0041] After extracting the time-spectrum information from the time-spectrum graph and the time-domain waveform information from the intrinsic mode function (IMF), the time-spectrum information and the time-domain waveform information are fused and analyzed to generate a sleep apnea classification result.

[0042] Specifically, the time-domain waveform information is processed through a multi-layer one-dimensional convolutional network to extract the time-domain features therein, and the time-spectrum information is processed through a multi-layer two-dimensional convolutional network to extract the time-spectrum features therein. Furthermore, the time-spectrum features and the time-domain features are processed in a unified dimension, and the time-spectrum features and the time-domain features of the unified dimension are fused to obtain fused features. The fused features are classified and predicted by a multilayer perceptron (MLP) to obtain the sleep apnea classification result.

[0043] The sleep respiratory state monitoring method provided by the present invention generates a time-frequency spectrum of a normal mixed cardiopulmonary bypass signal and uses a mask matrix to screen out key areas associated with the heart rate in the time-frequency spectrum to ensure that the frequency bands directly related to heart rate fluctuations are highlighted in the time-frequency spectrum, thereby obtaining high-quality time-frequency spectrum information with minimal noise. In addition, considering that the resolution of the time-frequency spectrum information is limited and it is difficult to capture the peak amplitude and variability of the heartbeat, distortion can be avoided by decomposing the intrinsic mode function of the normal mixed cardiopulmonary bypass signal and extracting time-domain waveform information therefrom. Furthermore, through the fusion analysis of the time-frequency spectrum information and the time-domain waveform information, a comprehensive analysis of sleep apnea events is achieved, and accurate sleep apnea classification results are generated, providing strong technical support for the diagnosis and treatment of sleep disorders.

[0044] In some embodiments, the normal mixed ballistocardiographic signal is obtained by: Collect physiological signals from the chest and abdomen through piezoelectric ceramic sensors; extracting an original cardiac signal from the physiological signal; The normal mixed cardiac ballistic signal is obtained by detecting the nonlinear characteristics of the original cardiac ballistic signal and filtering out the normal signal in the original cardiac ballistic signal.

[0045] Specifically, the sleep monitoring system utilizes a hardware design based on a piezoelectric ceramic sensor array, capable of collecting physiological signals from the chest and abdomen. The sensor array comprises 32 sensors arranged in an 8×4 matrix, with an overall size of 58 cm × 28 cm, covering the chest and abdomen of an adult, ensuring effective monitoring of physiological signals such as heartbeat and respiration. Each sensor consists of a circular piezoelectric element with a diameter of 3 cm and a brass base, mounted on a 3 cm × 5 cm printed circuit board. Through the piezoelectric effect, the sensor generates a weak current when pressure changes. This signal is amplified by an analog front-end circuit, then processed through low-pass filtering and analog-to-digital conversion. This design effectively reduces interference from non-target activities such as body movement, ensuring high-precision capture of physiological signals and providing reliable data for sleep apnea detection.

[0046] The mixed signal acquired by piezoelectric ceramic sensors typically contains multiple physiological signals, such as respiration and heartbeat. To accurately analyze these signals, sleep monitoring systems use filters to separate the cardiopulmonary signals from the mixed signal, particularly the heart rate information associated with sleep apnea. A Chebyshev I finite impulse response (FIR) digital filter can be used to separate the mixed signal into the respiratory signal (frequency 0.1-0.8 Hz) and the BCG signal (frequency range 0.8-15 Hz). Since interfering signals will need to be filtered out later, the original BCG signal can be considered here.

[0047] The BCG signal consists of an envelope waveform and an IJKL wave. The IJKL wave is closely related to cardiac contraction and aortic blood flow. The J peak (Jpeak) corresponds to the maximum amplitude of the heart cycle, and the interval between J peaks directly reflects heart rate changes.

[0048] For interference signals caused by non-periodic movements, such as turning over and body movements, the nonlinear characteristics of the original BCG signal, such as energy and approximate entropy (ApEn), can be detected to distinguish the interference signal from the normal signal. In other words, the normal signal in the original BCG signal can be screened out to obtain the normal mixed cardiac signal for subsequent processing.

[0049] The embodiment of the present invention obtains physiological signals during periodic sleep activities through piezoelectric ceramic sensors, ensuring the capture of high-precision physiological signals, further extracting the BCG signal that is most closely related to the heart rate from the physiological signals, and screening out normal signals in the BCG signal by monitoring the nonlinear characteristics of the BCG signal, ensuring that the sample obtained contains accurate BCG signal details, which can be used for subsequent accurate sleep apnea detection.

[0050] In some embodiments, based on step 101, generating a time-frequency spectrum of the mixed ballistocardiographic signal includes: Determine multiple different window lengths based on the resolution requirements in the time domain and frequency domain; Based on each of the window lengths, a discrete short-time Fourier transform is performed on the normal mixed cardiac ballistic signal to obtain time-frequency spectrograms of multiple scales.

[0051] It should be noted that DSTFT technology is used to extract precise time-frequency spectrum information from normal mixed BCG signals while avoiding the peak attenuation and spurious peaks common in traditional signal processing. HRV is a physiological indicator closely related to sleep apnea. HRV represents the variation in the intervals between heartbeats, which reflects the regulatory function of the autonomic nervous system. This variation can be disrupted by sleep apnea events during sleep. Specifically, transient hypoxemia and arousal responses caused by sleep apnea events can lead to HRV changes. Therefore, analyzing HRV variation patterns can effectively identify sleep apnea events.

[0052] Many active health devices use HRV analysis to identify sleep apnea, but this method requires extremely high HRV signal accuracy, making heart rate acquisition challenging for mattress-based sensor devices. Widely used filters can attenuate or offset signal peaks, or even introduce spurious peaks, leading to inaccurate signal identification and limiting the application of mattress-based devices for sleep apnea detection. To circumvent this traditional issue, a new method uses DSTFT to process the normal mixed BCG signal, enhancing signal accuracy and extracting more accurate feature information.

[0053] Specifically, DSTFT technology is used to perform multi-resolution DSTFT transform on the normal mixed BCG signal, converting the one-dimensional signal into a three-dimensional time-frequency spectrum to capture the frequency characteristics of heart rate changes and other physiological activities.

[0054] The core purpose of DSTFT is to analyze the frequency changes of the signal through a short-time window, so that we can capture the frequency components of the signal at different time points. The formula is as follows: in, is the normal mixed BCG signal in the time domain, is the frequency domain signal after DSTFT, It is a time window function that analyzes the frequency components of the signal by sliding the window in time. The length of the window L directly determines the balance between frequency resolution and time resolution.

[0055] The choice of window length directly affects the performance of the time-frequency spectrum. A longer window helps to improve the frequency resolution, allowing the frequency components to be identified more accurately, but this will lead to a decrease in time resolution and an inability to accurately capture the instantaneous changes of the signal. A shorter window can improve the time resolution and better track the rapid changes of the signal, but the frequency resolution will be reduced. Figure 3 As shown, Figure 3 The DSTFT process and target frequency band diagram of a normal mixed ballistocardiogram signal provided by the present invention are shown. The shaded area represents the spectral range of the target heartbeat activity. This process helps better capture signal characteristics associated with sleep apnea. In particular, when noise or motion artifacts are present in the signal, DSTFT preserves signal integrity, avoiding the peak attenuation or shifting that can occur with conventional filtering.

[0056] To address signal distortion caused by large heart rate fluctuations, we used various window lengths, such as 1.3 seconds, 2.6 seconds, and 5.1 seconds. Based on these window lengths, we performed a discrete short-time Fourier transform (STFT) on the normal mixed BCG signal to obtain time-frequency spectra at various scales. For each scale, we generated a corresponding time-frequency spectrogram, thus generating time-frequency spectrograms at various scales. For example, we generated a 240×48 multi-resolution image covering the frequency range of 0.4Hz to 2.8Hz.

[0057] In the discrete short-time Fourier transform (DST) process, a signal is typically segmented into multiple overlapping or non-overlapping short-time windows. A window function is applied to each window, and a discrete Fourier transform (DFT) is performed on each windowed signal segment to obtain the spectrum within that window. The DFT results for each window are then combined to form a time-spectrum spectrum. A time-spectrum spectrum is a three-dimensional dataset, where two dimensions represent time and frequency, and the third dimension represents the signal intensity or energy at that time and frequency. Finally, the time-spectrum spectrum data is visualized to generate a time-spectrum plot. This process typically involves mapping the intensity values ​​in the time-spectrum spectrum to colors to display the relationship between time and frequency on a two-dimensional plane.

[0058] The embodiment of the present invention implements multi-scale time-frequency analysis of the ballistocardi signal by screening out normal mixed ballistocardi signals and applying multiple discrete short-time Fourier transforms with different window lengths, thereby generating time-frequency spectrograms at multiple scales. This not only improves the time-frequency resolution of signal analysis, but also enhances the recognition of signal features, enabling a more comprehensive and detailed understanding of the dynamic characteristics of the ballistocardi signal, providing powerful feature information for the detection of sleep apnea.

[0059] In some embodiments, when the regions associated with the heart rate in the time-spectrogram are screened out based on the mask matrix to obtain the time-spectrogram information, the following steps are performed for each time-spectrogram: Inputting the time-spectrogram into the ViT model to obtain a probability matrix output by the ViT model; each probability value in the probability matrix is ​​used to reflect the confidence level that the image block at the corresponding position of the probability value in the time-spectrogram is associated with the heart rate; Determine the mask of the image block corresponding to the probability value greater than or equal to the preset threshold in the probability matrix as 1; Determine the mask of the image block corresponding to the probability value less than the preset threshold in the probability matrix as 0; generating a mask matrix according to the mask of each image block in the time-spectrogram; combining the mask matrix and the time-spectrogram to generate a region associated with the heart rate; The region associated with the heart rate is determined as time-frequency spectrum information.

[0060] It should be noted that the generated spectrogram covers a frequency range of 0.4Hz to 2.8Hz. Although this frequency band can capture cardiac activity, the heart rate frequency varies widely. Directly processing all frequency bands is susceptible to spectrum leakage and sidelobe effects. Therefore, by combining HRV information, HRV is used as an important basis for generating the spectrogram mask to ensure that the spectrogram focuses on the frequency bands directly related to heart rate fluctuations, effectively separating the target frequency bands from the interference frequency bands, and thus improving the detection accuracy of sleep apnea events.

[0061] In the process of generating the mask, the ViT model is selected to segment the time-frequency spectrum, and then the segmented time-frequency spectrum is predicted in blocks to predict whether it is related to the heart rate, and the mask is further generated based on the prediction results.

[0062] The ViT model is a deep learning model for image recognition. Its ability to segment time-frequency spectrograms is based on its advantages in global information capture and multi-scale feature processing. Traditional convolutional neural networks typically rely on local receptive fields, which can cause loss of detail or ineffective capture of global features in complex time-frequency signal processing. ViT, through its self-attention mechanism, can more effectively integrate global information in the frequency and time domains. This ensures the preservation of signal details and the integration of overall spectral information, particularly in the extraction of HRV-related features. Therefore, new image segmentation training methods such as ViT can be used to process multi-resolution DSTFT time-frequency spectrograms to accurately capture the characteristics of cardiac activity frequency.

[0063] Before applying the ViT model, it is necessary to perform image segmentation training on it, which can be combined with Figure 4 , Figure 4The figure is a schematic diagram of the image segmentation training of the ViT model provided by the present invention. The ViT model training process requires obtaining training data and its label data. The training data is sampled as a spectrum sample, and its label data uses mask data generated by its real heart rate.

[0064] Combine Figure 5 Figure 5 The figure is a schematic diagram of the mask generation process used during model training provided by the present invention. First, spectrogram samples of the training data are obtained as model training data, and mask data corresponding to the true heart rate is used as labels. The specific training mask generation process uses the 3-second average heart rate, rather than the instantaneous heart rate, to determine the center of the target frequency band. Therefore, the true heart rate of the spectrogram sample is determined, and the true heart rate is used as the center frequency. Furthermore, the range of the center frequency ± a preset frequency is used as the primary frequency band of the mask. The preset frequency can be 0.2 Hz. Therefore, the range of ±0.2 Hz of the center frequency is used as the primary frequency band of the mask. This frequency band covers normal heart rate fluctuations (typically around ±0.1 Hz). This generates mask data for the training spectrogram sample. In the spectrogram sample, the mask significantly highlights the target frequency region and suppresses other frequency bands not related to cardiac activity, thereby ensuring more accurate spectral interpretation. Ultimately, the generated mask maintains spectral stability even in the presence of large heart rate fluctuations, ensuring accurate detection of sleep apnea events. Furthermore, the ViT model is trained based on the training data and its labeled mask data.

[0065] During the image segmentation training process of ViT, the ViT model first divides the input spectrogram sample P into several image patches of size 3×3. Each patch is flattened into a vector , and add position information to each patch. Position encoding is used to assign position information to each patch so that the model can perceive its position in the time-spectrogram sample.

[0066] Each patch is flattened into a vector , and its calculation formula is: , ; in, is the vector representation of each patch, and N represents the number of patches.

[0067] Furthermore, the position code is generated by sine and cosine functions to provide position information for each patch. The calculation formula is: Among them, pos represents the position of the patch, i is the dimension index, is the embedding dimension. In this way, the ViT model is able to perceive the location of each patch in the spectrogram sample.

[0068] After completing the position encoding, the ViT model calculates the information interaction between each patch through the self-attention mechanism. The calculation formula of the self-attention mechanism is as follows: Where Q is the query vector (Query), K is the key vector (Key), and V is the value vector (Value). This mechanism captures the relationship between different patches, ensuring that the model can capture global features while focusing on local details.

[0069] Following the self-attention mechanism, the encoder in the ViT model generates weights for each patch, using these weights to determine the importance of each patch in the spectrogram. The model then processes these weights through a softmax layer, converting them into a probability distribution representing the probability that each patch belongs to the target heart rate region. The output of this process is a probability matrix representing whether each patch is associated with heart rate.

[0070] After repeating the above training process, the performance of the model is evaluated on the validation set, and indicators such as accuracy, recall rate, F1 score, etc. are used to measure the effect of the model. The training is terminated when the model reaches the preset performance effect.

[0071] Deploy the trained ViT model to a real-world application. Input the spectrograms obtained during the actual application into the ViT model to obtain a probability matrix output by the ViT model. The probability matrix includes multiple probability values, each of which reflects the confidence level that the image block at the corresponding position in the spectrogram is associated with the heart rate.

[0072] Each probability value in the probability matrix is ​​compared with a preset threshold. Based on the comparison result, the mask of the image block corresponding to the probability value greater than or equal to the preset threshold in the probability matrix is ​​determined to be 1, and the mask of the image block corresponding to the probability value less than the preset threshold in the probability matrix is ​​determined to be 0, where the preset threshold is set according to actual conditions.

[0073] A mask matrix is ​​generated according to the mask of each image block in the time-spectrogram.

[0074] The mask matrix and the time-spectrum map are combined to generate a region associated with the heart rate, and the region is determined as the time-spectrum information.

[0075] The mask matrix and the time-spectrum graph can be combined by direct multiplication. The non-zero elements (usually 1) in the mask matrix will retain the corresponding values ​​in the time-spectrum graph, while the zero elements will make the corresponding values ​​in the time-spectrum graph become zero.

[0076] The embodiments of the present invention can effectively utilize the high-dimensional feature extraction capability of the ViT model to screen out regions closely related to heart rate from the time-spectrogram and generate an accurate mask matrix. Then, by combining the mask matrix with the time-spectrogram, the time-spectrogram information corresponding to the heart rate signal can be accurately extracted, providing more refined and reliable data support for the detection of sleep apnea events.

[0077] In some embodiments, based on step 103, decomposing the intrinsic mode function of the normal mixed ballistocardiographic signal includes: The optimization problem of the variational mode decomposition algorithm is defined as minimizing the sum of the bandwidths of all modes; Determining the constraints of the problem to be optimized based on the time-frequency spectrum information; Under the constraints, variational modal decomposition is performed on the normal mixed ballistocardial signal, and a set of intrinsic modal functions is obtained by solving the problem to be optimized.

[0078] It should be noted that the DSTFT method, due to its limited time and frequency resolution, struggles to accurately capture the peak amplitude of each heartbeat and its heart rate variability. To address this issue, VDM technology was introduced. This nonlinear adaptive signal decomposition method can accurately extract the target modal components from the original signal without distortion. VMD not only effectively distinguishes the target signal components but also provides a clearer periodic signal for subsequent time-domain feature extraction.

[0079] The purpose of VDM is to convert a given signal Decompose into several limited modes , each mode has its center frequency The goal of this decomposition is to minimize the bandwidth of these modes while ensuring that their cumulative sum is as close as possible to the original signal. .

[0080] Specifically, the variational mode decomposition algorithm defines the optimization problem as minimizing the sum of the bandwidths of all modes. Furthermore, based on the time-frequency spectrum information obtained from the previous segmentation task, constraints are determined in solving the optimization problem, such as the target frequency and decomposition limits. To achieve this optimization goal, VMD uses the Lagrange multiplier method to transform the constrained optimization problem into an unconstrained one.

[0081] Its optimization goal can be expressed as: is the Dirac Delta function, which tends to infinity at t=0 and has an integral value of 1. This formula describes how to calculate the modal component and center frequency Optimize to decompose the signal and retain its main information.

[0082] Under the constraints, the normal mixed BCG signal is subjected to variational mode decomposition. By solving the problem to be optimized, a set of intrinsic mode functions (IMFs) are obtained. A set of intrinsic mode functions (IMFs) represent the modes decomposed from the signal. In one embodiment, the original signal is decomposed into five IMFs according to the distribution of the center frequency. Figure 6 As shown, Figure 6 It is a schematic diagram of multiple modes obtained by the variational modal decomposition provided by the present invention.

[0083] The embodiment of the present invention defines the problem to be optimized for the variational modal decomposition algorithm and, under the constraints determined by the time-frequency spectrum information, performs precise variational modal decomposition on the normal mixed ballistocardiogram signal. It successfully extracts the intrinsic modal function reflecting the heart rate characteristics, achieves efficient decomposition of the ballistocardiogram signal, improves the accuracy of signal processing, and provides more precise and reliable data support for the detection of sleep apnea events.

[0084] In some embodiments, based on step 104, extracting the time domain waveform information from the intrinsic mode function includes: Filtering out a plurality of target modal signals close to the heart rate, respiratory rate, and IJKL wave from the set of intrinsic modal functions; Time domain waveform information is extracted from each of the target modal signals.

[0085] Specifically, multiple target modal signals with frequencies close to those of heartbeat, respiration, and IJKL waves are screened out from a set of intrinsic mode functions (IMFs). These target modal signals can clearly observe the periodicity of heartbeat, respiration, and pulse, and can extract the amplitude changes of each J peak. At the same time, rich time domain waveform information can be extracted from each target modal signal, such as IA interval, HRV, peak-to-valley difference, and J peak value.

[0086] The embodiments of the present invention screen out multiple target modal signals with frequencies close to those of heartbeat, respiration, and IJKL waves, and extract time domain waveform information from them. This effectively extracts time domain waveform information, avoids distortion of the original signal, and provides more precise and reliable data support for the detection of sleep apnea events.

[0087] In some embodiments, based on step 105, the fusion analysis of the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result includes: Inputting each time domain waveform information into a one-dimensional convolutional network to obtain a time domain feature of each time domain waveform information output by the one-dimensional convolutional network; Inputting each time-frequency spectrum information into a two-dimensional convolutional network to obtain time-frequency spectrum features of each time-frequency spectrum graph output by the two-dimensional convolutional network; Controlling each of the time domain features and each of the time-frequency spectrum features in the same time dimension to obtain a plurality of target time domain features and a plurality of target time-frequency spectrum features; Stacking the multiple target time domain features and the multiple target time-frequency spectrum features to obtain a fusion feature; A sleep apnea classification result is determined based on the fused features.

[0088] It should be noted that since the time-frequency spectrum information and time-domain waveform information come from different modalities, an effective fusion strategy is required. Therefore, a multimodal convolutional neural network (Multi-CNN) is used to achieve efficient fusion and processing of time-frequency spectrum information and time-domain waveform information.

[0089] Specifically, on the one hand, time domain features are derived from the extraction of time domain waveform information such as heartbeat intervals and respiratory peaks. These one-dimensional time domain information is processed through a one-dimensional convolutional network (Conv1D) to extract relevant time domain patterns.

[0090] Input each time domain waveform information into the one-dimensional convolutional network for processing, and its calculation formula is: in, is the input i-th time domain waveform information, is the parameter of one-dimensional convolution, is the time domain feature extracted from the i-th time domain waveform information.

[0091] On the other hand, the time-frequency features are derived from the time-frequency spectrogram after DSTFT of the normal mixed BCG signal. To capture the frequency and time information in the signal, a two-dimensional convolutional network (Conv2D) is used to extract local features, and the important time-frequency features are gradually extracted through convolution operations.

[0092] The time-frequency spectrum information is input into the two-dimensional convolutional network for processing, and the calculation formula is: in, is the jth time-frequency spectrum information of the input, j=1,2,3,....,n, corresponding to the time-frequency spectrum information of different window sizes, are the parameters of the two-dimensional convolution, is the time-frequency feature extracted from the j-th time-frequency information.

[0093] The features processed by Multi-CNN need to be controlled in the same dimension to facilitate the fusion of different modalities. That is to say, each time domain feature and each time-frequency spectrum feature is controlled in the same time dimension, so that under the aligned time dimension, multiple target time domain features and multiple target time-frequency spectrum features can be stacked in the feature dimension to obtain fused features. The multiple target time domain features and multiple target time-frequency spectrum features obtained after the unified time dimension are stacked to obtain fused features, that is, the multiple target time domain features are first stacked to obtain a time domain feature matrix, and the multiple target time-frequency spectrum features are stacked at the same time to obtain a time-frequency spectrum feature matrix, and then the time-frequency spectrum feature matrix and the time domain feature matrix are stacked to obtain fused features, as shown in the following formula: .

[0094] Based on the fusion features, the fully connected layer and the Softmax layer are used for the final feature fusion and classification, and the classification results of sleep apnea are output: in, is the parameter of the fully connected layer, G is the classification result, and S is the fusion feature.

[0095] Furthermore, to verify the effectiveness of classification model training, Grad-CAM heatmap visualization technology can be used. Grad-CAM traces the model's gradients to generate a heatmap that displays the feature regions that the model focuses on most when making decisions. The Grad-CAM visualization results show that the model accurately captures signals related to heartbeat and respiration when processing time-frequency spectrum features, verifying the model's effectiveness and reliability.

[0096] The embodiment of the present invention inputs time-domain waveform information and time-frequency spectrum information into one-dimensional and two-dimensional convolutional networks respectively, extracts time-domain and time-frequency spectrum features, aligns these features in the time dimension and stacks them, and finally generates fused features. This achieves in-depth analysis and high-precision classification of sleep apnea phenomena, improves the performance of the sleep monitoring system, and provides strong technical support for the diagnosis and treatment of sleep disorders.

[0097] All of the above processes can be combined Figure 7 , Figure 7 This is an overall architecture diagram of the sleep respiratory status monitoring method provided by the present invention.

[0098] The sleep monitoring system provided by the present invention is described below. The sleep monitoring system described below and the sleep respiratory state monitoring method described above can be referenced to each other.

[0099] Reference Figure 8 , Figure 8 It is a structural schematic diagram of the sleep monitoring system provided by the present invention.

[0100] The sleep monitoring system comprises: The time-frequency spectrum generating module 810 is configured to generate a time-frequency spectrum of a normal mixed ballistocardial signal.

[0101] The time-frequency spectrum information generating module 820 is configured to filter out the region associated with the heart rate in the time-frequency spectrum graph based on the mask matrix to obtain time-frequency spectrum information.

[0102] The modal decomposition module 830 is configured to decompose the intrinsic modal function of the normal mixed ballistocardial signal.

[0103] The time domain waveform information generating module 840 is configured to extract the time domain waveform information from the intrinsic mode function.

[0104] The sleep apnea analysis module 850 is configured to fuse and analyze the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0105] The sleep monitoring system provided by the present invention generates a time-frequency spectrum of a normal mixed cardiac shock signal and uses a mask matrix to screen out key areas associated with the heart rate in the time-frequency spectrum to ensure that the frequency bands directly related to heart rate fluctuations are highlighted in the time-frequency spectrum, thereby obtaining high-quality time-frequency spectrum information with minimal noise. In addition, considering that the resolution of time-frequency spectrum information is limited and it is difficult to capture the peak amplitude and variability of the heartbeat, distortion can be avoided by decomposing the intrinsic mode function of the normal mixed cardiac shock signal and extracting time-domain waveform information therefrom. Furthermore, through the fusion analysis of time-frequency spectrum information and time-domain waveform information, a comprehensive analysis of sleep apnea events is achieved, and accurate sleep apnea classification results are generated, providing strong technical support for the diagnosis and treatment of sleep disorders.

[0106] Furthermore, the sleep monitoring system is also used to: Collect physiological signals from the chest and abdomen through piezoelectric ceramic sensors; extracting an original cardiac signal from the physiological signal; The normal mixed cardiac ballistic signal is obtained by detecting the nonlinear characteristics of the original cardiac ballistic signal and filtering out the normal signal in the original cardiac ballistic signal.

[0107] Furthermore, the time-spectrogram generation module 810 is further configured to: Determine multiple different window lengths based on the resolution requirements in the time domain and frequency domain; Based on each of the window lengths, a discrete short-time Fourier transform is performed on the normal mixed cardiac ballistic signal to obtain time-frequency spectrograms of multiple scales.

[0108] Furthermore, the time-frequency spectrum information generating module 820 is further configured to: Inputting the time-spectrogram into the ViT model to obtain a probability matrix output by the ViT model; each probability value in the probability matrix is ​​used to reflect the confidence level that the image block at the corresponding position of the probability value in the time-spectrogram is associated with the heart rate; Determine the mask of the image block corresponding to the probability value greater than or equal to the preset threshold in the probability matrix as 1; Determine the mask of the image block corresponding to the probability value less than the preset threshold in the probability matrix as 0; generating a mask matrix according to the mask of each image block in the time-spectrogram; combining the mask matrix and the time-spectrogram to generate a region associated with the heart rate; The region associated with the heart rate is determined as time-frequency spectrum information.

[0109] Furthermore, the modal decomposition module 830 is further configured to: The optimization problem of the variational mode decomposition algorithm is defined as minimizing the sum of the bandwidths of all modes; Determining the constraints of the problem to be optimized based on the time-frequency spectrum information; Under the constraints, variational modal decomposition is performed on the normal mixed ballistocardial signal, and a set of intrinsic modal functions is obtained by solving the problem to be optimized.

[0110] Furthermore, the time domain waveform information generating module 840 is further configured to: Filtering out a plurality of target modal signals close to the heart rate, respiratory rate, and IJKL wave from the set of intrinsic modal functions; Time domain waveform information is extracted from each of the target modal signals.

[0111] Furthermore, the sleep apnea analysis module 850 is further configured to: Inputting each time domain waveform information into a one-dimensional convolutional network to obtain a time domain feature of each time domain waveform information output by the one-dimensional convolutional network; Inputting each time-frequency spectrum information into a two-dimensional convolutional network to obtain time-frequency spectrum features of each time-frequency spectrum graph output by the two-dimensional convolutional network; Controlling each of the time domain features and each of the time-frequency spectrum features in the same time dimension to obtain a plurality of target time domain features and a plurality of target time-frequency spectrum features; Stacking the multiple target time domain features and the multiple target time-frequency spectrum features to obtain a fusion feature; A sleep apnea classification result is determined based on the fused features.

[0112] It should be noted that the sleep monitoring system provided by the present invention can execute the sleep respiratory state monitoring method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0113] Figure 9 Schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 may call logic instructions in the memory 930 to execute a sleep respiratory state monitoring method, which includes: generating a time-frequency spectrum of a normal mixed ballistocardiographic signal; screening out regions associated with the heart rate in the time-frequency spectrum based on a mask matrix to obtain time-frequency spectrum information; decomposing the intrinsic mode function of the normal mixed ballistocardiographic signal; extracting time-domain waveform information from the intrinsic mode function; and fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0114] Furthermore, the logic instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0115] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the sleep breathing state monitoring method provided by the above-mentioned embodiments, the method including: generating a time-frequency spectrum of a normal mixed cardiopulmonary bypass signal; based on a mask matrix, screening out the area associated with the heart rate in the time-frequency spectrum to obtain time-frequency spectrum information; decomposing the intrinsic mode function of the normal mixed cardiopulmonary bypass signal; extracting the time-domain waveform information in the intrinsic mode function; fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0116] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the sleep respiratory state monitoring method provided in the above-mentioned embodiments, the method comprising: generating a time-frequency spectrum of a normal mixed cardiopulmonary bypass signal; based on a mask matrix, screening out areas associated with the heart rate in the time-frequency spectrum to obtain time-frequency spectrum information; decomposing the intrinsic mode function of the normal mixed cardiopulmonary bypass signal; extracting time-domain waveform information from the intrinsic mode function; and fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

[0117] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0118] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for monitoring sleep breathing status, characterized in that: Applied to sleep monitoring systems, including: Generate a time-frequency spectrum of a normal mixed heartbeat signal; Based on the mask matrix, the region associated with the heart rate in the time-frequency spectrum diagram is screened out to obtain time-frequency spectrum information; Decomposing the intrinsic mode function of the normal mixed heart ballistic signal; Extracting time domain waveform information from the intrinsic mode function; The time-frequency spectrum information and the time-domain waveform information are fused and analyzed to generate a sleep apnea classification result.

2. The sleep breathing state monitoring method according to claim 1, characterized in that: The normal mixed heartbeat signal is obtained in the following manner: The physiological signals of the chest and abdomen are collected through piezoelectric ceramic sensors; Extracting an original cardiac impulse signal from the physiological signal; By detecting the nonlinear characteristics of the original cardiac impact signal, a normal signal in the original cardiac impact signal is screened out to obtain the normal mixed cardiac impact signal.

3. The sleep breathing state monitoring method according to claim 1, characterized in that: The generating of the time-frequency spectrum of the normal mixed heartbeat signal comprises: Determine multiple different window lengths based on the resolution requirements in the time domain and frequency domain; Based on each of the window lengths, a discrete short-time Fourier transform is performed on the normal mixed cardiac impulse signal to obtain time-frequency spectrograms of multiple scales.

4. The sleep breathing state monitoring method according to claim 3, characterized in that: When the area associated with the heart rate in the time-spectrogram is screened out based on the mask matrix to obtain the time-spectrogram information, the following steps are performed for each time-spectrogram: Inputting the time-frequency spectrogram into the ViT model to obtain a probability matrix output by the ViT model; each probability value in the probability matrix is ​​used to reflect the confidence level of the association between the image block at the corresponding position of the probability value in the time-frequency spectrogram and the heart rate; Determine the mask of the image block corresponding to the probability value greater than or equal to the preset threshold in the probability matrix as 1; Determine the mask of the image block corresponding to the probability value less than the preset threshold in the probability matrix as 0; Generate a mask matrix according to the mask of each image block in the time-spectrogram; combining the mask matrix and the time-frequency spectrum to generate a region associated with the heart rate; The region associated with the heart rate is determined as time-frequency spectrum information.

5. The sleep breathing state monitoring method according to claim 4, characterized in that: The step of decomposing the intrinsic mode function of the normal mixed heartbeat signal comprises: The optimization problem of the variational mode decomposition algorithm is defined as minimizing the sum of the bandwidths of all modes; Based on each time-frequency spectrum information, determining the constraint conditions of the problem to be optimized; Under the constraint conditions, variational modal decomposition is performed on the normal mixed heart shock signal, and a set of intrinsic modal functions is obtained by solving the problem to be optimized.

6. The sleep breathing state monitoring method according to claim 5, characterized in that: The step of extracting the time domain waveform information from the intrinsic mode function comprises: Filtering out a plurality of target modal signals close to the heart rate, the respiratory rate, and the IJKL wave from the set of intrinsic modal functions; Time domain waveform information is extracted from each of the target modal signals.

7. The sleep breathing state monitoring method according to claim 6, characterized in that: The step of fusing and analyzing the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result includes: Inputting each time domain waveform information into a one-dimensional convolutional network to obtain a time domain feature of each time domain waveform information output by the one-dimensional convolutional network; Inputting each time-frequency spectrum information into a two-dimensional convolutional network to obtain the time-frequency spectrum features of each time-frequency spectrum graph output by the two-dimensional convolutional network; Controlling each of the time domain features and each of the time-frequency spectrum features in the same time dimension to obtain a plurality of target time domain features and a plurality of target time-frequency spectrum features; The multiple target time domain features and the multiple target time-frequency spectrum features are stacked to obtain a fusion feature; Based on the fused features, a sleep apnea classification result is determined.

8. A sleep monitoring system, characterized in that: include: A time-frequency spectrum generating module, used for generating a time-frequency spectrum of a normal mixed heart shock signal; A time-frequency spectrum information generation module, used for filtering out the area associated with the heart rate in the time-frequency spectrum diagram based on a mask matrix to obtain time-frequency spectrum information; A modal decomposition module, used for decomposing the intrinsic modal function of the normal mixed heart shock signal; A time domain waveform information generation module, used to extract the time domain waveform information in the intrinsic mode function; The sleep apnea analysis module is used to fuse and analyze the time-frequency spectrum information and the time-domain waveform information to generate a sleep apnea classification result.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the sleep respiratory state monitoring method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, the computer-readable storage medium comprising a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the sleep breathing state monitoring method according to any one of claims 1 to 7 are implemented.

Images