Non-contact sleep staging method and device based on ultrasonic sensing and presence state recognition, terminal and medium

Abstract

The application provides a non-contact sleep staging method and device based on ultrasonic sensing and presence state recognition, a terminal and a medium. The method comprises: obtaining echo signals by using an ultrasonic transceiving mode for sleep monitoring and demodulating the echo signals to obtain an IQ signal sequence; detecting the presence state of the IQ signal sequence to obtain a stable in-bed time interval; obtaining corresponding stable in-bed interval respiratory waveform data based on the IQ signal sequence according to the stable in-bed time interval; inputting the stable in-bed interval respiratory waveform data into a sleep staging model obtained by stage-by-stage training to obtain corresponding sleep staging results. The application introduces a presence state detection mechanism before sleep staging, effectively avoids the problem that off-bed or environmental static fragments are misjudged as sleep states, and improves the accuracy of subsequent sleep staging. At the same time, through stage-by-stage model training, the model can still obtain stable sleep staging performance under small sample conditions.

Description

Technical Field

[0001] This application relates to the field of sleep staging technology, and in particular to a non-contact sleep staging method, device, terminal and medium based on ultrasonic sensing and presence state recognition. Background Technology

[0002] Sleep is a crucial process for maintaining physiological homeostasis. During sleep, the body alternates between different sleep stages, each exhibiting significant differences in brain function, respiratory rhythm, and body movement. Clinically and in sleep medicine, sleep staging is commonly used to describe these stages. This involves dividing the entire night's sleep into continuous segments based on fixed time scales (commonly 30-second or 1-minute time windows) and labeling each segment with its corresponding sleep stage. Typical staging systems include wakefulness, light and deep sleep stages within non-rapid eye movement (NREM) sleep (e.g., N1, N2, N3), and rapid eye movement (REM) sleep. Sleep staging results can be used to further calculate key indicators such as sleep efficiency, sleep latency, number of awakenings, the proportion of each stage, and transition patterns, providing crucial information for sleep quality assessment, sleep disorder screening, and long-term health management. Therefore, achieving long-term, stable, and non-invasive sleep staging in a natural home environment has always been one of the core goals of sleep monitoring technology development.

[0003] In existing technologies, sleep staging is mainly achieved through clinical monitoring based on polysomnography (PSG) and daily monitoring based on wearable devices. PSG simultaneously collects multiple physiological signals such as electroencephalography (EEG), electrooculography (EOG), electromyography (EMG), and electrocardiography (ECG), and staging is completed by technicians or automatic algorithms according to standard rules. It has high accuracy but the equipment is expensive, complex to set up, and uncomfortable to wear, making it difficult to use for long-term continuous monitoring at home. Wearable devices usually use signals such as acceleration, heart rate / blood oxygenation to infer sleep status. They have lower deployment costs, but they rely on long-term wear by users and are easily affected by wearer compliance, individual differences, and sensor quality fluctuations. Their long-term stability and staging accuracy are limited in complex home scenarios.

[0004] To reduce the burden of use and achieve long-term, non-invasive home sleep monitoring, wireless sensing-based non-contact sleep monitoring technologies have emerged in recent years. These technologies utilize signals such as radar, WiFi, and acoustics to sense respiration, heart rate, and body movement, and further infer sleep stages. While these methods offer advantages such as not requiring attached electrodes or worn devices and being suitable for long-term deployment, they still face significant challenges in real-world home environments. Firstly, wireless signals are easily affected by environmental noise, multipath effects, obstruction, and changes in spatial layout, leading to insufficient stability in physiological signal extraction. Secondly, many methods primarily focus on physiological characteristics after falling asleep during modeling, lacking reliable characterization of user behavior changes throughout the day, such as being in / out of / in bed. This can easily lead to the misinclusion of unoccupied or non-sleep segments in the sleep analysis process, resulting in errors in sleep interval delineation and staged results. Furthermore, labeled data for non-contact sensing is often difficult to obtain, with small data volumes and significant scene variations, limiting the model's generalization ability across environments and devices.

[0005] Among the aforementioned non-contact solutions, ultrasonic-based acoustic sensing technology, which utilizes speakers and microphones to transmit / receive high-frequency signals inaudible to the human ear, offers advantages such as privacy, no need for cameras, insensitivity to light, and readily available equipment, making it a potential candidate for long-term operation in home environments. Ultrasonic echoes contain subtle changes caused by human respiration and micro-movements, providing important clues for sleep stage inference. However, two key issues remain to be addressed when applying ultrasonic sensing for "all-day sleep staging":

[0006] (1) Presence state recognition problem: During long-term home monitoring, users are not always in bed. If the system cannot reliably determine "occupant / unoccupied" and impose temporal consistency constraints on the results, it is easy to misclassify static environments or short-term intense motion segments after leaving the bed as sleep segments, resulting in inaccurate sleep interval division and further affecting the reliability of sleep staging. Therefore, a robust presence state detection mechanism is needed to provide a reliable and effective analysis interval for subsequent sleep staging.

[0007] (2) The problem of small sample learning caused by the scarcity of labeled data: To obtain ultrasound data with standard sleep stage labels, it is usually necessary to collect data simultaneously with PSG and perform manual annotation, which is costly and difficult, resulting in a limited scale of available labeled data. Under small sample conditions, deep models are prone to overfitting, and there are modal differences and domain shifts between publicly available large-scale data (mostly PSG or other modalities) and ultrasound reconstructed signals, which often limits the effectiveness of direct transfer. Therefore, it is necessary to study learning strategies that can effectively utilize external data knowledge and alleviate cross-modal differences under small sample conditions in order to improve staging accuracy and generalization performance. Summary of the Invention

[0008] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a non-contact sleep staging method, device, terminal and medium based on ultrasonic sensing and presence state recognition, so as to solve the problem that the existing technology is prone to errors in sleep interval division, resulting in low accuracy of sleep staging results.

[0009] To achieve the above and other related objectives, a first aspect of this application provides a non-contact sleep staging method based on ultrasonic sensing and presence state recognition, comprising: acquiring echo signals and demodulating them using an ultrasonic transceiver method for sleep monitoring to obtain an IQ signal sequence; performing presence state detection on the IQ signal sequence to obtain a stable bed-resting time interval; obtaining corresponding stable bed-resting time interval respiratory waveform data based on the IQ signal sequence according to the stable bed-resting time interval; and inputting the stable bed-resting time interval respiratory waveform data into a sleep staging model obtained through phased training to obtain a corresponding sleep staging result; wherein, the phased training includes: pre-training, domain adaptive training, and fine-tuning training.

[0010] In some embodiments of the first aspect of this application, the presence state detection of the IQ signal sequence to obtain a stable bedside time interval includes: performing motion detection on the IQ signal sequence to obtain a corresponding motion detection result; determining the sequence portion in the IQ signal sequence to be detected for respiration based on the motion detection result, and performing respiration detection on the sequence portion to be detected for respiration to obtain an initial respiration detection result; determining whether signal enhancement processing is required based on the motion detection result and the initial respiration detection result; if signal enhancement processing is required, enhancing the candidate sequence portion in the determined IQ signal sequence to obtain an enhanced sequence portion; performing respiration detection again on the enhanced sequence portion to obtain a final respiration detection result; if signal enhancement processing is not required, using the initial respiration detection result as the final respiration detection result; and fusing and timing-correcting the motion detection result and the final respiration detection result to obtain a stable bedside time interval.

[0011] In some embodiments of the first aspect of this application, motion detection is performed on the IQ signal sequence to obtain corresponding motion detection results, including: calculating the correlation coefficient of the amplitude vectors of every two adjacent moments in the IQ signal sequence; determining a sliding time window, statistically analyzing the proportion of correlation coefficients lower than a set correlation coefficient threshold among the correlation coefficients corresponding to each sliding time window, obtaining the proportion of each sliding time window, and determining the time interval corresponding to the sliding time window with a proportion lower than a preset proportion threshold as having motion time, thereby generating the corresponding motion detection result.

[0012] In some embodiments of the first aspect of this application, the enhancement process includes: sequentially performing detrending and normalization processing, bandpass filtering processing, and channel filtering and fusion processing on the candidate sequence portion.

[0013] In some embodiments of the first aspect of this application, fusing and timing-correcting the motion detection results and the final respiratory detection results to obtain a stable bed-stay time interval includes: fusing the motion detection results and the final respiratory detection results to obtain a fused result; labeling the fused result with an existence state or an unoccupied state according to a set time interval length to obtain a labeled result; correcting the labeled result according to a timing correction rule to obtain a corrected result, and outputting the time interval corresponding to the existence state in the corrected result as the stable bed-stay time interval; wherein, the timing correction rule includes: if the interval between two adjacent time intervals labeled as having an existence state is less than a preset time threshold, then the two adjacent time intervals labeled as having an existence state are merged.

[0014] In some embodiments of the first aspect of this application, the training process of the sleep staging model specifically includes: constructing a sleep staging network; using the acquired labeled sleep data as a pre-training dataset to pre-train the sleep staging network to obtain a preliminary sleep staging model; using the labeled sleep data and the acquired unlabeled first ultrasound sleep data as a domain adaptive training dataset to perform domain adaptive training on the preliminary sleep staging model to obtain an intermediate sleep staging model; and using the acquired labeled second ultrasound sleep data as a fine-tuning training dataset to fine-tune the intermediate sleep staging model to obtain a sleep staging model.

[0015] In some embodiments of the first aspect of this application, the sleep staging network includes: a feature extraction module and a temporal modeling and classification module connected to the feature extraction module; wherein the feature extraction module adopts a ResNet18 architecture, and the temporal modeling and classification module adopts a multilayer bidirectional long short-term memory network.

[0016] To achieve the above and other related objectives, a second aspect of this application provides a non-contact sleep staging device based on ultrasonic sensing and presence state recognition, comprising: a demodulation module for acquiring echo signals via ultrasonic transceiver for sleep monitoring and demodulating them to obtain an IQ signal sequence; a sequence processing module for performing presence state detection on the IQ signal sequence to obtain a stable bed-resting time interval; a sleep staging module for obtaining corresponding stable bed-resting interval respiratory waveform data based on the IQ signal sequence according to the stable bed-resting time interval; and inputting the stable bed-resting interval respiratory waveform data into a sleep staging model obtained through phased training to obtain a corresponding sleep staging result; wherein the phased training includes: pre-training, domain adaptive training, and fine-tuning training.

[0017] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the non-contact sleep staging method based on ultrasonic sensing and presence state recognition.

[0018] To achieve the above and other related objectives, a fourth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the non-contact sleep staging method based on ultrasonic sensing and presence state recognition.

[0019] As described above, the non-contact sleep staging method, device, terminal, and medium based on ultrasonic sensing and presence state recognition of this application have the following beneficial effects: This application introduces a presence state detection mechanism before sleep staging, effectively avoiding the problem of misjudging the state of sleep when the patient is out of bed or in a static environment, thus improving the accuracy of subsequent sleep staging. Simultaneously, through staged training, the model can still achieve stable staging performance under small sample conditions. Attached Figure Description

[0020] Figure 1 The diagram shown is a flowchart of a non-contact sleep staging method based on ultrasonic sensing and presence state recognition in one embodiment of this application.

[0021] Figure 2 The diagram shown is a schematic representation of the specific process of sleep staging in one embodiment of this application.

[0022] Figure 3 The diagram shown is a schematic representation of a state detection process in a specific embodiment of this application.

[0023] Figure 4 The diagram shown is a schematic representation of the sleep staging network in a specific embodiment of this application.

[0024] Figure 5 The diagram shown is a schematic block diagram of a non-contact sleep staging device based on ultrasonic sensing and presence state recognition in one embodiment of this application.

[0025] Figure 6 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0028] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0029] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0030] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1This document illustrates a flowchart of a non-contact sleep staging method based on ultrasonic sensing and presence state recognition, as described in an embodiment of the present invention. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition in this embodiment mainly includes the following steps:

[0031] Step S101: Acquire echo signals using ultrasound transceiver method through sleep monitoring and demodulate them to obtain IQ signal sequences.

[0032] In one embodiment, the ultrasonic transceiver for sleep monitoring can be implemented using a smart device with audio hardware. Examples of smart devices with audio hardware include smartphones, computers, smart speakers, etc. In these devices, the audio hardware can be equipped with the function of transmitting and / or receiving ultrasonic waves for sleep monitoring through software upgrades or built-in integration.

[0033] In one embodiment, an ultrasonic wave for sleep monitoring can be transmitted via one smart device and the echo signal can be received via another smart device. Alternatively, the transmission and reception of the ultrasonic wave for sleep monitoring can be performed using the same smart device. It should be noted that the number of smart devices can be selected based on actual needs, and this invention does not limit this selection.

[0034] In one embodiment, the ultrasound used for sleep monitoring is ultrasound within the range inaudible to the human ear. For example, the inaudible range can be 17-24 kHz. The ultrasound signal is in the form of frequency-modulated continuous wave (FMCW), with the frequency varying linearly with time. It should be understood that an FMCW signal is a common signal form used in ultrasound sensing, where the frequency modulation method causes the signal frequency to increase linearly within a period. The expression for the FMCW signal S(t) is as follows:

[0035] (1)

[0036] in, Let B be the starting frequency of the signal, B be the signal bandwidth, T be the modulation period, and t be the time variable. The frequency of the signal changes linearly over time, and the frequency change is determined by... control.

[0037] When the ultrasonic signal is emitted, it will be reflected when it encounters a human body or an object in the environment. The expression for the echo signal R(t) received by the smart device is as follows:

[0038] (2)

[0039] in, Let be the attenuation coefficient of the i-th reflection path. Let be the time delay of the i-th path, representing the propagation delay of the signal from transmission to reception. , Let be the length of the i-th path, and c be the speed of sound. The echo signal R(t) is the superposition of signals from all reflected paths, and the signal frequency and time delay of each reflected path depend on its reflection source.

[0040] Based on this, in order to extract phase information related to human motion from the echo signal, the echo signal needs to be demodulated. Specifically, the echo signal is compared with the transmitted signal. The product is multiplied to obtain the result. The multiplied result is then low-pass filtered to remove high-frequency components, resulting in a filtered multiplied result. Next, an FFT (Fast Fourier Transform) is performed on the filtered multiplied result to obtain complex numbers consisting of in-phase (I) and quadrature (Q) components corresponding to multiple frequency bands. For the k-th frequency band... ,here It is the imaginary unit.

[0041] Through the above processing, an IQ signal sequence varying over time is obtained, and its phase changes reflect the changes in the ultrasound propagation path caused by human respiration and stillness. To better illustrate the ultrasonic sleep monitoring device, a specific application scenario is provided: in bedside sleep monitoring at home, a smart device with audio hardware is deployed at the bedside, periodically emitting FMCW ultrasound signals, and the smart device with audio hardware collects environmental echoes.

[0042] Step S102: Perform presence-state detection-based processing on the IQ signal sequence to obtain stable bedside respiratory data.

[0043] After acquiring the IQ signal sequence of the ultrasonic echo components, the IQ signal sequence is further processed to determine whether a user exists in the monitoring environment and whether the user is in a stable bed-bound state. For ease of explanation, the time interval corresponding to the IQ signal sequence will be referred to as the total time interval below.

[0044] like Figure 2 as well as Figure 3 As shown, motion detection is performed first. When a user gets into bed, gets out of bed, or turns over, the human body produces significant movement, causing changes in the distribution of the ultrasound echo signal along the distance dimension. Therefore, motion events are detected by the correlation between the amplitude distributions of adjacent time points. Specifically, the correlation coefficient of the amplitude vectors of every two adjacent time points in the IQ signal sequence is calculated. Assuming the amplitude vector at time t is A(t), the correlation coefficient between the amplitude vectors of adjacent time points t and t+1 is calculated. :

[0045] (3)

[0046] Where i represents the i-th frequency band. It should be understood that the amplitude vector is a quantitative description of the amplitude characteristics of the complex IQ signal. Essentially, it is a one-dimensional vector composed of the amplitude values ​​of the IQ complex signal at each time moment, reflecting the change in the signal's energy strength over time.

[0047] Finally, the correlation coefficients for each moment within the total time interval are obtained.

[0048] When significant motion occurs, the correlation coefficient between adjacent time points decreases significantly. Further, the length of the sliding time window is determined, and the sliding time window slides along the time axis of the total time interval with a certain step size. The proportion of correlation coefficients below a set correlation coefficient threshold for each sliding time window is calculated to obtain the proportion of each sliding time window. The time intervals corresponding to sliding windows with proportions below a preset proportion threshold are determined to be time intervals where motion exists. The final motion detection result includes time intervals (one or more) within the total time interval that are determined to have motion. It should be noted that the step size, the length of the sliding time window, the correlation coefficient threshold, and the proportion threshold can all be set according to requirements, and this invention does not limit them.

[0049] Next, time intervals outside the periods where movement was detected are considered time intervals where no significant movement was detected. For example, if the total time interval is from 1:00 AM to 6:00 AM, and 2:30 AM to 2:35 AM is determined to be a period where movement was detected, then the time intervals from 1:00 AM to 2:30 AM and from 2:35 AM to 6:00 AM are time intervals where no significant movement was detected. Further, the IQ signal sequence portion corresponding to the time intervals where no significant movement was detected is designated as the sequence portion to be tested for respiration. During the periods when no significant movement was detected, human respiration causes low-frequency, near-periodic changes in the ultrasound echo signal. Therefore, respiration detection is performed on the sequence portion to be tested for respiration to determine whether stable human respiratory activity exists.

[0050] Specifically, the sequence portion to be detected for respiration is subjected to frequency domain analysis according to the set respiration detection time interval. The spectral energy is calculated within a preset respiration frequency range, and the energy within this frequency band is compared with the overall spectral energy. If the energy in the respiration frequency band is significantly higher than the background level, a respiration signal is determined to exist within that respiration detection time interval; otherwise, no respiration is detected. The initial respiration detection result is then obtained. This initial respiration detection result includes the time interval within which a respiration signal was determined to exist.

[0051] In some cases, due to changes in posture, occlusion, or a decrease in signal-to-noise ratio, the respiratory signal may be weak, leading to instability in single-channel detection. Therefore, signal enhancement processing is required to improve detection stability. Specifically, based on the motion detection results and the initial respiratory detection results, it is determined whether signal enhancement processing is needed. When the motion detection results and the initial respiratory detection results do not cover the total time interval, signal enhancement processing is considered necessary. In cases requiring signal enhancement processing, the portion of the IQ signal sequence not covered by the motion detection results and the initial respiratory detection results (i.e., the candidate sequence portion) is enhanced to obtain the enhanced sequence portion. The enhanced sequence portion is then subjected to the aforementioned respiratory detection again, yielding the enhanced portion's respiratory detection result. The enhanced portion's respiratory detection result is merged with the initial respiratory detection result to obtain the final respiratory detection result. When the motion detection results and the initial respiratory detection results cover the total time interval, signal enhancement processing is considered unnecessary. The initial respiratory detection result is used as the final respiratory detection result. The final respiratory detection result includes the time interval within which a respiratory signal was detected within the total time interval.

[0052] The enhancement process includes the following steps:

[0053] 1) Detrending and normalization processing: Detrending processing is performed on the candidate sequence to eliminate slowly changing background components, and the signal is normalized for subsequent analysis.

[0054] 2) Bandpass filtering: Apply bandpass filtering to the normalized signal to retain only the signal components within the breathing frequency range.

[0055] 3) Channel screening and fusion: Correlation analysis is performed on respiratory signals from different distance channels or frequency bands, channels with similar changing trends are selected, and weighted superposition is performed according to the signal-to-noise ratio of each channel to obtain the enhanced sequence portion.

[0056] The enhanced sequence portion is used again for respiratory detection to compensate for detection failures in cases of weak breathing or micro-movement.

[0057] Furthermore, the motion detection results are fused with the final respiratory detection results to obtain a fused result. Specifically,

[0058] The motion detection results are merged with the final respiratory detection results to obtain a fusion result. The fusion result mainly includes the time intervals within the total time interval where respiratory signals were detected and the time intervals where motion was determined to exist. Then, according to a set time length (for example, 1 minute), the total time interval is divided into multiple time intervals to be labeled. When motion or respiratory signals are detected, the corresponding time interval to be labeled is marked as present; when no motion or respiratory signals are detected, the corresponding time interval to be labeled is marked as unoccupied.

[0059] To avoid frequent state switching due to short-term false detections, this invention introduces a timing correction rule to correct the existence status. Specifically, the timing correction rule includes: if the interval between two adjacent time intervals marked as having an existence status is less than a preset time threshold, then the two adjacent time intervals marked as having an existence status are merged. It should be noted that the time threshold can be set to 1 hour. It can also be set to other values; this invention does not limit this.

[0060] To better illustrate the timing correction rules, the following explanation uses examples:

[0061] First, let's clarify the concept of adjacent time intervals marked as "existing": Time interval A and time interval B are two consecutive time intervals. Since both time interval A and time interval B are marked as "existing," they can be considered adjacent time intervals marked as "existing." Second, time interval A and time interval B are two non-consecutive time intervals. Both are marked as "existing," but there are no time intervals marked as "existing" between them; only time intervals marked as "unoccupied" exist. In this case, time interval A and time interval B can also be considered adjacent time intervals marked as "existing." The timing correction in this embodiment addresses the second case.

[0062] Next, the timing correction rules will be explained. Time intervals C and D are two adjacent time intervals marked as present. Between time intervals C and D, there are one or more time intervals marked as unoccupied. When the time interval between time intervals C and D is less than a preset time threshold, all these unoccupied time intervals between C and D are changed to present. This process is called merging, to avoid short-term interruptions affecting the division of sleep intervals throughout the night. When the time interval between time intervals C and D is greater than the preset time threshold, no modification is needed.

[0063] Using the above method, this invention can generate continuous and stable bed rest intervals during long-term monitoring, clearly identifying continuous time periods requiring sleep staging. Based on the actual collected data, statistical analysis is performed on the output results of each processing step of this invention.

[0064] In one specific embodiment, such as Figure 3 As shown, the total time interval is the time interval represented by the ground truth, where the stable bed interval is as indicated by arrows A and B.

[0065] Step S103: Based on the stable bed rest time interval, obtain the corresponding stable bed rest interval respiratory waveform data based on the IQ signal sequence; input the stable bed rest interval respiratory waveform data into the sleep staging model obtained through phased training to obtain the corresponding sleep staging result.

[0066] In one embodiment, respiratory waveform data within the stable bed-staying time interval is extracted from the IQ signal sequence based on the stable bed-staying time interval. It should be understood that the respiratory waveform extraction method can follow existing methods and is not limited here.

[0067] In one embodiment, respiratory waveform data within a stable bedtime interval is divided into time intervals based on a preset basic time unit, resulting in multiple time intervals of respiratory waveform data within the stable bedtime interval. The sleep staging model classifies the respiratory waveform data within each time interval and outputs the corresponding sleep stage result. The sleep stage results corresponding to the respiratory waveform data within each time interval are arranged chronologically to form the sleep staging results. The sleep stage results are categorized into four types: awake, light sleep, deep sleep, and REM sleep. Correspondingly, the sleep stage labels summarized during model training are also categorized into four types: awake, light sleep, deep sleep, and REM sleep.

[0068] For example, the basic time unit is 1 minute, and a sleep stage label is output for each minute of stable bed breathing data.

[0069] In one embodiment, in an ultrasound sleep monitoring scenario, the amount of labeled data available is limited, and there are differences in signal distribution between different data sources. Therefore, this invention employs a phased training process that sequentially includes pre-training, domain adaptive training, and fine-tuning training to fully utilize external data knowledge and gradually adapt to the respiratory signal characteristics under ultrasound sensing conditions, thereby obtaining stable sleep staging results under small sample conditions.

[0070] The following explains the specific process of phased training:

[0071] First, a sleep staging network is constructed. This network includes a feature extraction module, used to extract representations reflecting respiratory rhythm and morphological features from respiratory waveforms. It also includes a temporal modeling and classification module connected to the feature extraction module, used to model the feature representations over continuous time, learn the changes in sleep stages over time, and output corresponding sleep stage labels. The feature extraction module adopts the ResNet18 architecture, specifically consisting of four ResBlock units connected sequentially; the last ResBlock unit is connected to an FC layer. Each ResBlock unit includes a Conv1D layer and a BN layer connected sequentially. The input to the Conv1D layer is the input to the ResBlock unit, and the input of the Conv1D layer is concatenated with the output of the BN layer before being input to the ReLU layer. The temporal modeling and classification module employs a multi-layer bidirectional Long Short-Term Memory (LSTM) network. The multi-layer bidirectional LSTM network is an advanced structure of LSTM, combining a multi-layer structure and a bidirectional mechanism to more efficiently capture long-term dependencies and bidirectional contextual information in sequence data. The multilayer bidirectional long short-term memory network includes multiple BiLSTM (Bidirectional Long Short-Term Memory) modules.

[0072] Next, due to the limited amount of available labeled data, directly training a deep model on this data can easily lead to performance instability. Therefore, we now utilize large-scale labeled sleep data from other data sources as a pre-training dataset to train the aforementioned sleep staging network and obtain a preliminary sleep staging model. The labeled sleep data includes multiple respiratory waveform data and their corresponding sleep stage labels. During pre-training, the model's respiratory waveform data is used as input, and the corresponding sleep stage labels are used as supervision signals for training. Assume the model's predicted sleep stage at time j is... The actual sleep stage is labeled as The optimization objective during the pre-training phase is defined as follows:

[0073] (4)

[0074] in, Let N represent the cross-entropy loss function, and N represent the number of time steps in the pre-training dataset. This represents the loss function during the pre-training phase.

[0075] Through this pre-training process, the model can acquire an initial ability to judge sleep stage tasks, providing a foundation for subsequent adaptation to ultrasound data.

[0076] Furthermore, echo signals are acquired using the methods described in the above embodiments, and respiratory waveform signals are extracted from them as target ultrasound sleep data. The dataset used for pre-training differs from the target ultrasound sleep data in terms of signal source and distribution characteristics. Directly applying the preliminary sleep staging model obtained from pre-training to the ultrasound sleep data can easily lead to performance degradation due to inconsistent distribution. Therefore, after pre-training, domain adaptive training is introduced to reduce the feature differences between the two types of data.

[0077] For domain-adaptive training, the source domain (the dataset used for pre-training) and the target domain (target ultrasound sleep data) are used as the training datasets to train the initial sleep staging model and obtain the intermediate sleep staging model. It is important to note that the target ultrasound sleep data does not have any sleep stage labels. During this training phase, by constraining the distribution of the source and target domains in the feature space, the features extracted by the model are made consistent across different data sources.

[0078] Assume the source domain features are represented as The target domain features are represented as The optimization objective in the domain adaptation phase is:

[0079] (5)

[0080] in, This represents a distance function used to measure the difference in characteristic distributions. Let N represent the cross-entropy loss function, and N represent the number of time points in the dataset.

[0081] In this stage, the model mainly updates the parameters of the feature extraction module to gradually adapt it to the respiratory signal features under ultrasound sensing, without involving the classification loss of sleep stages.

[0082] After completing the domain-adaptive training, the initial sleep staging model is further fine-tuned using a small amount of ultrasound sleep data labeled with sleep stages to improve the model's actual staging performance in the target scenario. Specifically, in the fine-tuning phase, the model is trained using only the supervision signal from the sleep staging task itself. Assume the model's prediction for minute j is... The corresponding real label is The optimization objective in the fine-tuning phase is defined as:

[0083] (6)

[0084] It should be understood that echo signals are acquired and respiratory waveform signals are extracted as ultrasound sleep data using the same method described in the above embodiments. Through this fine-tuning process, the model can further adapt to the specific characteristics of ultrasound sleep data while retaining the knowledge learned in the pre-training and domain adaptive training phases, thereby achieving stable sleep staging performance under small sample conditions.

[0085] The advantages of this invention are:

[0086] This invention proposes a non-contact sleep staging method based on ultrasonic sensing and in-sleep state perception. By processing and analyzing ultrasonic echo signals, it achieves long-term, continuous monitoring of users' sleep states in a home environment. Compared with existing technologies, this invention introduces an in-sleep state detection mechanism before sleep staging. Through joint analysis of motion and respiratory features, combined with time consistency constraints, it generates stable in-bed intervals, effectively avoiding the problem of misclassifying out-of-bed or static environmental segments as sleep states, thereby improving the reliability of sleep interval division. Based on this, this invention uses respiratory waveforms of basic time units as input for four-category sleep staging, reflecting changes in sleep structure throughout the night while maintaining temporal resolution. Addressing the issues of high cost and limited sample size for ultrasonic sleep data annotation, this invention employs a training process combining pre-training, domain adaptation, and fine-tuning. While fully utilizing existing sleep data knowledge, it explicitly considers the distribution differences between different data sources, enabling the model to achieve stable staging performance even with small sample sizes. Through the synergistic effect of the above-mentioned technical means, the present invention improves the applicability and stability of non-contact sleep staging in complex home environments without the need for wearing devices or attaching electrodes, and has good practical application value and engineering implementation prospects.

[0087] Figure 5 This is a schematic block diagram of a non-contact sleep staging device based on ultrasonic sensing and presence state recognition provided in an embodiment of this application. Figure 5 As shown, the non-contact sleep staging device 500 based on ultrasonic sensing and presence state recognition includes:

[0088] The demodulation module 501 is used to acquire echo signals and demodulate them using ultrasound transceiver for sleep monitoring to obtain IQ signal sequences.

[0089] Sequence processing module 502 is used to perform presence state detection on the IQ signal sequence to obtain a stable bed rest time interval;

[0090] The sleep staging module 503 is used to obtain the corresponding stable bed rest interval respiratory waveform data based on the IQ signal sequence according to the stable bed rest interval; input the stable bed rest interval respiratory waveform data into the sleep staging model obtained through phased training to obtain the corresponding sleep staging result; wherein, the phased training includes: pre-training, domain adaptive training and fine-tuning training.

[0091] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0092] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0093] In one embodiment, the presence state detection of the IQ signal sequence to obtain a stable bedside time interval includes: performing motion detection on the IQ signal sequence to obtain a corresponding motion detection result; determining the sequence portion of the IQ signal sequence to be detected for respiration based on the motion detection result, and performing respiration detection on the sequence portion to be detected for respiration to obtain an initial respiration detection result; determining whether signal enhancement processing is required based on the motion detection result and the initial respiration detection result; if signal enhancement processing is required, enhancing the candidate sequence portion in the determined IQ signal sequence to obtain an enhanced sequence portion; performing respiration detection again on the enhanced sequence portion to obtain a final respiration detection result; if signal enhancement processing is not required, using the initial respiration detection result as the final respiration detection result; and fusing and timing-correcting the motion detection result and the final respiration detection result to obtain a stable bedside time interval.

[0094] In one embodiment, motion detection is performed on the IQ signal sequence to obtain corresponding motion detection results, including: calculating the correlation coefficient of the amplitude vectors of every two adjacent moments in the IQ signal sequence; determining a sliding time window; statistically analyzing the proportion of correlation coefficients below a set correlation coefficient threshold for each sliding time window; obtaining the proportion of each sliding time window; and determining the time interval corresponding to the sliding time window with a proportion below a preset proportion threshold as having motion time, thereby generating the corresponding motion detection result.

[0095] In one embodiment, the enhancement process includes: sequentially performing detrending and normalization processing, bandpass filtering processing, and channel filtering and fusion processing on the candidate sequence portion.

[0096] In one embodiment, fusing and timing-correcting the motion detection results and the final respiratory detection results to obtain a stable bed-stay time interval includes: fusing the motion detection results and the final respiratory detection results to obtain a fused result; labeling the fused result with an existence state or an unoccupied state according to a set time interval length to obtain a labeled result; correcting the labeled result according to a timing-correction rule to obtain a corrected result, and outputting the time interval corresponding to the existence state in the corrected result as the stable bed-stay time interval; wherein, the timing-correction rule includes: if the interval between two adjacent time intervals labeled as having an existence state is less than a preset time threshold, then the two adjacent time intervals labeled as having an existence state are merged.

[0097] In one embodiment, the training process of the sleep staging model specifically includes: constructing a sleep staging network; using the acquired labeled sleep data as a pre-training dataset to pre-train the sleep staging network and obtain a preliminary sleep staging model; using the labeled sleep data and the acquired unlabeled first ultrasound sleep data as a domain adaptive training dataset to perform domain adaptive training on the preliminary sleep staging model and obtain an intermediate sleep staging model; and using the acquired labeled second ultrasound sleep data as a fine-tuning training dataset to fine-tune the intermediate sleep staging model and obtain a sleep staging model.

[0098] In one embodiment, the sleep staging network includes: a feature extraction module and a temporal modeling and classification module connected to the feature extraction module; wherein the feature extraction module adopts a ResNet18 architecture, and the temporal modeling and classification module adopts a multilayer bidirectional long short-term memory network.

[0099] Figure 6 This is a schematic block diagram of the electronic terminal provided in an embodiment of this application. Figure 6 As shown, the electronic terminal includes at least one processor 601, a memory 602, at least one network interface 603, and a user interface 605. The various components in the device are coupled together via a bus system 604. It is understood that the bus system 604 is used to implement communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 6 The general will label all buses as bus systems.

[0100] The user interface 605 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0101] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0102] In this embodiment of the invention, the memory 602 is used to store various types of data to support the operation of the electronic terminal 600. Examples of this data include: any executable program for operation on the electronic terminal 600, such as the operating system 6021 and application program 6022; the operating system 6021 contains various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 6022 may contain various applications, such as a media player, browser, etc., for implementing various application services. The implementation of the non-contact sleep staging method based on ultrasonic sensing and presence state recognition provided in this embodiment of the invention can be included in the application program 6022.

[0103] The methods disclosed in the above embodiments of the present invention can be applied to processor 601, or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 601 or by instructions in the form of software. The processor 601 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 601 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0104] In an exemplary embodiment, the electronic terminal 600 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to perform the aforementioned method.

[0105] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 1 The embodiment shown illustrates a non-contact sleep staging method based on ultrasonic sensing and presence state recognition.

[0106] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 1 The embodiment shown illustrates a non-contact sleep staging method based on ultrasonic sensing and presence state recognition.

[0107] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0108] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0114] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0116] In summary, this application provides a non-contact sleep staging method, device, terminal, and medium based on ultrasonic sensing and presence state recognition. The method includes: acquiring echo signals via ultrasonic transceiver during sleep monitoring and demodulating them to obtain an IQ signal sequence; performing presence state detection on the IQ signal sequence to obtain a stable bed-being time interval; obtaining corresponding respiratory waveform data for the stable bed-being time interval based on the IQ signal sequence; and inputting the respiratory waveform data for the stable bed-being time interval into a sleep staging model obtained through phased training to obtain the corresponding sleep staging result. This application introduces a presence state detection mechanism before sleep staging, effectively avoiding the problem of misclassifying bed-being or static environmental segments as sleep states, thus improving the accuracy of subsequent sleep staging. Simultaneously, through phased model training, the model can still achieve stable sleep staging performance under small sample conditions. Therefore, this application effectively overcomes various shortcomings of existing technologies and has high industrial applicability.

[0117] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A non-contact sleep staging method based on ultrasonic sensing and presence state recognition, characterized in that, include: The echo signal was acquired by ultrasound transceiver during sleep monitoring and then demodulated to obtain the IQ signal sequence. The presence state of the IQ signal sequence is detected to obtain the stable bed rest time interval; Based on the stable bed rest time interval, the corresponding respiratory waveform data of the stable bed rest interval is obtained based on the IQ signal sequence; The stable respiratory waveform data in the bed zone is input into the sleep staging model obtained through phased training to obtain the corresponding sleep staging results; wherein, the phased training includes: pre-training, domain adaptive training and fine-tuning training.

2. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 1, characterized in that, The presence state of the IQ signal sequence is detected to obtain a stable bed rest time interval, including: Motion detection is performed on the IQ signal sequence to obtain the corresponding motion detection results; Based on the motion detection results, determine the sequence portion of the IQ signal sequence to be detected for breathing, and perform breathing detection on the sequence portion to be detected for breathing to obtain an initial breathing detection result; Based on the motion detection results and the initial respiratory detection results, determine whether signal enhancement processing is required; If signal enhancement processing is required, the candidate sequence portion in the determined IQ signal sequence is enhanced to obtain the enhanced sequence portion; the enhanced sequence portion is then subjected to respiratory detection again to obtain the final respiratory detection result. Without the need for signal enhancement processing, the initial respiratory detection result is used as the final respiratory detection result; The motion detection results and the final respiratory detection results are fused and time-series corrected to obtain a stable bed rest time interval.

3. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 2, characterized in that, Motion detection is performed on the IQ signal sequence to obtain the corresponding motion detection results, including: Calculate the correlation coefficient of the amplitude vectors of every two adjacent time points in the IQ signal sequence; Determine the sliding time window, count the proportion of correlation coefficients below the set correlation coefficient threshold for each sliding time window, obtain the proportion of each sliding time window, and determine the time interval corresponding to the sliding time window with the proportion below the preset proportion threshold as having motion time, and then generate the corresponding motion detection result.

4. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 2, characterized in that, The enhancement process includes: sequentially performing detrending and normalization processing, bandpass filtering processing, and channel filtering and fusion processing on the candidate sequence portion.

5. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 2, characterized in that, The motion detection results and the final respiratory detection results are fused and time-series corrected to obtain a stable bed rest time interval, including: The motion detection results and the final respiratory detection results are fused to obtain a fused result; According to the set time interval length, the fusion result is labeled with either an existence state or an unmanned state to obtain the labeled result; According to the time-series correction rules, the labeled results are corrected to obtain the corrected results, and the time intervals corresponding to the states in the corrected results are output as stable bed rest time intervals; wherein, the time-series correction rules include: If the time interval between two adjacent time intervals marked as existing is less than a preset time threshold, the two adjacent time intervals marked as existing will be merged.

6. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 1, characterized in that, The training process of the sleep staging model specifically includes: Constructing a sleep staging network; The acquired labeled sleep data is used as a pre-training dataset to pre-train the sleep staging network, thereby obtaining a preliminary sleep staging model. The labeled sleep data and the acquired unlabeled first ultrasound sleep data are used as the dataset for domain adaptive training. The preliminary sleep staging model is then subjected to domain adaptive training to obtain the intermediate sleep staging model. The acquired labeled second ultrasound sleep data was used as the dataset for fine-tuning training. The intermediate sleep staging model was then fine-tuned to obtain the sleep staging model.

7. The non-contact sleep staging method based on ultrasonic sensing and presence state recognition according to claim 6, characterized in that, The sleep staging network includes a feature extraction module and a temporal modeling and classification module connected to the feature extraction module; wherein the feature extraction module adopts the ResNet18 architecture, and the temporal modeling and classification module adopts a multi-layer bidirectional long short-term memory network.

8. A non-contact sleep staging device based on ultrasonic sensing and presence state recognition, characterized in that, include: The demodulation module is used to acquire echo signals via ultrasound transceiver during sleep monitoring and demodulate them to obtain IQ signal sequences. The sequence processing module is used to perform presence state detection on the IQ signal sequence to obtain a stable bed rest time interval; The sleep staging module is used to obtain the corresponding respiratory waveform data of the stable bed rest interval based on the IQ signal sequence according to the stable bed rest interval; The stable respiratory waveform data in the bed zone is input into the sleep staging model obtained through phased training to obtain the corresponding sleep staging results; wherein, the phased training includes: pre-training, domain adaptive training and fine-tuning training.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 7.

Images