Sleep monitoring method and apparatus, electronic device, and storage medium

Abstract

This application provides a sleep monitoring method, device, electronic device, and storage medium. The method includes: acquiring first-type sleep data and second-type sleep data of a target subject; the first-type sleep data describes the target subject's physiological condition, and the second-type sleep data describes the target subject's body movement; both the first-type and second-type sleep data are obtained using a non-contact device; predicting the target subject's sleep stage using the first-type sleep data to obtain a first prediction result, and predicting the target subject's sleep stage using the second-type sleep data to obtain a second prediction result; obtaining the target subject's sleep monitoring result based on the first and second prediction results; and the sleep monitoring result indicates the target subject's current sleep stage. This application solves the problems of high cost, inconvenience, and inaccuracy in related technologies for sleep monitoring.

Description

Technical Field

[0001] This application relates to the field of sleep monitoring technology, and more specifically, to a sleep monitoring method, device, electronic device, and storage medium. Background Technology

[0002] Good sleep is an important factor in ensuring human health, and people are paying more and more attention to it. Generally, the sleep process includes stages such as wakefulness, rapid eye movement (REM) sleep, and non-rapid eye movement (NREM) sleep. A full night's sleep of 6-8 hours is roughly composed of 3-5 sleep cycles, and the entire sleep process is a continuous alternation of these different sleep stages. By monitoring the sleep stages people are in during sleep and conducting sleep analysis, we can understand their sleep status and quality, and thus further adjust and improve their sleep.

[0003] Currently, various sleep monitoring products have appeared on the market, such as wearable devices and polysomnography (PSG) devices. However, wearable devices are generally not accurate enough, and the contact nature of wearable devices can affect users' sleep and user experience. While monitoring sleep using EEG, ECG, EMG and other data acquisition devices is accurate, it is expensive, often requires specialized places such as hospitals, and requires professional operation, making it impossible to conduct daily sleep monitoring at home.

[0004] As can be seen from the above, sleep monitoring still suffers from high costs, inconvenience, and inaccuracy. Summary of the Invention

[0005] This application provides a sleep monitoring method, device, electronic device, and storage medium, which can solve the problems of high cost, inconvenience, and inaccuracy in related technologies. The technical solutions are as follows:

[0006] According to one aspect of this application, a sleep monitoring method is provided, the method comprising: acquiring first type of sleep data and second type of sleep data of a target object; the first type of sleep data being used to describe the physiological condition of the target object, and the second type of sleep data being used to describe the body movement of the target object; both the first type of sleep data and the second type of sleep data being obtained based on a non-contact device; using the first type of sleep data to predict the sleep stage of the target object to obtain a first prediction result, and using the second type of sleep data to predict the sleep stage of the target object to obtain a second prediction result; and obtaining a sleep monitoring result of the target object based on the first prediction result and the second prediction result; the sleep monitoring result being used to indicate the sleep stage of the target object.

[0007] According to one aspect of this application, a sleep monitoring device includes: a data acquisition module, configured to acquire first type of sleep data and second type of sleep data of a target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; both the first type of sleep data and the second type of sleep data are obtained based on a non-contact device; a prediction module, configured to predict the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and to predict the sleep stage of the target object using the second type of sleep data to obtain a second prediction result; and a result acquisition module, configured to obtain a sleep monitoring result of the target object based on the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

[0008] In an exemplary embodiment, the first prediction result is obtained by calling a trained first sleep stage model; the device further includes a first model training module for training the first sleep stage model; the first model training module includes: a first data acquisition unit for acquiring first training data carrying a first label; the first label is generated based on first indicator data collected from a training object using a first device, and is used to indicate the sleep stage of the training object; the first training data is a first type of sleep data of the training object; and a first training unit for inputting the first training data carrying the first label into a first neural network to train the first neural network and obtain the first sleep stage model.

[0009] In an exemplary embodiment, the prediction module includes: a first feature extraction unit, configured to extract features from the first type of sleep data to obtain a first feature; and a first probability calculation unit, configured to calculate the probability that the target object belongs to each sleep stage based on the first feature, and obtain the first prediction result based on the probability.

[0010] In one exemplary embodiment, the second prediction result is obtained by calling a trained second sleep stage model; the device further includes a second model training module for training the second sleep stage model; the second model training module includes: a second data acquisition unit for acquiring second training data carrying a second label; the second label is generated based on second indicator data collected from the training object using a second device, and is used to indicate the sleep stage of the training object; the second training data is a second type of sleep data of the training object; and a second training unit for inputting the second training data carrying the second label into a second neural network to train the second neural network and obtain the second sleep stage model.

[0011] In an exemplary embodiment, the prediction module includes: a second feature extraction unit, configured to extract features from the second type of sleep data to obtain a second feature; and a second probability calculation unit, configured to calculate the probability that the target object belongs to each sleep stage based on the second feature, and obtain the second prediction result based on the probability.

[0012] In an exemplary embodiment, the result acquisition module includes: a weight determination unit, configured to determine the weights corresponding to the first prediction result and the second prediction result respectively; a probability acquisition unit, configured to obtain the probability of the target object being in each sleep stage based on the probability of the target object being in each sleep stage indicated by the first prediction result and the second prediction result respectively, and the weights corresponding to the first prediction result and the second prediction result respectively; and a result acquisition unit, configured to obtain the sleep monitoring result based on the probability of the target object being in each sleep stage.

[0013] In an exemplary embodiment, the result acquisition module further includes: a first configuration unit, configured to configure the weight of the first prediction result to be less than the weight of the second prediction result for the first sleep stage; and / or a second configuration unit, configured to configure the weight of the first prediction result to be greater than the weight of the second prediction result for the second sleep stage.

[0014] In an exemplary embodiment, the result acquisition module includes: a difference detection unit, configured to perform difference detection on the first prediction result and the second prediction result to obtain a difference detection result, the difference detection result being used to indicate whether the difference between the first prediction result and the second prediction result is large; a deletion unit, configured to remove the sleep stages with large differences and their probabilities from the first prediction result if the difference detection result indicates that the difference between the first prediction result and the second prediction result is large; and a detection result acquisition unit, configured to obtain the sleep monitoring result based on the deleted first prediction result and the second prediction result.

[0015] In an exemplary embodiment, the data acquisition module includes: a signal receiving unit for receiving radar signals; the radar signals are obtained by detecting the target object; and a signal separation unit for separating the radar signals to obtain first type of sleep data and second type of sleep data; the first type of sleep data includes respiratory data and heart rate data; and the second type of sleep data includes body movement data.

[0016] According to one aspect of this application, a sleep monitoring method includes: receiving a sleep monitoring message; the sleep monitoring message is obtained by analyzing a first type of sleep data, a second type of sleep data, and / or sleep monitoring results of a target object, wherein the sleep monitoring results are obtained by predicting the sleep stage of the target object using the first type of sleep data and the second type of sleep data; and displaying the sleep monitoring message on a monitoring page.

[0017] According to one aspect of this application, a sleep monitoring device includes: a monitoring message receiving module for receiving sleep monitoring messages; the sleep monitoring messages are obtained by analyzing a first type of sleep data, a second type of sleep data, and / or sleep monitoring results of a target object, wherein the sleep monitoring results are obtained by predicting the sleep stage of the target object using the first type of sleep data and the second type of sleep data; and a monitoring message display module for displaying the sleep monitoring messages on a monitoring page.

[0018] According to one aspect of this application, an electronic device includes at least one processor and at least one memory, wherein program instructions or code are stored in the memory; the program instructions or code are loaded and executed by the processor, causing the electronic device to implement the sleep monitoring method as described above.

[0019] According to one aspect of this application, a storage medium has program instructions or code stored thereon, which are loaded and executed by a processor to implement a sleep monitoring method as described above.

[0020] According to one aspect of this application, an application product includes program instructions or code stored in a storage medium. The processor of an electronic device reads the program instructions or code from the storage medium, loads and executes the program instructions or code, causing the electronic device to implement the sleep monitoring method as described above.

[0021] The beneficial effects of the technical solution provided in this application are:

[0022] In the above technical solution, first-type sleep data and second-type sleep data of the target object are acquired. The first-type sleep data describes the target object's physiological condition, and the second-type sleep data describes its body movement. The sleep stage of the target object is then predicted using the first-type and second-type sleep data respectively, yielding a first prediction result and a second prediction result. Finally, the first and second prediction results are combined to obtain the sleep monitoring result of the target object, indicating its current sleep stage. This application divides sleep data into first-type and second-type sleep data, predicting the target object's sleep stage separately. On the one hand, this allows the prediction process to more fully utilize the characteristic information of each type of sleep data; on the other hand, the second prediction result assists the first prediction result to obtain a more accurate sleep monitoring result, thereby improving prediction accuracy. Both the first-type and second-type sleep data can be acquired using non-contact devices, eliminating the need for expensive and complex professional acquisition equipment. This is convenient, feasible, and low-cost, and avoids contact with the user, preventing disruption to sleep and providing a better user experience. This effectively solves the problems of high cost, inconvenience, and inaccuracy in related technologies for sleep monitoring. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0024] Figure 1 This is a schematic diagram based on the implementation environment involved in this application;

[0025] Figure 2 This is a flowchart illustrating a sleep monitoring method according to an exemplary embodiment;

[0026] Figure 3 yes Figure 2 In one embodiment, step 200 is shown in a flowchart.

[0027] Figure 4 yes Figure 2 A flowchart of step 220 in one embodiment corresponds to the following example;

[0028] Figure 5 This is a flowchart illustrating the training process of a first sleep staging model according to an exemplary embodiment;

[0029] Figure 6 This is a flowchart illustrating the training process of a second sleep staging model according to an exemplary embodiment;

[0030] Figure 7 yes Figure 2A flowchart of step 240 in one embodiment corresponds to the following example;

[0031] Figure 8 yes Figure 7 A flowchart from one embodiment preceding step 700 in the corresponding embodiment;

[0032] Figure 9 yes Figure 2 A flowchart of step 240 in another embodiment, corresponding to the embodiment;

[0033] Figure 10 This is a flowchart illustrating another sleep monitoring method according to an exemplary embodiment;

[0034] Figure 11A This is a schematic diagram illustrating the specific implementation of a sleep monitoring method in an application scenario;

[0035] Figure 11B , Figure 11C , Figure 11D This is a schematic diagram of the training data acquisition process;

[0036] Figure 11E , Figure 11F As shown, it illustrates the training. Figure 11A Schematic diagrams of neural networks 1 and 2;

[0037] Figure 12 This is a structural block diagram of a sleep monitoring device according to an exemplary embodiment;

[0038] Figure 13 This is a structural block diagram of another sleep monitoring device according to an exemplary embodiment;

[0039] Figure 14 This is a hardware structure diagram of a terminal according to an exemplary embodiment;

[0040] Figure 15 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0041] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0042] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0043] The following is an introduction and explanation of several terms used in this application:

[0044] Sleep staging refers to the process of dividing the human sleep process into segments and classifying each segment into different sleep stages. According to internationally accepted standards, sleep staging is usually divided into: wakefulness, non-rapid eye movement (NREM) sleep, and rapid eye movement (REM) sleep. NREM sleep can be further divided into four stages: NREM1-4. NREM1-2 can be considered light sleep, while NREM3-4 can be considered deep sleep.

[0045] PSG stands for Polysomnography, which involves using specialized PSG equipment to collect and record more than 10 indicators, such as electroencephalogram (EEG), electrocardiogram (ECG), and respiration, throughout the night's sleep. The instrument automatically analyzes the data, and then the data is manually corrected item by item to monitor sleep time, sleep efficiency, and sleep stages, thereby objectively evaluating the patient's sleep quality.

[0046] ATC stands for Actigraphy, which is a motion recorder.

[0047] TST stands for Total Sleep Time.

[0048] SE stands for Sleep Efficiency.

[0049] WASO stands for Wake After Sleep Onset, which is the wake-up time after falling asleep.

[0050] As mentioned earlier, due to people's increasing emphasis on sleep, various sleep monitoring products have emerged on the market, such as wearable devices and polysomnography (PSG) devices.

[0051] However, wearing wearable devices during sleep can affect user comfort and reduce the user experience. Furthermore, wearable devices used to determine sleep status are typically localized (e.g., wristbands, watches), worn on a specific part of the body, making it difficult to capture complete body movements and resulting in limited sleep data and inaccurate sleep status determination. In addition, wearable devices have short battery life, requiring frequent charging and maintenance, which is inconvenient.

[0052] Polysomnography (PSG) acquires data such as electroencephalogram (EEG), electromyography (EMG), electrocardiogram (ECG), electrooculogram (EOG), blood oxygen saturation, and pulse, and then analyzes the data to determine the user's sleep state. It can accurately monitor sleep, but it requires multiple electrodes and other devices to be attached to the user. It requires expensive equipment and a professional medical team to operate and analyze the data, which is costly, inconvenient, and difficult to promote and apply to daily sleep monitoring.

[0053] As can be seen from the above, the relevant technologies still have shortcomings such as high cost, inconvenience, and inaccuracy in sleep monitoring.

[0054] Therefore, the sleep monitoring method provided in this application can effectively improve the accuracy, convenience, and user experience of sleep monitoring. Accordingly, the sleep monitoring method is applicable to sleep monitoring devices, which can be deployed on electronic devices. These electronic devices can be non-contact devices with target detection capabilities (such as radar devices), or computer devices configured with a von Neumann architecture. For example, the computer devices can include desktop computers, laptops, servers, etc., or network devices with central control functions, such as gateways.

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0056] Please see Figure 1 , Figure 1 This is a schematic diagram of the implementation environment involved in a sleep monitoring method. The implementation environment includes at least a user terminal 110, a smart device 130, a server 170, and network equipment. Figure 1 In this context, network devices include gateway 150 and router 190, but this is not intended to be a specific limitation.

[0057] The user terminal 110, which can also be considered as a user terminal or terminal, can deploy (or install) the client associated with the smart device 130. This user terminal 110 can be an electronic device such as a smartphone, tablet, laptop, desktop computer, smart control panel, or other device with display and control functions, and is not limited here.

[0058] The client, associated with the smart device 130, is essentially where the user registers an account and configures the smart device 130. This client can be in the form of an application or a webpage. Correspondingly, the interface displayed on the client for the device can be in the form of a program window or a webpage, and there is no limitation here.

[0059] Smart device 130 is deployed in gateway 150 and communicates with gateway 150 through its own configured communication module, thereby being controlled by gateway 150. It should be understood that smart device 130 generally refers to one of multiple smart devices 130. This application embodiment only uses smart device 130 as an example; that is, this application embodiment does not limit the number or type of smart devices deployed in gateway 150. In one application scenario, smart device 130 is deployed in gateway 150 by accessing it through a local area network. The process of smart device 130 accessing gateway 150 through a local area network includes: gateway 150 first establishes a local area network, and smart device 130 joins the local area network established by gateway 150 by connecting to it. This local area network includes, but is not limited to, ZIGBEE or Bluetooth. Smart device 130 can be a contactless device (such as a radar device) used to collect sleep data from a target object.

[0060] The interaction between user terminal 110 and smart device 130 can be achieved through a local area network (LAN) or a wide area network (WAN). In one application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via router 190, such as Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same LAN, thus enabling user terminal 110 to interact with smart device 130 via the LAN path. In another application scenario, user terminal 110 establishes a wired or wireless communication connection with gateway 150 via server 170, such as 2G, 3G, 4G, 5G, or Wi-Fi, allowing user terminal 110 and gateway 150 to be deployed on the same WAN, thus enabling user terminal 110 to interact with smart device 130 via the WAN path.

[0061] Among them, server 170 can also be considered as cloud, cloud platform, platform side, server side, etc. This server 170 can be a single server, a server cluster composed of multiple servers, or a cloud computing center composed of multiple servers.

[0062] In one application scenario, the intelligent device 130 (radar device) detects a target object and obtains radar signals. After separating first-type sleep data and second-type sleep data from the radar signals, the first-type sleep data and second-type sleep data are transmitted to the server 170. The server 170 uses the first-type sleep data and second-type sleep data respectively to predict the sleep stage of the target object, obtaining a first prediction result and a second prediction result. Based on the first prediction result and the second prediction result, the sleep monitoring result of the target object is obtained. Furthermore, the server 170 can also send the sleep monitoring result to the user terminal 110 for display.

[0063] Of course, in other application scenarios, the sleep monitoring process completed by the server 170 can also be implemented by the smart device 130 (radar device). In this case, the acquisition of the first type of sleep data and the second type of sleep data, as well as the obtaining of sleep monitoring results using the first type of sleep data and the second type of sleep data, are all completed by the smart device 130 (radar device). The sleep monitoring results are then sent to the user terminal 110 for display via a wide area network path / local area network path. For example, if the first prediction result is obtained by calling the first sleep stage model, and the second prediction result is obtained by calling the second sleep stage model, the first sleep stage model and the second sleep stage model can be deployed in the smart device 130 (radar device). Then, the prediction of the sleep stage of the target object can be performed locally on the smart device 130 (radar device), thus realizing the sleep monitoring method.

[0064] Please see Figure 2 This application provides a sleep monitoring method, which is applicable to electronic devices, and the electronic devices can be... Figure 1 The server-side 170 or gateway 150 shown in the implementation environment can also be... Figure 1 The image shows a smart device 130 (radar device) in the implementation environment.

[0065] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0066] like Figure 2 As shown, the method may include the following steps:

[0067] Step 200: Obtain the first type of sleep data and the second type of sleep data of the target object.

[0068] The target object refers to the object that can be detected by non-contact devices (such as radar devices). For example, the target object can be a person. The radar device can detect a person to obtain radar signals, and then obtain the corresponding first type of sleep data and second type of sleep data based on the radar signals.

[0069] The first type of sleep data is used to describe the physiological condition of the target subject, and the second type of sleep data is used to describe the body movement of the target subject. Both types of sleep data are obtained at the same time or within the same time period based on non-contact devices.

[0070] Non-contact devices refer to devices that can sense or measure a target object without direct contact with it. For example, non-contact devices can be radar devices, but no specific limitation is made here.

[0071] In one possible implementation, the first type of sleep data includes breathing data and heart rate data, and the second type of sleep data includes body movement data, which is not intended to constitute a specific limitation.

[0072] In one possible implementation, such as Figure 3 As shown, step 200 may include the following steps:

[0073] Step 300: Receive radar signals.

[0074] Among them, radar signals are obtained by radar equipment detecting target objects.

[0075] In one possible implementation, the radar device could be a millimeter-wave radar, capable of sensitively capturing micro-motion signals (micro-vibrations / movements) generated by the target object, which would then be considered radar signals.

[0076] Step 320: Perform signal separation on the radar signal to obtain the first type of sleep data and the second type of sleep data.

[0077] The first type of sleep data includes breathing data and heart rate data, while the second type of sleep data includes body movement data.

[0078] The amplitude of local vibrations on the body surface caused by respiration is approximately 5 mm, while the amplitude of local vibrations caused by heartbeat is approximately 200–500 μm. In other words, human respiration, heartbeat, and limb movement (body motion) generate micro-motion signals. Radar equipment can detect these micro-motion signals, and by processing and analyzing these signals, data such as human respiration, heartbeat, and body motion can be obtained.

[0079] In one possible implementation, separation algorithms such as wavelet transform, empirical mode decomposition, and Fourier transform are used to separate respiratory data, heart rate data, and body movement data from radar signals.

[0080] In one possible implementation, respiratory data, heart rate data, and body movement data can be separated from radar signals based on differences in amplitude and frequency. For example, body movement has the largest amplitude and an irregular frequency, while respiratory and heart rate frequencies are regular, their ranges differ, and the body amplitudes caused by respiratory and heart rate are different.

[0081] Step 220: Use the first type of sleep data to predict the sleep stage of the target object and obtain a first prediction result; then use the second type of sleep data to predict the sleep stage of the target object and obtain a second prediction result.

[0082] The first prediction result is used to indicate the sleep stage of the target object predicted based on the first type of sleep data, and the second prediction result is used to indicate the sleep stage of the target object predicted based on the second type of sleep data.

[0083] In one possible implementation, sleep stages can include: wakefulness, non-rapid eye movement (NREM) sleep, and rapid eye movement (REM) sleep. It's important to understand that the division of sleep stages can be flexibly adjusted according to the actual needs of the application scenario. For example, sleep stages can also include REM sleep and wakefulness, as well as NREM-light sleep and NREM-deep sleep, derived from NREM sleep.

[0084] In one possible implementation, prediction refers to calculating the probability that a target object belongs to each sleep stage based on Type I / Type II sleep data.

[0085] Specifically, such as Figure 4 As shown, step 220 may include the following steps:

[0086] Step 400: Extract features from the first type of sleep data to obtain the first feature.

[0087] Feature extraction can be achieved through neural networks (such as CNN and RNN) or traditional signal processing (such as Hilbert transform and empirical mode decomposition).

[0088] In one possible implementation, the first feature is a precise numerical description of the first type of sleep data. It can be a parameter indicating the first type of sleep data. It is understood that different types of first-type sleep data will have different first features; that is, the first feature is used to uniquely identify the first type of sleep data. This first feature includes respiratory rate characteristics and / or heart rate characteristics.

[0089] Step 420: Calculate the probability of the target object belonging to each sleep stage based on the first feature, and obtain the first prediction result based on the probability.

[0090] In one possible implementation, the first feature is input into a first sleep stage model, and the classifier in the first sleep stage model is used to output the probability that the target object belongs to different sleep stages. Here, the first sleep stage model is a trained machine learning model capable of predicting the sleep stage of the target object; this machine learning model can be a neural network or the like.

[0091] The first prediction result can indicate the probability of the target object belonging to different sleep stages, and can also indicate the sleep stage with the highest probability for the target object. For example, the first prediction result indicates that the probability of the target object being in NREM-light sleep is 0.8, the probability of being in NREM-deep sleep is 0.7, the probability of being in REM sleep is 0.5, and the probability of being in WAKE sleep is 0.6; or, for example, the probability of NREM-light sleep is the highest, and the first prediction result indicates that the target object is in NREM-light sleep, without making a specific limitation here.

[0092] Step 440: Extract features from the second type of sleep data to obtain the second feature.

[0093] Feature extraction can be achieved through neural networks (such as CNN and RNN) or traditional signal processing (such as Hilbert transform and empirical mode decomposition).

[0094] In one possible implementation, the second feature is a precise numerical description of the second type of sleep data. It can be a parameter indicating the second type of sleep data. It is understood that different types of second-type sleep data will have different second features; that is, the second feature is used to uniquely identify the second type of sleep data. This second feature includes body movement characteristics.

[0095] Step 460: Calculate the probability that the target object belongs to each sleep stage based on the second feature, and obtain the second prediction result based on the probability.

[0096] In one possible implementation, the second feature is input into a second sleep stage model, and the classifier in this model is used to output the probability that the target object belongs to different sleep stages. The second sleep stage model is a trained machine learning model capable of predicting the sleep stage of the target object; this machine learning model can be a neural network or similar mechanism.

[0097] Similarly, the second prediction result can indicate the probability of the target object belonging to different sleep stages, and can also indicate the sleep stage with the highest probability for the target object.

[0098] Step 240: Based on the first prediction result and the second prediction result, obtain the sleep monitoring results of the target object.

[0099] Among them, sleep monitoring results are used to indicate the sleep stage of the target subject.

[0100] In one possible implementation, corresponding weights are assigned to the first prediction result and the second prediction result, so as to obtain the sleep monitoring result of the target object based on their respective weights, as well as the first prediction result and the second prediction result.

[0101] Through the above process, first-type sleep data and second-type sleep data of the target object are obtained. The first-type sleep data describes the target object's physiological condition, and the second-type sleep data describes its body movement. The first and second-type sleep data are then used to predict the target object's sleep stage, yielding a first prediction result and a second prediction result. Finally, the first and second prediction results are combined to obtain the target object's sleep monitoring result, which indicates the target object's sleep stage. This application divides sleep data into first-type and second-type sleep data, and predicts the target object's sleep stage separately. On the one hand, this allows the prediction process to make fuller use of the characteristic information of each type of sleep data; on the other hand, the second prediction result assists the first prediction result to obtain a more accurate sleep monitoring result, thereby improving prediction accuracy. Both the first-type and second-type sleep data can be acquired using non-contact devices, eliminating the need for expensive and complex professional acquisition equipment. This is convenient, feasible, and low-cost, and avoids contact with the user, preventing disruption to the user's sleep and providing a better user experience. This effectively solves the problems of high cost, inconvenience, and inaccuracy in related technologies for sleep monitoring.

[0102] In an exemplary embodiment, the first prediction result in step 220 is obtained by calling a trained first sleep stage model.

[0103] The first sleep staging model is obtained by training a neural network, such as... Figure 5As shown, the training process of the first sleep staging model may include the following steps:

[0104] Step 500: Obtain the first training data carrying the first label.

[0105] The first label is generated based on the first indicator data collected from the training subject using the first device, and is used to indicate the sleep stage of the training subject. The first training data is the first type of sleep data of the training subject.

[0106] It is worth noting that while the first device collects the first indicator data for the training object, the non-contact device collects the first training data for the training object. In other words, the training object simultaneously receives data collection from both the first device and the non-contact device.

[0107] The first device is a professional sleep monitoring device that can include multiple sensors to monitor and record data such as human respiration, heart rate, body movement, and body temperature in order to achieve sleep monitoring.

[0108] In one possible implementation, the first device refers to a polysomnography (PSG) device. The PSG device collects data from the training subject, including multiple indicators such as respiratory volume, apnea duration, snoring loudness, blood oxygen saturation, electromyography (EMG), electroencephalography (EEG), and electrocardiography (ECG). This data allows the determination of the training subject's sleep stage. In other words, the PSG device can acquire the training subject's sleep stage, and this acquired sleep stage serves as the primary label.

[0109] While using a PSG device to acquire the sleep stage (first label) of the training subject, the breathing data and heart rate data of the training subject are acquired through a non-contact device, and the breathing data and heart rate data (first training data) of the training subject are labeled with the first label in the time dimension, thereby constructing the first training data with the first label required for training the neural network.

[0110] For example, if the respiratory and heart rate data of the training subject are obtained from 11:00 to 11:15 using a non-contact device, and at the same time period, the training subject's sleep stage from 11:00 to 11:15 is identified as NERM1 using a PSG device, then the training subject's respiratory and heart rate data from 11:00 to 11:15 will be labeled with the first tag of NERM1.

[0111] In one possible implementation, in order to obtain a sufficient amount of initial training data, the above process can be performed multiple times for different training objects to obtain a sufficient amount of initial training data.

[0112] Step 520: Input the first training data carrying the first label into the first neural network to train the first neural network and obtain the first sleep stage model.

[0113] The first neural network can be LeNet, convolutional neural networks, long short-term memory, etc., without specific limitations here.

[0114] In one possible implementation, during the training of the first neural network, the first neural network outputs a first training result, which is the sleep stage predicted by the first neural network for the training object, i.e., the prediction result. The first label is the actual sleep stage of the training object, i.e., the correct result. A first loss value is used to evaluate the difference between the prediction result (first training result) and the correct result (first label), i.e., the error. Minimizing the first loss function used to calculate the first loss value means that the smaller the error between the prediction result and the correct result of the first neural network, the stronger the prediction ability of the first neural network, and the more accurately it can predict the sleep stage of the target object.

[0115] In one possible implementation, the first neural network is trained by minimizing a first loss function as the convergence condition. Of course, the convergence condition can be flexibly set according to the actual needs of the application scenario. For example, in one application scenario, the convergence condition is set to a specified number of iterations to improve the efficiency of model training.

[0116] In one possible implementation, when the loss function stops decreasing or decreases very slowly, we consider the neural network to have converged. At this point, we can stop training, save the trained first neural network, and obtain the first sleep stage model, so as to make subsequent predictions of the sleep stage of the target object.

[0117] In one possible implementation, the parameters of the neural network are updated based on the first loss value. Specifically, the first loss value is passed from the output layer to the input layer of the neural network through an error backpropagation algorithm, and the weights and biases of each neuron are adjusted according to the magnitude of the first loss value to continue training the neural network.

[0118] In one exemplary embodiment, the second prediction result in step 220 is obtained by calling a trained second sleep stage model.

[0119] The second sleep staging model is obtained by training a second neural network, such as... Figure 6 As shown, the training process of the second sleep staging model may include the following steps:

[0120] Step 600: Obtain the second training data carrying the second label.

[0121] The second label is generated based on the second indicator data collected from the training subject using the second device. It is used to indicate the sleep stage of the training subject. The second training data is the second type of sleep data of the training subject.

[0122] It is worth noting that while the second device is used to collect the second indicator data for the training subjects, the non-contact device is also collecting the second training data for the training subjects. In other words, the training subjects are simultaneously receiving data collection from both the second device and the non-contact device.

[0123] The second device is a professional sleep monitoring device, which can include multiple sensors to monitor and record data such as human respiration, heart rate, body movement, and body temperature in order to achieve sleep monitoring.

[0124] In one possible implementation, the second device is an ATC device, which refers to a motion recorder. Using an ATC device to collect data from the training subject, it can acquire motion data and sleep analysis parameters, such as sleep latency, total sleep time, sleep efficiency, and sleep-wake rhythm (ATC data, or second indicator data). This ATC data can be used to stage the training subject's sleep during the testing period to determine their current sleep stage. In other words, the ATC device can obtain the training subject's current sleep stage, and this sleep stage obtained through the ATC device can be used as a second label.

[0125] While using an ATC device to obtain the sleep stage (second label) of the training object, the body motion data (second training data) of the training object is obtained through a non-contact device, and the body motion data of the training object is labeled with a second label in the time dimension, thereby constructing the second training data required for training the neural network.

[0126] For example, if the body movement data of the training subject is obtained from 12:00 to 12:15 using a non-contact device, and at the same time period, the sleep stage of the training subject from 12:00 to 12:15 is identified as WAKE using an ATC device, then the body movement data of the training subject from 12:00 to 12:15 will be labeled with the first tag of WAKE.

[0127] In one possible implementation, in order to obtain a sufficient amount of second training data, the above process can be performed multiple times for different training objects to obtain a sufficient amount of second training data.

[0128] Step 620: Input the second training data carrying the second label into the second neural network to train the second neural network and obtain the second sleep stage model.

[0129] The second neural network can be LeNet, convolutional neural networks, long short-term memory, etc., without specific limitations here.

[0130] In one possible implementation, during the training of the second neural network, the second neural network outputs a second training result, which is used to indicate the sleep stage of the training object predicted by the second neural network. Based on the difference between the second training result and the second label, a second loss value is calculated, and it is determined whether the second loss value meets the convergence condition.

[0131] It should be understood that the second training result is the sleep stage of the training object predicted by the neural network, i.e., the prediction result; the second label is the actual sleep stage of the training object, i.e., the correct result; and the second loss value is used to evaluate the difference between the neural network's prediction result (second training result) and the correct result (second label), i.e., the error. Minimizing the second loss function used to calculate the second loss value means that the smaller the error between the neural network's prediction result and the correct result, the stronger the prediction ability of the second neural network, and the more accurately it can predict the sleep stage of the target object.

[0132] In one possible implementation, the second neural network is trained by minimizing the second loss function as the convergence condition. Of course, the convergence condition can be flexibly set according to the actual needs of the application scenario. For example, in one application scenario, the convergence condition is set to a specified number of iterations to improve the efficiency of model training.

[0133] In one possible implementation, when the second loss function stops decreasing or decreases very slowly, we consider the second neural network to have converged. At this point, we can stop training, save the trained second neural network, and obtain the second sleep stage model, so as to make predictions about the sleep stage of the target object later.

[0134] In one possible implementation, the parameters of the neural network are updated based on the second loss value. Specifically, the first loss value is passed from the output layer to the input layer of the neural network through the error backpropagation algorithm, and the weights and biases of each neuron are adjusted according to the magnitude of the first loss value to continue training the neural network.

[0135] Under the above embodiments, different training data are constructed based on PSG and ATC devices for repeated iterative training of the neural network. Through repeated iterative training, the predictive ability of the neural network is continuously improved, and finally a trained first sleep stage model and a trained second sleep stage model are obtained. The trained first sleep stage model reflects the mathematical mapping relationship between the first training data and the first label, and the trained second sleep stage model reflects the mathematical mapping relationship between the second training data and the second label. Thus, it is possible to accurately predict the sleep stage of the target object based on the first type of sleep data and the second type of sleep data of the target object.

[0136] Furthermore, training the first sleep stage model and the second sleep stage model separately based on different training data, instead of training a single sleep stage model with all training data, has several advantages. First, the first prediction result obtained by the first sleep stage model can assist the second prediction result obtained by the second sleep stage model, avoiding errors in the target sleep stage due to inaccurate first prediction results, thus effectively improving the accuracy of model prediction. Second, it can reduce the risk of overfitting in the neural network and improve its generalization ability. Third, training two models with small-scale training data is more conducive to improving model training efficiency than training a model with a large-scale training data, which in turn allows for more iterative training batches within the same training time, ultimately further improving the accuracy of model prediction.

[0137] Please see Figure 7 In one exemplary embodiment, step 240 may include the following steps:

[0138] Step 700: Determine the weights corresponding to the first prediction result and the second prediction result, respectively.

[0139] The first and second prediction results indicate the probability that the target object is in each sleep stage, respectively.

[0140] For example, the weights corresponding to the first and second prediction results are determined as follows: In the first prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1, 1, 1, and 1, respectively; In the second prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1.3, 0.9, 0.8, and 0.8, respectively.

[0141] In one possible implementation, the weight values ​​are related to the prediction accuracy of the first sleep stage model and the second sleep stage model.

[0142] Optionally, such as Figure 8 As shown, at least one of the following steps may be included before step 700:

[0143] Step 703: For the first sleep stage, the weight of the first prediction result is set to be less than the weight of the second prediction result.

[0144] Step 704: For the second sleep stage, the weight of the first prediction result is set to be greater than the weight of the second prediction result.

[0145] In one possible implementation, the first sleep stage refers to the WAKE stage, and the second sleep stage includes the REM stage, NREM (light sleep), and NREM (deep sleep).

[0146] As mentioned earlier, the first and second prediction results are obtained by calling the trained first and second sleep stage models, respectively. The training data for the first sleep stage model is the first training data carrying the first label, and the training data for the second sleep stage model is the second training data carrying the second label. The first label is generated based on the PSG device, and the second label is generated based on the ATC device. However, the inventors realized that although the sleep monitoring of the ATC device and the PSG device is accurate and reliable, and the results are well correlated, the ATC device tends to overestimate TST (Total Sleep Duration) and SE (Sleep Efficiency) and underestimate WASO (Wake-Up Time). This leads to the second sleep stage model predicting a higher probability of REM sleep, NREM-light sleep, and NREM-deep sleep, while predicting a lower probability of WAKE sleep. Based on this, when using the first sleep stage model and the second sleep stage model to obtain the probability of the target object being in each sleep stage, the weight of the first prediction result is less than the weight of the second prediction result for the WAKE stage; for the REM stage, NREM-light sleep stage, and NREM-deep sleep stage, the weight of the first prediction result is greater than the weight of the second prediction result. For example, in the first prediction result, the weights of the WAKE stage, REM stage, NREM-light sleep stage, and NREM-deep sleep stage are 1, 1, 1, and 1, respectively; in the second prediction result, the weights of the WAKE stage, REM stage, NREM-light sleep stage, and NREM-deep sleep stage are 1.3, 0.9, 0.8, and 0.8, respectively.

[0147] By configuring weights, the first and second prediction results can be corrected, increasing the accuracy of sleep prediction results.

[0148] Step 720: Based on the probability of the target object being in each sleep stage indicated by the first prediction result and the second prediction result respectively, and the weights corresponding to the first prediction result and the second prediction result respectively, the probability of the target object being in each sleep stage is obtained.

[0149] For example, the first prediction indicates a probability of 0.8 for being in NREM-light sleep, 0.7 for NREM-deep sleep, 0.5 for REM sleep, and 0.6 for WAKE sleep; the second prediction indicates a probability of 0.9 for NREM-light sleep, 0.8 for NREM-deep sleep, 0.5 for REM sleep, and 0.5 for WAKE sleep. In the first prediction, the weights of WAKE, REM, NREM-light sleep, and NREM-deep sleep are 1, 1, 1, and 1, respectively; in the second prediction, the weights of WAKE, REM, NREM-light sleep, and NREM-deep sleep are 1.3, 0.9, 0.8, and 0.8, respectively. The probability of the WAKE phase is 0.6×1+0.5×1.3=1.25, the probability of the NREM-light sleep phase is 0.8×1+0.9×0.8=1.52, the probability of the NREM-deep sleep phase is 0.7×1+0.8×0.8=1.34, and the probability of the REM phase is 0.5×1+0.5×0.9=0.95.

[0150] Step 740: Obtain sleep monitoring results based on the probability of the target subject being in each sleep stage.

[0151] In one possible implementation, the sleep stage with the highest probability is selected as the sleep stage in which the target object is located. For example, if the probability of NREM-light sleep stage is the highest, then the actual sleep stage in which the target object is located is considered to be NREM-light sleep stage.

[0152] By determining the weights corresponding to the first and second prediction results through the above process, and obtaining sleep monitoring results using these weights, more accurate sleep monitoring results can be obtained. When the probabilities of each sleep stage in the first and second prediction results are not accurately predicted, assigning weights to the first and second prediction results can correct the inaccuracies and further improve the accuracy of sleep monitoring.

[0153] Please see Figure 9 In one exemplary embodiment, step 240 may include the following steps:

[0154] Step 241: Perform difference detection on the first prediction result and the second prediction result to obtain the difference detection result.

[0155] The difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large.

[0156] For example, the first prediction result indicates the highest probability of deep sleep, but the second prediction result indicates the lowest probability of deep sleep. In this case, the difference detection result indicates that there is a large difference between the first and second prediction results, and the sleep stage with a large difference is deep sleep.

[0157] Step 242: If the difference detection result indicates a large difference between the first prediction result and the second prediction result, then the sleep stages with large differences and their probabilities are deleted from the first prediction result.

[0158] Continuing with the example above, sleep stages with significant differences (deep sleep) are removed from the first prediction result. For instance, in sleep monitoring during a certain period, the highest probability predicted by the first prediction result is NREM-deep sleep. The corresponding body movement data in the second prediction result indicates large or high-frequency body movement during this time. However, considering that large or high-frequency body movement is unlikely when the body is in NREM-deep sleep, the highest probability NREM-deep sleep stage is removed, and other sleep stages with relatively high probabilities are considered in turn. If no abnormal body movement is observed during this period, the final analysis result for the sleep stage is obtained based on probability calculation and ranking.

[0159] Step 243: Based on the deleted first and second prediction results, obtain the sleep monitoring results.

[0160] Using the previous example, after removing sleep stages and their probabilities that differ significantly, the sleep result depends on the second most probable sleep stage in the first prediction and the most probable sleep stage in the second prediction.

[0161] Under the above embodiments, by combining the first prediction result and the second prediction result, the second prediction result assists the first prediction result to obtain a more accurate sleep monitoring result, thereby improving the accuracy of the prediction.

[0162] Please see Figure 10 This application provides another sleep monitoring method, which is applicable to electronic devices, such as... Figure 1 The user terminal 110 in the implementation environment is shown.

[0163] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0164] like Figure 10 As shown, the method may include the following steps:

[0165] Step 800: Receive sleep monitoring messages.

[0166] Sleep monitoring messages are obtained by analyzing the target subject's Type I sleep data, Type II sleep data, and / or sleep monitoring results. The sleep monitoring results are used to predict the target subject's sleep stage using Type I sleep data and Type II sleep data.

[0167] It should be understood that the process of obtaining sleep monitoring messages by analyzing the first type of sleep data, the second type of sleep data, and / or sleep monitoring results of the target object can be completed by non-contact devices (such as radar devices) or by other electronic devices. For example, after the non-contact device collects the first type of sleep data and the second type of sleep data, the first type of sleep data and the second type of sleep data are transmitted to the server, and the server executes the sleep monitoring method provided in this application to obtain sleep monitoring results. Then, the sleep monitoring messages are obtained by analyzing the first type of sleep data, the second type of sleep data, and / or sleep monitoring results of the target object.

[0168] Sleep monitoring messages can include sleep reports, abnormal warnings, etc., without specific limitations. For example, sleep reports can include: deep sleep duration, light sleep duration, respiratory rate during sleep, heart rate during sleep, number and amplitude of body movements during sleep, number of awakenings during sleep and duration of each awakening, etc. Sleep reports can be for one night's sleep or long-term sleep reports. Abnormal warnings can include abnormal notifications generated in response to abnormal heart rate and / or respiratory values ​​(such as: tachypnea, apnea, bradycardia, tachycardia).

[0169] Step 820: Display the sleep monitoring message on the monitoring page.

[0170] The testing page refers to the display interface of the user terminal client. The client can be in the form of an application or a webpage. Correspondingly, the client's display interface can be in the form of a program window or a webpage. There is no limitation here.

[0171] During the process described above, after the sleep monitoring message is generated, it will be transmitted to the user terminal for display, so that the user can understand the quality of sleep and sleep breathing problems, etc., which can help the user better understand their sleep situation and effectively solve sleep problems.

[0172] Figure 11A This is a schematic diagram illustrating the specific implementation of a sleep monitoring method in an application scenario.

[0173] In this application scenario, the real-time collected radar signals are first subjected to feature extraction. The radar signals are separated and extracted to obtain respiratory rate features, heart rate features, and body movement features. Then, the separated heart rate and respiratory rate are input into a trained neural network 1 to obtain a preliminary classification result 1. The classification result 1 includes the probability of each corresponding category label (sleep stage). For example, the probability of a human being in NREM-light sleep stage is 0.8, the probability of a human being in NREM-deep sleep stage is 0.7, the probability of a human being in REM sleep stage is 0.5, and the probability of a human being in WAKE sleep stage is 0.6.

[0174] The body motion signal is then separated from the radar signal and input into the trained neural network 2 to obtain a preliminary classification result 2. The classification result 2 includes the probability of each label (sleep stage) corresponding to the input. For example, the probability of the human body being in NREM-light sleep is 0.9, the probability of the human body being in NREM-deep sleep is 0.8, the probability of the human body being in REM sleep is 0.5, and the probability of the human body being in WAKE sleep is 0.5.

[0175] Finally, classification results 1 and 2 are weighted and calculated to obtain the final sleep stage classification result. The final classification result includes the probability of each label (sleep stage) corresponding to the radar signal. For example, the probability of a person being in NREM (light sleep) is 1.52, the probability of a person being in NREM (deep sleep) is 1.34, the probability of a person processing REM sleep is 0.95, and the probability of a person being in WAKE sleep is 1.25. Based on the order of probability, the classification label (sleep stage) with the highest probability is given priority as the output sleep stage result.

[0176] The trained neural networks 1 and 2 were obtained based on training data, such as... Figure 11B , Figure 11C , Figure 11D As shown, each diagram illustrates the process of acquiring training data.

[0177] exist Figure 11B , Figure 11C , Figure 11D In this test, the test subject was simultaneously monitored by PSG equipment, ATC equipment, and had data collected by millimeter-wave radar in the same test.

[0178] PSG can obtain indicators such as respiratory volume, apnea duration, snoring loudness, blood oxygen saturation, electromyography, electroencephalography, and electrocardiography. These indicators can be used to stage the sleep of the subject during the test, that is, to label the subject in the time dimension (e.g., 11:00-11:15 WEAK, 12:00-12:15 NERM1).

[0179] ATC can obtain the subject's body movement data and sleep analysis parameters, such as sleep latency, total sleep time, sleep efficiency, sleep-wake rhythm, etc. These indicators can be used to stage the subject's sleep during the test, that is, to label the subject in the time dimension (e.g., 11:00-11:15 WEAK, 12:00-12:15 SLEEP).

[0180] Millimeter-wave radar collects radar signals from the subject and further separates them to obtain respiratory rate, heart rate, and body movement. There are many algorithms to choose from for separating respiratory rate, heart rate, and body movement (amplitude and frequency of human body vibration) from radar signals (e.g., wavelet transform, empirical mode decomposition, Fourier transform, etc.).

[0181] After preparing the training data (respiratory rate, heart rate, body movement, label 1, label 2), neural networks 1 and 2 can be trained, as follows: Figure 11E , Figure 11F As shown, it illustrates a schematic diagram of training neural networks 1 and 2.

[0182] You can choose convolutional neural networks, support vector machines (SVM), LeNet, long short-term memory (LSTM), etc. as neural network 1 and 2, without making specific restrictions here.

[0183] Using the prepared respiratory rate, heart rate, and label 1 as input, neural network 1 is trained. The trained neural network 1 establishes a mapping relationship between respiratory rate, heart rate, and label 1.

[0184] Using the prepared body movement data and label 2 as input, neural network 2 is trained. The trained neural network 2 establishes a mapping relationship between body movement and label 2.

[0185] Studies have shown that ATC and PSG sleep tests are accurate and reliable, and the results are well correlated. However, ATC tends to overestimate TST and SE and underestimate WASO time. This also leads to the neural network 2 trained based on ATC overestimating TST and SE and underestimating WASO time. To ensure the accuracy of the final sleep staging results, when using classification result 1 and classification result 2 to obtain the final sleep staging results, the weight of classification result 1 for WAKE stage is less than the weight of classification result 2. For REM stage, NREM-light sleep stage, and NREM-deep sleep stage, the weight of classification result 1 is greater than the weight of classification result 2.

[0186] It should be noted that the weight values ​​are related to the accuracy of the training results of the selected neural networks 1 and 2. ATC and PSG are only one possible factor affecting the classification accuracy of neural networks 1 and 2. Therefore, the above weight allocation is just an example. In practice, all factors affecting the accuracy of the training results of neural networks 1 and 2 should be considered comprehensively.

[0187] In this application scenario, the PSG and ATC devices are only used to build training data to train the neural network. After the neural network is trained, it is only necessary to separate the respiratory rate, heart rate, and body movement from the radar signal to perform sleep monitoring. In other words, sleep monitoring can be achieved based on non-contact devices (radar devices), which has little interference with the user's normal sleep, is friendly to burn and scald patients and children, and does not require the complicated operation of professional devices such as PSG and ATC. It is simple and easy to operate, can be carried out at home, and has low cost.

[0188] The following are embodiments of the apparatus described in this application, which can be used to execute the sleep monitoring method involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the sleep monitoring method involved in this application.

[0189] Please see Figure 12 This application provides a sleep monitoring device 900, including but not limited to: a data acquisition module 910, a prediction module 930, and a result acquisition module 950.

[0190] The data acquisition module 910 is used to acquire a first type of sleep data and a second type of sleep data of the target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; both the first type of sleep data and the second type of sleep data are obtained based on non-contact devices.

[0191] The prediction module 930 is used to predict the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and to predict the sleep stage of the target object using the second type of sleep data to obtain a second prediction result.

[0192] The result acquisition module 950 is used to obtain the sleep monitoring results of the target object based on the first prediction result and the second prediction result; the sleep monitoring results are used to indicate the sleep stage of the target object.

[0193] In an exemplary embodiment, the first prediction result is obtained by calling a trained first sleep stage model; the device 900 further includes a first model training module 960 for training the first sleep stage model; the first model training module 960 includes: a first data acquisition unit 961 for acquiring first training data carrying a first label; the first label is generated based on first indicator data collected from a training object using a first device, and is used to indicate the sleep stage of the training object; the first training data is a first type of sleep data of the training object; a first training unit 962 for inputting the first training data carrying the first label into a first neural network to train the first neural network and obtain the first sleep stage model.

[0194] In an exemplary embodiment, the prediction module 930 includes: a first feature extraction unit 931, configured to extract features from the first type of sleep data to obtain a first feature; and a first probability calculation unit 932, configured to calculate the probability that the target object belongs to each sleep stage based on the first feature, and obtain the first prediction result based on the probability.

[0195] In an exemplary embodiment, the second prediction result is obtained by calling a trained second sleep stage model; the device 900 further includes a second model training module 970 for training the second sleep stage model; the second model training module 970 includes: a second data acquisition unit 971 for acquiring second training data carrying a second label; the second label is generated based on second indicator data collected from the training object using a second device, and is used to indicate the sleep stage of the training object; the second training data is a second type of sleep data of the training object; a second training unit 972 for inputting the second training data carrying the second label into a second neural network to train the second neural network and obtain the second sleep stage model.

[0196] In an exemplary embodiment, the prediction module 930 includes: a second feature extraction unit 933, used to extract features from the second type of sleep data to obtain a second feature; and a second probability calculation unit 934, used to calculate the probability that the target object belongs to each sleep stage based on the second feature, and to obtain the second prediction result based on the probability.

[0197] In an exemplary embodiment, the result acquisition module 950 includes: a weight determination unit 951, configured to determine the weights corresponding to the first prediction result and the second prediction result respectively; a probability acquisition unit 952, configured to obtain the probability of the target object being in each sleep stage based on the probability of the target object being in each sleep stage indicated by the first prediction result and the second prediction result respectively, and the weights corresponding to the first prediction result and the second prediction result respectively; and a result acquisition unit 953, configured to obtain the sleep monitoring result based on the probability of the target object being in each sleep stage.

[0198] In an exemplary embodiment, the result acquisition module 950 includes: a difference detection unit 954, configured to perform difference detection on the first prediction result and the second prediction result to obtain a difference detection result, the difference detection result being used to indicate whether the difference between the first prediction result and the second prediction result is large; a deletion unit 955, configured to delete the sleep stages with large differences and their probabilities from the first prediction result if the difference detection result indicates that the difference between the first prediction result and the second prediction result is large; and a detection result acquisition unit 956, configured to obtain the sleep monitoring result based on the deleted first prediction result and the second prediction result.

[0199] In an exemplary embodiment, the result acquisition module 950 further includes: a first configuration unit 957, configured to configure the weight of the first prediction result to be less than the weight of the second prediction result for the first sleep stage; or a second configuration unit 958, configured to configure the weight of the first prediction result to be greater than the weight of the second prediction result for the second sleep stage.

[0200] In an exemplary embodiment, the data acquisition module 910 includes: a signal receiving unit 911 for receiving radar signals; the radar signals are obtained by detecting the target object; and a signal separation unit 912 for separating the radar signals to obtain first type of sleep data and second type of sleep data; the first type of sleep data includes breathing data and heart rate data; and the second type of sleep data includes body movement data.

[0201] Please see Figure 13 This application provides another sleep monitoring device 1000, including but not limited to: a monitoring message receiving module 1010 and a monitoring message display module 1030.

[0202] The monitoring message receiving module 1010 is used to receive sleep monitoring messages.

[0203] The monitoring message display module 1030 is used to display sleep monitoring messages on the monitoring page.

[0204] It should be noted that the sleep monitoring device and sleep monitoring message display device provided in the above embodiments are only illustrated by the division of the above functional modules when performing sleep monitoring and sleep monitoring message display. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the sleep monitoring device and sleep monitoring message display device will be divided into different functional modules to complete all or part of the functions described above.

[0205] Furthermore, the sleep monitoring device, sleep monitoring message display device, sleep monitoring method, and sleep monitoring message display method provided in the above embodiments belong to the same concept. The specific way in which each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0206] Please see Figure 14 , Figure 14 This is a schematic diagram illustrating the structure of a terminal according to an exemplary embodiment. The terminal is suitable for… Figure 1 The user terminal 110 in the implementation environment is shown.

[0207] It should be noted that this terminal is merely an example adapted to this application and should not be considered as any limitation on the scope of use of this application. Furthermore, this terminal should not be construed as requiring or depending on any specific feature. Figure 14 One or more components of the exemplary terminal 1100 shown.

[0208] like Figure 14 As shown, terminal 1100 includes memory 101, memory controller 103, and one or more ( Figure 14 (Only one is shown) Processor 105, peripheral interface 107, radio frequency module 109, positioning module 111, camera module 113, audio module 115, touch screen 117, and button module 119. These components communicate with each other through one or more communication buses / signal lines 121.

[0209] The memory 101 can be used to store program instructions or code, such as the program instructions or code corresponding to the sleep monitoring method, sleep monitoring message display method and device in the exemplary embodiments of this application. The processor 105 executes various functions and data processing by running the program instructions or code stored in the memory 101, that is, to complete the sleep monitoring method and the sleep monitoring message display method.

[0210] The memory 101, as a carrier for resource storage, can be random access memory, such as high-speed random access memory, non-volatile memory, such as one or more magnetic storage devices, flash memory, or other solid-state memory. The storage method can be temporary storage or permanent storage.

[0211] The peripheral interface 107 may include at least one wired or wireless network interface, at least one serial-to-parallel conversion interface, at least one input / output interface, and at least one USB interface, etc., for coupling various external input / output devices to the memory 101 and the processor 105 to realize communication with various external input / output devices.

[0212] The radio frequency module 109 is used to transmit and receive electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, thereby enabling communication with other devices through a communication network. The communication network includes cellular telephone networks, wireless local area networks, or metropolitan area networks, and these communication networks can use various communication standards, protocols, and technologies.

[0213] The positioning module 111 is used to obtain the current geographical location of the terminal 1100. Examples of positioning modules 111 include, but are not limited to, Global Positioning System (GPS), positioning technologies based on wireless local area networks or mobile communication networks.

[0214] The camera module 113 is part of the camera and is used to capture pictures or videos. The captured pictures or videos can be stored in the memory 101 or transmitted to the host computer via the radio frequency module 109.

[0215] The audio module 115 provides an audio interface to the user, which may include one or more microphone jacks, one or more speaker jacks, and one or more headphone jacks. Audio data is exchanged with other devices through the audio interface. Audio data can be stored in the memory 101 and can also be transmitted via the radio frequency module 109.

[0216] The touchscreen 117 provides an input / output interface between the terminal 1100 and the user. Specifically, the user can perform input operations through the touchscreen 117, such as clicking, touching, and swiping gestures, so that the terminal 1100 can respond to the input operations. The terminal 1100 then displays the output content, which can be text, images, or videos in any form or combination thereof, to the user through the touchscreen 117.

[0217] The button module 119 includes at least one button, providing an interface for users to input information into the terminal 1100. Users can press different buttons to enable the terminal 1100 to perform different functions. For example, the volume adjustment button allows users to adjust the volume of the sound played by the terminal 1100.

[0218] Understandable. Figure 14 The structure shown is for illustrative purposes only; the terminal 1100 may also include components that are more advanced than those shown. Figure 14 The more or fewer components shown, or having the same Figure 14 The different components are shown. Figure 14The components shown can be implemented using hardware, software, or a combination thereof.

[0219] Please see Figure 15 This application provides an electronic device 4000, which may include: desktop computer, laptop computer, server, smartphone, etc.

[0220] exist Figure 15 In this context, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0221] The data interaction between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. This communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 15 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0222] Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0223] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0224] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program instructions or code in the form of instructions or data structures and accessible by the electronic device 4000, but not limited thereto.

[0225] The memory 4003 stores program instructions or code, and the processor 4001 can read the program instructions or code stored in the memory 4003 through the communication bus 4002.

[0226] When the program instructions or code are executed by the processor 4001, the sleep monitoring method and the sleep monitoring message display method in the above embodiments are implemented.

[0227] Furthermore, this application embodiment provides a storage medium storing program instructions or code, which is loaded and executed by a processor to implement the sleep monitoring method and sleep monitoring message display method as described above.

[0228] This application provides an application product, which includes program instructions or code stored in a storage medium. The processor of the electronic device reads the program instructions or code from the storage medium, loads and executes the program instructions or code, so that the electronic device implements the sleep monitoring method and the sleep monitoring message display method as described above.

[0229] Compared with related technologies, this application has the following advantages:

[0230] 1. Acquire first-type and second-type sleep data of the target subject, wherein the first-type sleep data describes the target subject's physiological condition, and the second-type sleep data describes the target subject's body movement; then, use the first-type and second-type sleep data respectively to predict the target subject's sleep stage, obtaining a first prediction result and a second prediction result; finally, combine the first and second prediction results to obtain the target subject's sleep monitoring result. This application divides sleep data into first-type and second-type sleep data and performs predictions separately, which can more fully utilize the characteristic information of each sleep data, thereby improving prediction accuracy.

[0231] Both the first and second types of sleep data can be acquired using non-contact devices (such as radar), eliminating the need for expensive and complex professional equipment. This approach is convenient, feasible, and cost-effective, meeting the needs of daily sleep monitoring in households. Furthermore, it requires no contact with the user, minimizing disruption to their normal sleep. It is also user-friendly for burn and scald patients and children, providing a better user experience.

[0232] In summary, this application effectively solves the problems of high cost, inconvenience, and inaccuracy in related technologies for sleep monitoring.

[0233] 2. Training data is constructed based on PSG and ATC devices. The resulting labeled training data includes the input and its corresponding ground truth label. During training, the neural network uses the input as the data for its input layer. After computation and processing by the neural network, the training result is obtained. This training result is compared with the ground truth label to calculate the loss value. The parameters of the neural network are then adjusted based on the loss value, enabling the neural network to predict the output more accurately. Through repeated iterative training, the predictive ability of the neural network continuously improves, ultimately resulting in a well-trained sleep staging model. The first well-trained sleep staging model includes the mapping relationship between the first training data and the first label, and the second well-trained sleep staging model includes the mapping relationship between the second training data and the second label. Training the first and second sleep staging models separately can improve the prediction accuracy of the neural network, while also reducing the risk of overfitting and improving the generalization ability of the neural network.

[0234] 3. Determine the weights corresponding to the first and second prediction results respectively. Obtaining sleep monitoring results using weights can lead to more accurate sleep monitoring results. When the probabilities of each sleep stage in the first and second prediction results are not accurately predicted, assigning weights to the first and second prediction results can correct the inaccuracies and further improve the accuracy of sleep monitoring.

[0235] 4. After generating sleep monitoring messages, they will be transmitted to the user's terminal for display, making it easier for the user to understand sleep quality and sleep breathing problems, etc., thereby helping the user to better understand their sleep situation and effectively solve sleep problems.

[0236] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0237] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A sleep monitoring method, characterized in that, The method includes: Acquire first-type sleep data and second-type sleep data of the target object; the first-type sleep data is used to describe the physiological condition of the target object, and the second-type sleep data is used to describe the body movement of the target object; both the first-type sleep data and the second-type sleep data are obtained based on non-contact devices; The sleep stage of the target object is predicted using the first type of sleep data to obtain a first prediction result, and the sleep stage of the target object is predicted using the second type of sleep data to obtain a second prediction result; the first prediction result and the second prediction result are used to indicate the multiple sleep stages of the target object and the probability of each sleep stage, respectively. By using the second prediction result to assist the first prediction result, abnormal sleep stages and their probabilities in the first prediction result are deleted; the step of using the second prediction result to assist the first prediction result to delete abnormal sleep stages and their probabilities in the first prediction result includes: if the sleep stage with the highest probability in the first prediction result is deep sleep, and the corresponding body movement information in the second prediction result indicates that the target object has large or high-frequency body movements during the deep sleep period, then the sleep stage with the highest probability in the first prediction result and its probability are deleted. Based on the deleted first and second prediction results and their respective weights, the sleep monitoring results of the target object are obtained; the sleep monitoring results are used to indicate the sleep stage of the target object; the sleep stage includes at least a first sleep stage and a second sleep stage. The second prediction result is obtained by calling the second sleep stage model, which is trained based on the second training data carrying the second label, and the second label is generated based on the second device. When the second device includes a body movement recorder, the weight of the first prediction result is configured to be less than the weight of the second prediction result for the first sleep stage, and the weight of the first prediction result is configured to be greater than the weight of the second prediction result for the second sleep stage.

2. The method as described in claim 1, characterized in that, The first prediction result is obtained by calling the trained first sleep stage model; The training process of the first sleep staging model includes: Acquire first training data carrying a first label; the first label is generated based on first indicator data collected from the training object using a first device, and is used to indicate the sleep stage of the training object; the first training data is the first type of sleep data of the training object. The first training data carrying the first label is input into the first neural network to train the first neural network and obtain the first sleep staging model.

3. The method as described in claim 1, characterized in that, The step of predicting the sleep stage of the target object using the first type of sleep data to obtain a first prediction result includes: Feature extraction is performed on the first type of sleep data to obtain the first feature; The probability of the target object belonging to each sleep stage is calculated based on the first feature, and the first prediction result is obtained based on the probability.

4. The method as described in claim 1, characterized in that, The training process of the second sleep staging model includes: Acquire the second training data carrying the second label; the second label is generated based on the second indicator data collected from the training object using the second device, and is used to indicate the sleep stage of the training object; the second training data is the second type of sleep data of the training object; The second training data carrying the second label is input into the second neural network to train the second neural network and obtain the second sleep staging model.

5. The method as described in claim 1, characterized in that, The step of using the second type of sleep data to predict the sleep stage of the target object and obtaining a second prediction result includes: Feature extraction is performed on the second type of sleep data to obtain the second feature; The probability of the target object belonging to each sleep stage is calculated based on the second feature, and the second prediction result is obtained based on the probability.

6. The method as described in claim 1, characterized in that, The step of obtaining the sleep monitoring results of the target object based on the deleted first and second prediction results and the weights corresponding to the first and second prediction results respectively includes: Determine the weights corresponding to the first prediction result and the second prediction result, respectively; Based on the probability of the target object being in each sleep stage indicated by the first prediction result and the second prediction result, respectively, and the weights corresponding to the first prediction result and the second prediction result, the probability of the target object being in each sleep stage is obtained. The sleep monitoring results are obtained based on the probability of the target object being in each sleep stage.

7. The method as described in claim 1, characterized in that, The step of using the second prediction result to assist the first prediction result in removing abnormal sleep stages and their probabilities from the first prediction result further includes: A difference detection is performed between the first prediction result and the second prediction result to obtain a difference detection result; the difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large. If the difference detection result indicates a large difference between the first prediction result and the second prediction result, then the sleep stages with large differences and their probabilities will be removed from the first prediction result.

8. The method according to any one of claims 1 to 7, characterized in that, The acquisition of the first type of sleep data and the second type of sleep data of the target object includes: Receive radar signals; the radar signals are obtained by detecting the target object; The radar signal is separated to obtain the first type of sleep data and the second type of sleep data; The first type of sleep data includes breathing data and heart rate data; The second type of sleep data includes body movement data.

9. A sleep monitoring method, characterized in that, The method includes: Receive sleep monitoring messages; the sleep monitoring messages are obtained by analyzing the first type of sleep data, the second type of sleep data, and / or sleep monitoring results of the target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; both the first type of sleep data and the second type of sleep data are obtained based on non-contact devices; the sleep monitoring results are obtained by using the first type of sleep data and the second type of sleep data to predict the sleep stage of the target object, respectively, and the weights corresponding to the first prediction results and the second prediction results; the second prediction result obtained from the second type of sleep data is used to assist the first prediction result obtained from the first type of sleep data, so as to delete abnormal sleep stages and their probabilities in the first prediction result by assisting the first prediction result with the second prediction result; the first prediction result and the second prediction result are respectively used to indicate the sleep stage of the target object. The prediction includes multiple sleep stages and the probability of each sleep stage; wherein the sleep stages include at least a first sleep stage and a second sleep stage; wherein the second prediction result is obtained by calling a second sleep staging model, the second sleep staging model is trained based on second training data carrying a second label, and the second label is generated based on a second device; when the second device includes a body movement recorder, the weight of the first prediction result is configured to be less than the weight of the second prediction result for the first sleep stage, and the weight of the first prediction result is configured to be greater than the weight of the second prediction result for the second sleep stage; the step of deleting abnormal sleep stages and their probabilities in the first prediction result by assisting the first prediction result with the second prediction result includes: if the sleep stage with the highest probability in the first prediction result is the deep sleep stage, and the corresponding body movement situation in the second prediction result indicates that the target object has large or high-frequency body movements during the deep sleep stage, then the sleep stage with the highest probability in the first prediction result and its probability are deleted. The sleep monitoring messages will be displayed on the monitoring page.

10. A sleep monitoring device, characterized in that, The device includes: The data acquisition module is used to acquire first-type sleep data and second-type sleep data of the target object; the first-type sleep data is used to describe the physiological condition of the target object, and the second-type sleep data is used to describe the body movement of the target object; both the first-type sleep data and the second-type sleep data are obtained based on non-contact devices; The prediction module is used to predict the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and to predict the sleep stage of the target object using the second type of sleep data to obtain a second prediction result; the first prediction result and the second prediction result are respectively used to indicate the multiple sleep stages of the target object and the probability of each sleep stage; The result acquisition module is used to assist the first prediction result with the second prediction result, delete abnormal sleep stages and their probabilities from the first prediction result, and obtain the sleep monitoring result of the target object based on the deleted first prediction result and the second prediction result, and the weights corresponding to the first prediction result and the second prediction result respectively; the sleep monitoring result is used to indicate the sleep stage of the target object; wherein, the sleep stage includes at least a first sleep stage and a second sleep stage; wherein, the second prediction result is obtained by calling a second sleep staging model, the second sleep staging model is trained based on second training data carrying a second label, and the second label is based on the second sleep staging model. The prediction is generated by two devices; when the second device includes a body movement recorder, the weight of the first prediction result is less than the weight of the second prediction result for the first sleep stage, and the weight of the first prediction result is greater than the weight of the second prediction result for the second sleep stage; the step of deleting abnormal sleep stages and their probabilities from the first prediction result by assisting the first prediction result with the second prediction result includes: if the sleep stage with the highest probability in the first prediction result is deep sleep, and the corresponding body movement situation in the second prediction result indicates that the target object has large or high-frequency body movements during the deep sleep stage, then the sleep stage with the highest probability in the first prediction result and its probability are deleted.

11. A sleep monitoring device, characterized in that, The device includes: A monitoring message receiving module is used to receive sleep monitoring messages; the sleep monitoring messages are obtained by analyzing the first type of sleep data, the second type of sleep data, and / or sleep monitoring results of the target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; both the first type of sleep data and the second type of sleep data are obtained based on non-contact devices; the sleep monitoring results are obtained by predicting the sleep stage of the target object using the first type of sleep data and the second type of sleep data respectively, and by the weights corresponding to the first prediction results and the second prediction results respectively; the second prediction result obtained from the second type of sleep data is used to assist the first prediction result obtained from the first type of sleep data, so as to delete abnormal sleep stages and their probabilities in the first prediction result by assisting the first prediction result with the second prediction result; the first prediction result and the second prediction result are respectively used to indicate the target object's sleep stage and its probability. The target object is in multiple sleep stages and the probability of each sleep stage; wherein the sleep stages include at least a first sleep stage and a second sleep stage; wherein the second prediction result is obtained by calling a second sleep staging model, the second sleep staging model is trained based on second training data carrying a second label, and the second label is generated based on a second device; when the second device includes a body movement recorder, the weight of the first prediction result for the first sleep stage is configured to be less than the weight of the second prediction result, and the weight of the first prediction result for the second sleep stage is configured to be greater than the weight of the second prediction result; the step of using the second prediction result to assist the first prediction result in deleting abnormal sleep stages and their probabilities in the first prediction result includes: if the sleep stage with the highest probability in the first prediction result is deep sleep, and the corresponding body movement situation in the second prediction result indicates that the target object has large or high-frequency body movements during the deep sleep period, then the sleep stage with the highest probability in the first prediction result and its probability are deleted. The monitoring message display module is used to display the sleep monitoring messages on the monitoring page.

12. An electronic device, characterized in that, include: At least one processor and at least one memory, wherein, The memory stores program instructions or code; The program instructions or code are loaded and executed by the processor, causing the electronic device to implement the sleep monitoring method as described in any one of claims 1 to 9.

13. A storage medium storing program instructions or code thereon, characterized in that, The program instructions or code are loaded and executed by the processor to implement the sleep monitoring method as described in any one of claims 1 to 9.

Images