Sleep monitoring method and device, electronic equipment and storage medium

Abstract

The invention provides a sleep monitoring method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first type of sleep data and a second type of sleep data of a target object; the first type of sleep data is used for describing the physiological condition of the target object, and the second type of sleep data is used for describing the body movement condition of the target object; the first type of sleep data and the second type of sleep data are obtained based on a non-contact device; predicting the sleep stage of the target object by using the first type of sleep data to obtain a first prediction result, and predicting the sleep stage of the target object by using the second type of sleep data to obtain a second prediction result; obtaining a sleep monitoring result of the target object according to the first prediction result and the second prediction result; the sleep monitoring result is used for indicating the sleep stage of the target object. According to the method and the device, the problems of high cost, inconvenience and inaccuracy of sleep monitoring in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the technical field of sleep monitoring, and specifically, to a sleep monitoring method, device, electronic device and storage medium. Background Art

[0002] Good sleep is an important factor in ensuring human health, and people are paying more and more attention to sleep. Usually, the sleep process includes wakefulness, rapid eye movement (REM) and non-rapid eye movement (NREM) sleep stages. A full night of sleep of 6 to 8 hours is roughly composed of 3 to 5 sleep cycles. The entire sleep process is a process of continuous alternation of various sleep stages. By monitoring the sleep stages people are in during sleep and performing sleep analysis, we can understand the sleep status and quality, and further adjust and improve sleep.

[0003] At present, there are various sleep monitoring products on the market, such as wearable devices, polysomnography (PSG), etc., but wearable devices are generally not accurate enough, and the contact characteristics of wearable devices will also affect the user's sleep and usage experience; monitoring sleep with the help of EEG, ECG, EMG and other acquisition devices, although accurate, is costly and often needs to be performed in special places such as hospitals, and requires professional operation, and cannot be performed for daily sleep monitoring at home.

[0004] From the above, we can see that there are still problems with sleep monitoring, such as high cost, inconvenience and inaccuracy. Summary of the invention

[0005] The present application provides a sleep monitoring method, device, electronic device and storage medium, which can solve the problems of high cost, inconvenience and inaccuracy of sleep monitoring in related technologies. The technical solution is as follows:

[0006] According to one aspect of the present application, a sleep monitoring method includes: obtaining first-category sleep data and second-category sleep data of a target object; the first-category sleep data is used to describe the physiological condition of the target object, and the second-category sleep data is used to describe the body movement of the target object; the first-category sleep data and the second-category sleep data are both obtained based on a non-contact device; using the first-category sleep data to predict the sleep stage of the target object to obtain a first prediction result, and using the second-category sleep data to predict the sleep stage of the target object to obtain a second prediction result; obtaining a sleep monitoring result of the target object based on the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

[0007] According to one aspect of the present application, a sleep monitoring device includes: a data acquisition module, used to acquire a first type of sleep data and a second type of sleep data of a target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; the first type of sleep data and the second type of sleep data are both obtained based on a non-contact device; a prediction module, used to use the first type of sleep data to predict the sleep stage of the target object to obtain a first prediction result, and use the second type of sleep data to predict the sleep stage of the target object to obtain a second prediction result; a result acquisition module, used to obtain the sleep monitoring result of the target object based on the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

[0008] In an exemplary embodiment, the first prediction result is obtained by calling a trained first sleep staging model; the device also includes a first model training module, used to train the first sleep staging model; the first model training module includes: a first data acquisition unit, used to acquire first training data carrying a first label; the first label is generated based on first indicator data collected from a training object using a first device, and is used to indicate the sleep stage of the training object; the first training data is the first type of sleep data of the training object; a first training unit, used to input the first training data carrying the first label into a first neural network, so as to train the first neural network and obtain the first sleep staging model.

[0009] In an exemplary embodiment, the prediction module includes: a first feature extraction unit, used to extract features from the first type of sleep data to obtain a first feature; a first probability calculation unit, used to calculate the probability that the target object belongs to each sleep stage based on the first feature, and obtain the first prediction result based on the probability.

[0010] In an exemplary embodiment, the second prediction result is obtained by calling a trained second sleep staging model; the device also includes a second model training module, used to train the second sleep staging model; the second model training module includes: a second data acquisition unit, used to acquire second training data carrying a second label; the second label is generated based on second indicator data collected from a training object using a second device, and is used to indicate the sleep stage of the training object; the second training data is the second type of sleep data of the training object; a second training unit, used to input the second training data carrying the second label into a second neural network, so as to train the second neural network and obtain the second sleep staging model.

[0011] In an exemplary embodiment, the prediction module includes: a second feature extraction unit, used to extract features from the second type of sleep data to obtain a second feature; a second probability calculation unit, used to calculate the probability that the target object belongs to each sleep stage based on the second feature, and obtain the second prediction result based on the probability.

[0012] In an exemplary embodiment, the result acquisition module includes: a weight determination unit, used to determine the weights corresponding to the first prediction result and the second prediction result respectively; a probability acquisition unit, used to obtain the probability that the target object is in each sleep stage according to the probabilities of the target object being in each sleep stage respectively indicated by the first prediction result and the second prediction result, and the weights corresponding to the first prediction result and the second prediction result respectively; the result acquisition unit is used to obtain the sleep monitoring result according to the probability that the target object is in each sleep stage.

[0013] In an exemplary embodiment, the result acquisition module also includes: a first configuration unit, used to configure the weight of the first prediction result for the first sleep stage to be smaller than the weight of the second prediction result; or a second configuration unit, used to configure the weight of the first prediction result for the second sleep stage to be greater than the weight of the second prediction result.

[0014] In an exemplary embodiment, the result acquisition module includes: a difference detection unit, used to perform difference detection on the first prediction result and the second prediction result to obtain a difference detection result, wherein the difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large; a deletion unit, used to remove the sleep stages and their probabilities with large differences from the first prediction result if the difference detection result indicates that the difference between the first prediction result and the second prediction result is large; a detection result acquisition unit, used to obtain the sleep monitoring result based on the deleted first prediction result and the second prediction result.

[0015] In an exemplary embodiment, the data acquisition module includes: a signal receiving unit, used to receive a radar signal; the radar signal is obtained by detecting the target object; a signal separation unit, used to separate the radar signal to obtain the first category of sleep data and the second category of sleep data; the first category of sleep data includes breathing data and heartbeat data; the second category of sleep data includes body movement data.

[0016] According to one aspect of the present application, a sleep monitoring method includes: receiving a sleep monitoring message; the sleep monitoring message is obtained by analyzing a first category of sleep data, a second category of sleep data, and / or a sleep monitoring result of a target object, and the sleep monitoring result is obtained by predicting the sleep stage of the target object using the first category of sleep data and the second category of sleep data; and displaying the sleep monitoring message on a monitoring page.

[0017] According to one aspect of the present application, a sleep monitoring device includes: a monitoring message receiving module, used to receive sleep monitoring messages; the sleep monitoring messages are obtained by analyzing the first category of sleep data, the second category of sleep data, and / or the sleep monitoring results of the target object, and the sleep monitoring results are obtained by predicting the sleep stage of the target object using the first category of sleep data and the second category of sleep data; a monitoring message display module, used to display the sleep monitoring messages on a monitoring page.

[0018] According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores program instructions or codes; the program instructions or codes are loaded and executed by the processor, so that the electronic device implements the sleep monitoring method as described above.

[0019] According to one aspect of the present application, a storage medium stores program instructions or codes thereon, wherein the program instructions or codes are loaded and executed by a processor to implement the sleep monitoring method as described above.

[0020] According to one aspect of the present application, an application product includes program instructions or codes, the program instructions or codes are stored in a storage medium, a processor of an electronic device reads the program instructions or codes from the storage medium, loads and executes the program instructions or codes, so that the electronic device implements the sleep monitoring method as described above.

[0021] The beneficial effects of the technical solution provided by this application are:

[0022] In the above technical solution, the first type of sleep data and the second type of sleep data of the target object are obtained, wherein the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; then the first type of sleep data and the second type of sleep data are used to predict the sleep stage of the target object, respectively, to obtain the first prediction result and the second prediction result, and finally the first prediction result and the second prediction result are combined to obtain the sleep monitoring result of the target object, and the sleep monitoring result indicates the sleep stage of the target object. The present application divides the sleep data into the first type of sleep data and the second type of sleep data, and predicts the sleep stage of the target object respectively. On the one hand, the prediction process can make more full use of the characteristic information of each type of sleep data, and on the other hand, the second prediction result is used to assist the first prediction result to obtain a more accurate sleep monitoring result, thereby improving the prediction accuracy; wherein both the first type of sleep data and the second type of sleep data can be obtained using non-contact equipment, without the need for expensive and complex professional acquisition equipment, which is convenient, highly feasible, and low-cost, and does not require contact with the user, avoiding affecting the user's sleep, and has a better user experience, thereby effectively solving the problems of high cost, inconvenience, and inaccuracy of sleep monitoring in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in describing the embodiments of the present application.

[0024] Figure 1 It is a schematic diagram of the implementation environment involved in this application;

[0025] Figure 2 is a flow chart of a sleep monitoring method according to an exemplary embodiment;

[0026] Figure 3 yes Figure 2 A flowchart of step 200 in one embodiment corresponding to the embodiment;

[0027] Figure 4 yes Figure 2 A flowchart of an embodiment corresponding to step 220 in an embodiment;

[0028] Figure 5 is a flowchart of a training process of a first sleep staging model according to an exemplary embodiment;

[0029] Figure 6 is a flowchart of a training process of a second sleep staging model according to an exemplary embodiment;

[0030] Figure 7 yes Figure 2A flowchart of an embodiment corresponding to step 240 in an embodiment;

[0031] Figure 8 yes Figure 7 The flowchart of an embodiment before step 700 in the corresponding embodiment;

[0032] Fig. 9 yes Figure 2 A flowchart of step 240 in another embodiment corresponding to the embodiment;

[0033] Fig.10 is a flow chart of another sleep monitoring method according to an exemplary embodiment;

[0034] Fig.11A It is a schematic diagram of a specific implementation of a sleep monitoring method in an application scenario;

[0035] Fig. 11B , Fig. 11C , Fig.11D It is a schematic diagram of the process of obtaining training data;

[0036] Fig.11E , Fig.11F As shown, it shows the training Fig.11A Schematic diagram of neural network 1 and 2;

[0037] Fig.12 is a structural block diagram of a sleep monitoring device according to an exemplary embodiment;

[0038] Fig.13 is a structural block diagram of another sleep monitoring device according to an exemplary embodiment;

[0039] Fig.14 is a hardware structure diagram of a terminal according to an exemplary embodiment;

[0040] Fig.15 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0042] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0043] The following is an introduction and explanation of several terms involved in this application:

[0044] Sleep staging refers to the process of segmenting the human sleep process and classifying each segment into different sleep stages. According to internationally accepted standards, sleep staging is usually divided into: wakefulness (Wake), non-rapid eye movement sleep (NREM) and rapid eye movement sleep (REM). Among them, the non-rapid eye movement sleep stage can be further divided into four stages: NREM1-4. For these four stages, NREM1-2 can be considered as light sleep, and NREM3-4 can be considered as deep sleep.

[0045] PSG refers to Polysomnography, which refers to the use of dedicated PSG equipment to collect and record more than 10 indicators such as EEG, ECG, and respiration during a full night of sleep. The equipment automatically analyzes the indicators and then manually corrects them item by item to monitor sleep time, sleep efficiency, and stages, thereby objectively evaluating the patient's sleep quality.

[0046] ATC refers to Actigraphy, which is a body motion recorder.

[0047] TST refers to Total Sleep Time, which is the total sleep time.

[0048] SE refers to Sleep Efficiency.

[0049] WASO refers to Wake After Sleep Onset, which is the time you wake up after sleep.

[0050] As mentioned earlier, as people pay more and more attention to sleep, various sleep monitoring products have appeared on the market, such as wearable devices, polysomnography (PSG), etc.

[0051] However, wearing wearable devices during sleep will affect the user's comfort and reduce the user experience. In addition, wearable devices used to determine sleep status are usually local wearable devices (such as bracelets and watches). They are worn on a part of the body, making it difficult to obtain the user's complete body movements. The sleep data obtained is limited, resulting in inaccurate determination of sleep status. In addition, wearable devices have a short battery life and need to be frequently charged and maintained, which is inconvenient to use.

[0052] Polysomnography (PSG) can accurately monitor sleep by acquiring data such as brain waves, electromyogram, electrocardiogram, electrooculogram, blood oxygen saturation and pulse, and then determining the user's sleep state through data analysis. However, it requires multiple electrodes and other equipment to be fixed on the user, and requires expensive equipment and a professional medical team to operate and analyze. It is costly, inconvenient, and difficult to promote and apply to daily sleep monitoring.

[0053] From the above, it can be seen that the relevant technology still has defects such as high cost, inconvenience and inaccuracy of sleep monitoring.

[0054] To this end, the sleep monitoring method provided by the present application can effectively improve the accuracy, convenience, and user experience of sleep monitoring. Accordingly, the sleep monitoring method is suitable for a sleep monitoring device, which can be deployed in an electronic device. The electronic device can be a non-contact device with a target detection function (such as a radar device), or a computer device configured with a von Neumann architecture. For example, the computer device may include a desktop computer, a laptop computer, a server, etc., or refers to a network device with a central control function, for example, the network device includes but is not limited to a gateway.

[0055] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0056] See also Figure 1 , Figure 1 Schematic diagram of an implementation environment involved in a sleep monitoring method. The implementation environment at least includes a user terminal 110, a smart device 130, a server 170, and a network device. Figure 1 In the embodiment, the network devices include a gateway 150 and a router 190, which is not specifically limited herein.

[0057] Among them, the user terminal 110, which can also be considered as a user end or terminal, can deploy (also understood as install) the client associated with the smart device 130. This user terminal 110 can be an electronic device such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart control panel, other devices with display and control functions, etc., which are not limited here.

[0058] Among them, the client is associated with the smart device 130, which essentially means that the user registers an account in the client and configures the smart device 130 in the client. This client can be in the form of an application or a web page. Correspondingly, the interface displayed by the client for the device can be in the form of a program window or a web page, which is not limited here.

[0059] The smart device 130 is deployed in the gateway 150, and communicates with the gateway 150 through its own configured communication module, and is then controlled by the gateway 150. It should be understood that the smart device 130 generally refers to one of the multiple smart devices 130, and the embodiment of the present application only takes the smart device 130 as an example, that is, the embodiment of the present application does not limit the number and device type of smart devices deployed in the gateway 150. In an application scenario, the smart device 130 accesses the gateway 150 through a local area network, and is thus deployed in the gateway 150. The process of the smart device 130 accessing the gateway 150 through the local area network includes: the gateway 150 first establishes a local area network, and the smart device 130 joins the local area network established by the gateway 150 by connecting to the gateway 150. This local area network includes but is not limited to: ZIGBEE or Bluetooth. Among them, the smart device 130 can be a contactless device (such as a radar device) for collecting sleep data for a target object.

[0060] The interaction between the user terminal 110 and the smart device 130 can be achieved through a local area network or a wide area network. In one application scenario, the user terminal 110 establishes a communication connection between the router 190 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to WIFI, so that the user terminal 110 and the gateway 150 are deployed in the same local area network, so that the user terminal 110 can interact with the smart device 130 through the local area network path. In another application scenario, the user terminal 110 establishes a communication connection between the server 170 and the gateway 150 in a wired or wireless manner. For example, the wired or wireless manner includes but is not limited to 2G, 3G, 4G, 5G, WIFI, etc., so that the user terminal 110 and the gateway 150 are deployed in the same wide area network, so that the user terminal 110 can interact with the smart device 130 through the wide area network path.

[0061] Among them, the server end 170 can also be considered as the cloud, cloud platform, platform end, service end, etc. This server end 170 can be a server, a server cluster composed of multiple servers, or a cloud computing center composed of multiple servers.

[0062] In one application scenario, the smart device 130 (radar device) detects the target object to obtain a radar signal, separates the first type of sleep data and the second type of sleep data from the radar signal, and transmits the first type of sleep data and the second type of sleep data to the server 170. The server 170 uses the first type of sleep data and the second type of sleep data to predict the sleep stage of the target object, obtains the first prediction result and the second prediction result, and then obtains the sleep monitoring result of the target object based on the first prediction result and the second prediction result. In addition, the server 170 can also send the sleep monitoring result to the user terminal 110 for display.

[0063] Of course, in other application scenarios, the sleep monitoring process completed by the server 170 can also be implemented by the smart device 130 (radar device). In this case, the first type of sleep data and the second type of sleep data are obtained, and the sleep monitoring results are obtained by using the first type of sleep data and the second type of sleep data. The sleep monitoring results are then sent to the user terminal 110 for display through the wide area network path / local area network path. For example, the first prediction result is obtained by calling the first sleep staging model, and the second prediction result is obtained by calling the second sleep staging model. The first sleep staging model and the second sleep staging model can be deployed in the smart device 130 (radar device), and then the sleep stage of the target object is predicted locally in the smart device 130 (radar device) to implement the sleep monitoring method.

[0064] See also Figure 2 The present application embodiment provides a sleep monitoring method, which is applicable to an electronic device, which may be Figure 1 The server 170 or gateway 150 in the illustrated implementation environment may also be Figure 1 A smart device 130 (radar device) is shown in an implementation environment.

[0065] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0066] like Figure 2 As shown, the method may include the following steps:

[0067] Step 200: Acquire first-category sleep data and second-category sleep data of a target object.

[0068] Among them, the target object refers to an object that can be detected by a non-contact device (such as a radar device). For example, the target object may be a person. The radar device may detect the person to obtain a radar signal, and then obtain corresponding first-category sleep data and second-category sleep data based on the radar signal.

[0069] The first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object, and both the first type of sleep data and the second type of sleep data are obtained at the same time based on the non-contact device.

[0070] A non-contact device refers to a device that can sense or measure a target object without directly contacting the target object. For example, the non-contact device may be a radar device, which is not specifically limited here.

[0071] In a possible implementation, the first type of sleep data includes breathing data and heartbeat data, and the second type of sleep data includes body movement data, which is not intended to be a specific limitation.

[0072] In one possible implementation, Figure 3 As shown, step 200 may include the following steps:

[0073] Step 300: receiving a radar signal.

[0074] Among them, the radar signal is obtained by the radar equipment detecting the target object.

[0075] In one possible implementation, the radar device may be a millimeter wave radar, which can sensitively capture micro-motion signals (tiny vibrations / movements) generated by a target object, which are regarded as radar signals.

[0076] Step 320 , performing signal separation on the radar signal to obtain the first type of sleep data and the second type of sleep data.

[0077] Among them, the first type of sleep data includes breathing data and heartbeat data, and the second type of sleep data includes body movement data.

[0078] The amplitude of local vibration on the body surface caused by breathing is about 5mm, and the amplitude of local vibration on the body surface caused by heartbeat is about 200-500μm, that is, human breathing, heartbeat and limb movement (body movement) will cause the body to generate micro-motion signals. Radar equipment can detect these micro-motion signals, and through processing and analysis of these micro-motion signals, it can separate the human breathing data, heartbeat data and body movement data.

[0079] In a possible implementation, separation algorithms such as wavelet transform, empirical mode decomposition, and Fourier transform are used to separate respiratory data, heartbeat data, and body movement data from radar signals.

[0080] In a possible implementation, breathing data, heartbeat data, and body movement data are separated from radar signals based on the difference in amplitude and frequency. For example, body movement has the largest amplitude and irregular frequency, while breathing and heartbeat have regular frequencies, and the ranges of breathing and heartbeat frequencies are also different. In addition, breathing and heartbeat cause different amplitudes of body vibrations.

[0081] Step 220, predicting the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and predicting the sleep stage of the target object using the second type of sleep data to obtain a second prediction result.

[0082] The first prediction result is used to indicate the sleep stage of the target object predicted based on the first type of sleep data, and the second prediction result is used to indicate the sleep stage of the target object predicted based on the second type of sleep data.

[0083] In a possible implementation, the sleep stages may include: wakefulness (Wake), non-rapid eye movement sleep (NREM) and rapid eye movement sleep (REM). It should be understood that the division of sleep stages can be flexibly adjusted according to the actual needs of the application scenario. For example, the sleep stages may also include: rapid eye movement sleep (REM) and wakefulness (Wake), as well as NREM-light sleep and NREM-deep sleep obtained by dividing the non-rapid eye movement sleep (NREM).

[0084] In a possible implementation, prediction refers to respectively calculating the probability that the target object belongs to each sleep stage based on the first category / second category sleep data.

[0085] Specifically, if Figure 4 As shown, step 220 may include the following steps:

[0086] Step 400: extract features from the first type of sleep data to obtain a first feature.

[0087] Among them, feature extraction can be achieved through neural networks (such as CNN, RNN) or traditional signal processing (such as Hilbert transform, empirical mode decomposition, etc.).

[0088] In a possible implementation, the first feature accurately describes the first type of sleep data in digital form, and may be a relevant parameter used to indicate the first type of sleep data. It can be understood that different first type of sleep data has different first features, that is, the first feature is used to uniquely identify the first type of sleep data. The first feature includes a respiratory rate feature and / or a heart rate feature.

[0089] Step 420, calculating the probability that the target object belongs to each sleep stage according to the first feature, and obtaining a first prediction result according to the probability.

[0090] In a possible implementation, the first feature is input into a first sleep staging model, and a classifier in the first sleep staging model is used to output the probability that the target object belongs to different sleep stages. The first sleep staging model is a machine learning model that has been trained and has the ability to predict the sleep stage of the target object, and the machine learning model can be a neural network, etc.

[0091] The first prediction result may indicate the probability that the target object belongs to different sleep stages, and may also indicate the sleep stage with the highest probability of the target object. For example, the first prediction result indicates that the probability that the target object is in the NREM-light sleep stage is 0.8, the probability that it is in the NREM-deep sleep stage is 0.7, the probability that it is in the REM stage is 0.5, and the probability that it is in the WAKE stage is 0.6; for another example, the probability of the NREM-light sleep stage is the highest, and the first prediction result indicates that the target object is in the NREM-light sleep stage, which is not specifically limited here.

[0092] Step 440: extract features from the second type of sleep data to obtain second features.

[0093] Among them, feature extraction can be achieved through neural networks (such as CNN, RNN) or traditional signal processing (such as Hilbert transform, empirical mode decomposition, etc.).

[0094] In a possible implementation, the second feature accurately describes the second type of sleep data in digital form, and may be a relevant parameter for indicating the second type of sleep data. It can be understood that different second types of sleep data have different second features, that is, the second feature is used to uniquely identify the second type of sleep data. The second feature includes a body movement feature.

[0095] Step 460, calculating the probability that the target object belongs to each sleep stage according to the second feature, and obtaining a second prediction result according to the probability.

[0096] In a possible implementation, the second feature is input into a second sleep staging model, and a classifier in the second sleep staging model is used to output the probability that the target object belongs to different sleep stages. The second sleep staging model is a machine learning model that has been trained and has the ability to predict the sleep stage of the target object, and the machine learning model can be a neural network, etc.

[0097] Similar to the first prediction result, the second prediction result can indicate the probability of the target object belonging to different sleep stages, and can also indicate the sleep stage with the highest probability of the target object.

[0098] Step 240: Obtain the sleep monitoring result of the target object according to the first prediction result and the second prediction result.

[0099] The sleep monitoring result is used to indicate the sleep stage of the target object.

[0100] In a possible implementation, corresponding weights are assigned to the first prediction result and the second prediction result, so as to obtain the sleep monitoring result of the target object based on the respective corresponding weights and the first prediction result and the second prediction result.

[0101] Through the above process, the first type of sleep data and the second type of sleep data of the target object are obtained, wherein the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; then the first type of sleep data and the second type of sleep data are used to predict the sleep stage of the target object, respectively, to obtain the first prediction result and the second prediction result, and finally the first prediction result and the second prediction result are combined to obtain the sleep monitoring result of the target object, and the sleep monitoring result indicates the sleep stage of the target object. The present application divides the sleep data into the first type of sleep data and the second type of sleep data, and predicts the sleep stage of the target object respectively. On the one hand, the prediction process can make more full use of the characteristic information of each type of sleep data, and on the other hand, the second prediction result is used to assist the first prediction result to obtain a more accurate sleep monitoring result, thereby improving the prediction accuracy; wherein both the first type of sleep data and the second type of sleep data can be obtained using non-contact equipment, without the need for expensive and complex professional acquisition equipment, which is convenient, highly feasible, and low-cost, and does not require contact with the user, thereby avoiding affecting the user's sleep, and having a better user experience, thereby effectively solving the problems of high cost, inconvenience, and inaccuracy of sleep monitoring existing in the relevant technology.

[0102] In an exemplary embodiment, the first prediction result in step 220 is obtained by calling a trained first sleep staging model.

[0103] The first sleep stage model is obtained by training a neural network, such as Figure 5As shown, the training process of the first sleep stage model may include the following steps:

[0104] Step 500: Acquire first training data carrying a first label.

[0105] The first label is generated based on first indicator data collected from the training object using the first device, and is used to indicate the sleep stage of the training object. The first training data is the first type of sleep data of the training object.

[0106] It is worth noting that while the first device is used to collect the first indicator data for the training object, the non-contact device collects the first training data for the training object, that is, the training object accepts data collection from the first device and the non-contact device at the same time.

[0107] The first device is a professional sleep monitoring device, which may include multiple sensors that can monitor and record human breathing, heart rate, body movement, body temperature and other data to achieve sleep monitoring.

[0108] In a possible implementation, the first device refers to a PSG device, which refers to a polysomnography instrument. The PSG device is used to collect data from the training object, and multiple index data of the training object can be collected, such as respiratory volume, apnea duration, snoring loudness, blood oxygen saturation, electromyographic signals, electroencephalographic signals, electrocardiographic signals and other index data. The sleep stage of the training object can be obtained through these index data. In other words, the sleep stage of the training object can be obtained through the PSG device, and the sleep stage of the training object obtained through the PSG device is used as the first label.

[0109] While using the PSG device to obtain the sleep stage (first label) of the training subject, the breathing data and heartbeat data of the training subject are obtained through a non-contact device, and the breathing data and heartbeat data (first training data) of the training subject are labeled with a first label in the time dimension, thereby constructing the first training data carrying the first label required for training the neural network.

[0110] For example, the breathing data and heart rate data of the training object from 11:00 to 11:15 are obtained through a contactless device. During the same period, the PSG device is used to obtain that the sleep stage of the training object from 11:00 to 11:15 is NERM1, and the breathing data and heart rate data of the training object from 11:00 to 11:15 are marked with the first label of NERM1.

[0111] In a possible implementation, in order to obtain enough first training data, the above process may be performed multiple times for different training objects to obtain enough first training data.

[0112] Step 520: input the first training data carrying the first label into the first neural network to train the first neural network and obtain a first sleep stage model.

[0113] The first neural network can be LeNet, a convolutional neural network, a long short-term memory, etc., which is not specifically limited here.

[0114] In a possible implementation, during the training of the first neural network, the first neural network outputs a first training result, the first training result is the sleep stage of the training object predicted by the first neural network, that is, the prediction result, and the first label is the sleep stage of the actual training object, that is, the correct result, and the first loss value is used to evaluate the difference between the prediction result (first training result) and the correct result (first label) of the first neural network, that is, the error. Minimization of the first loss function used to calculate the first loss value means that the smaller the error between the prediction result and the correct result of the first neural network, that is, the stronger the prediction ability of the first neural network is, and the more accurately the sleep stage of the target object can be predicted.

[0115] In a possible implementation, during the training process of the first neural network, the first neural network is trained with minimization of the first loss function as a convergence condition. Of course, the convergence condition can also be flexibly set according to the actual needs of the application scenario. For example, in one application scenario, the convergence condition is set to a specified number of iterations to improve the efficiency of model training.

[0116] In one possible implementation, when the loss function no longer decreases, or decreases very slowly, we believe that the neural network has converged. At this time, we can stop training, save the trained first neural network, and obtain the first sleep staging model to facilitate subsequent prediction of the target object's sleep stage.

[0117] In one possible implementation, the parameters of the neural network are updated according to the first loss value. Specifically, the first loss value is transferred from the output layer to the input layer of the neural network through the error back propagation algorithm, and the weight and bias of each neuron are adjusted according to the size of the first loss value to continue training the neural network.

[0118] In an exemplary embodiment, the second prediction result in step 220 is obtained by calling a trained second sleep staging model.

[0119] The second sleep stage model is obtained by training the second neural network, such as Figure 6 As shown, the training process of the second sleep stage model may include the following steps:

[0120] Step 600: Acquire second training data carrying a second label.

[0121] The second label is generated based on second indicator data collected from the training object using the second device, and is used to indicate the sleep stage of the training object. The second training data is the second type of sleep data of the training object.

[0122] It is worth noting that while the second device is used to collect the second indicator data for the training object, the non-contact device collects the second training data for the training object, that is, the training object accepts data collection from the second device and the non-contact device at the same time.

[0123] The second device is a professional sleep monitoring device, which may include multiple sensors that can monitor and record human breathing, heart rate, body movement, body temperature and other data to achieve sleep monitoring.

[0124] In a possible implementation, the second device is an ATC device, which refers to a body motion recorder. The ATC device is used to collect data from the training object, and the body motion data and sleep analysis parameters of the training object can be collected, such as sleep latency, total sleep time, sleep efficiency, sleep-wake rhythm and other ATC data (second indicator data). The sleep of the training object during the test can be staged through these ATC data to obtain the sleep stage of the training object. In other words, the sleep stage of the training object can be obtained through the ATC device, and the sleep stage of the training object obtained by the ATC device is used as the second label.

[0125] While using the ATC device to obtain the sleep stage (second label) of the training subject, the body motion data (second training data) of the training subject is obtained through a non-contact device, and the body motion data of the training subject is labeled with a second label in the time dimension, thereby constructing the second training data required for training the neural network.

[0126] For example, the body motion data of the training subject from 12:00 to 12:15 is obtained through a contactless device. During the same period, the ATC device is used to obtain the training subject's sleep stage from 12:00 to 12:15 as WAKE, and the body motion data of the training subject from 12:00 to 12:15 is marked with the first label of WAKE.

[0127] In a possible implementation, in order to obtain sufficient second training data, the above process may be performed multiple times for different training objects to obtain sufficient second training data.

[0128] Step 620: input the second training data carrying the second label into the second neural network to train the second neural network and obtain a second sleep stage model.

[0129] The second neural network can be LeNet, convolutional neural network, long short-term memory, etc., which is not specifically limited here.

[0130] In one possible implementation, during the process of training the second neural network, the second neural network outputs a second training result, where the second training result is used to indicate the sleep stage of the training object predicted by the second neural network, and based on the difference between the second training result and the second label, a second loss value is calculated, and it is determined whether the second loss value meets the convergence condition.

[0131] It should be understood that the second training result is the sleep stage of the training object predicted by the neural network, that is, the predicted result, the second label is the sleep stage of the actual training object, that is, the correct result, and the second loss value is used to evaluate the gap between the prediction result of the neural network (the second training result) and the correct result (the second label), that is, the error. Minimizing the second loss function used to calculate the second loss value means that the smaller the error between the prediction result of the neural network and the correct result, the stronger the prediction ability of the second neural network is, and the more accurately the sleep stage of the target object can be predicted.

[0132] In a possible implementation, during the training process of the second neural network, the second neural network is trained with minimization of the second loss function as a convergence condition. Of course, the convergence condition can also be flexibly set according to the actual needs of the application scenario. For example, in one application scenario, the convergence condition is set to a specified number of iterations to improve the efficiency of model training.

[0133] In one possible implementation, when the second loss function no longer decreases, or decreases very slowly, we believe that the second neural network has converged. At this time, we can stop training, save the trained second neural network, and obtain the second sleep staging model to facilitate subsequent prediction of the sleep stage of the target object.

[0134] In one possible implementation, the parameters of the neural network are updated according to the second loss value. Specifically, the first loss value is transferred from the output layer to the input layer of the neural network through the error back propagation algorithm, and the weight and bias of each neuron are adjusted according to the size of the first loss value to continue training the neural network.

[0135] Under the influence of the above-mentioned embodiments, different training data are constructed based on the PSG device and the ATC device for repeated iterative training of the neural network. Through repeated iterative training, the prediction ability of the neural network is continuously improved, and finally a trained first sleep staging model and a second sleep staging model are obtained. The trained first sleep staging model reflects the mathematical mapping relationship between the first training data and the first label, and the trained second sleep staging model reflects the mathematical mapping relationship between the second training data and the second label, thereby enabling accurate prediction of the sleep stage of the target object based on the first type of sleep data and the second type of sleep data of the target object.

[0136] In addition, the first sleep staging model and the second sleep staging model are trained separately based on different training data instead of training one sleep staging model with all the training data. On the one hand, the first prediction result obtained by the first sleep staging model is used to assist the second prediction result obtained by the second sleep staging model. This can avoid the incorrect sleep stage of the target object due to the inaccuracy of the first prediction result, thereby effectively improving the accuracy of model prediction. On the other hand, it can also reduce the overfitting risk of the neural network and improve the generalization ability of the neural network. On the other hand, training two models with small-scale training data is more conducive to improving the model training efficiency than training one model with large-scale training data, which is conducive to increasing the batches of repeated iterative training under the same training time, and ultimately helps to further improve the accuracy of model prediction.

[0137] See also Figure 7 In an exemplary embodiment, step 240 may include the following steps:

[0138] Step 700: Determine the weights corresponding to the first prediction result and the second prediction result respectively.

[0139] The first prediction result and the second prediction result respectively indicate the probability that the target object is in each sleep stage.

[0140] For example, the weights corresponding to the first prediction result and the second prediction result are determined as follows: in the first prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1, 1, 1, and 1, respectively; in the second prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1.3, 0.9, 0.8, and 0.8, respectively.

[0141] In a possible implementation, the value of the weight is related to the prediction accuracy of the first sleep staging model and the second sleep staging model.

[0142] Alternatively, if Figure 8 As shown, the following steps may be included before step 700:

[0143] Step 703 : configuring the weight of the first prediction result to be smaller than the weight of the second prediction result for the first sleep stage.

[0144] Step 704 : configuring the weight of the first prediction result to be greater than the weight of the second prediction result for the second sleep stage.

[0145] In a possible implementation, the first sleep stage refers to the WAKE stage, and the second sleep stage includes the REM stage, the NREM-light sleep stage, and the NREM-deep sleep stage.

[0146] As mentioned above, the first prediction result and the second prediction result are obtained by calling the trained first sleep staging model and the second sleep staging model respectively. The training data of the first sleep staging model is the first training data carrying the first label, and the training data of the second sleep staging model is the second training data carrying the second label. The first label is generated based on the PSG device, and the second label is generated based on the ATC device. However, the inventors realized that although the sleep monitoring of the ATC device and the PSG device is accurate and reliable, and the results are well correlated, the ATC has a tendency to over-evaluate TST (total sleep time) and SE (sleep efficiency) and under-evaluate WASO (awake time after sleep), which leads to the second sleep staging model predicting a higher probability of REM, NREM-light sleep, and NREM-deep sleep, while predicting a lower probability of the WAKE period. Based on this, when the probability of the target object being in each sleep stage is obtained using the first sleep staging model and the second sleep staging model, for the WAKE stage, the weight of the first prediction result is less than the weight of the second prediction result; for the REM stage, NREM-light sleep, and NREM-deep sleep stage, the weight of the first prediction result is greater than the weight of the second prediction result. For example, in the first prediction result, the weights of the WAKE stage, REM stage, NREM-light sleep stage, and NREM-deep sleep stage are 1, 1, 1, and 1 respectively; in the second prediction result, the weights of the WAKE stage, REM stage, NREM-light sleep stage, and NREM-deep sleep stage are 1.3, 0.9, 0.8, and 0.8 respectively.

[0147] By configuring the weights, the first prediction result and the second prediction result can be corrected to increase the accuracy of the sleep prediction result.

[0148] Step 720, obtaining the probability that the target object is in each sleep stage according to the probabilities of the target object being in each sleep stage respectively indicated by the first prediction result and the second prediction result, and the weights respectively corresponding to the first prediction result and the second prediction result.

[0149] For example, the first prediction result indicates that the probability of being in the NREM-light sleep period is 0.8, the probability of being in the NREM-deep sleep period is 0.7, the probability of being in the REM period is 0.5, and the probability of being in the WAKE period is 0.6; the second prediction result indicates that the probability of being in the NREM-light sleep period is 0.9, the probability of being in the NREM-deep sleep period is 0.8, the probability of being in the REM period is 0.5, and the probability of being in the WAKE period is 0.5. In the first prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1, 1, 1, and 1 respectively; in the second prediction result, the weights of the WAKE period, REM period, NREM-light sleep period, and NREM-deep sleep period are 1.3, 0.9, 0.8, and 0.8 respectively. Then the probability of the WAKE period is 0.6×1+0.5×1.3=1.25, the probability of the NREM-light sleep period is 0.8×1+0.9×0.8=1.52, the probability of the NREM-deep sleep period is 0.7×1+0.8×0.8=1.34, and the probability of the REM period is 0.5×1+0.5×0.9=0.95.

[0150] Step 740, obtaining a sleep monitoring result according to the probability of the target object being in each sleep stage.

[0151] In a possible implementation, the sleep stage with the highest probability is selected as the sleep stage of the target object. For example, if the probability of NREM-light sleep is the highest, it is considered that the sleep stage actually in which the target object is in is NREM-light sleep.

[0152] Through the above process, the weights corresponding to the first prediction result and the second prediction result are determined, and the sleep monitoring result is obtained in a weighted manner, so that a more accurate sleep monitoring result can be obtained. When the probability of each sleep stage in the first prediction result and the second prediction result is not accurately predicted, by assigning weights to the first prediction result and the second prediction result, the inaccuracy can be corrected, which can further promote the accuracy of sleep monitoring.

[0153] See also Fig. 9 In an exemplary embodiment, step 240 may include the following steps:

[0154] Step 241, performing a difference detection on the first prediction result and the second prediction result to obtain a difference detection result.

[0155] The difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large.

[0156] For example, the first prediction result indicates that the probability of deep sleep is the highest, but the second prediction result indicates that the probability of deep sleep is the lowest. At this time, the difference detection result indicates that the difference between the first prediction result and the second prediction result is large, and the sleep stage with a large difference is deep sleep.

[0157] Step 242: If the difference detection result indicates that the difference between the first prediction result and the second prediction result is large, the sleep stage with the large difference and its probability are removed from the first prediction result.

[0158] Continuing with the above example, the sleep stage (deep sleep stage) with a large difference is deleted from the first prediction result.

[0159] Step 243: Obtain a sleep monitoring result based on the deleted first prediction result and the second prediction result.

[0160] Still using the above example to illustrate, after deleting the sleep stages and their probabilities with large differences, in order to obtain the sleep result, it depends on the sleep stage with the second highest probability in the first prediction result and the sleep stage with the highest probability in the second prediction result.

[0161] Under the effect of the above-mentioned embodiment, by combining the first prediction result and the second prediction result, the second prediction result can assist the first prediction result to obtain a more accurate sleep monitoring result, which is beneficial to improving the accuracy of the prediction.

[0162] See also Fig.10 , the present application embodiment provides another sleep monitoring method, the method is applicable to an electronic device, the electronic device may be Figure 1 A user terminal 110 is shown in an implementation environment.

[0163] In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

[0164] like Fig.10 As shown, the method may include the following steps:

[0165] Step 800, receiving a sleep monitoring message.

[0166] The sleep monitoring message is obtained based on the analysis of the first category sleep data, the second category sleep data, and / or the sleep monitoring result of the target object. The sleep monitoring result is obtained by predicting the sleep stage of the target object using the first category sleep data and the second category sleep data.

[0167] It should be understood that the process of obtaining a sleep monitoring message based on the target object's first category sleep data, second category sleep data, and / or sleep monitoring results can be completed by a non-contact device (such as a radar device) or by other electronic devices. For example, after the non-contact device collects and obtains the first category sleep data and the second category sleep data, the first category sleep data and the second category sleep data are transmitted to a server, and the server executes the sleep monitoring method provided in the present application to obtain the sleep monitoring results, and then obtains the sleep monitoring message based on the target object's first category sleep data, second category sleep data, and / or sleep monitoring results.

[0168] Sleep monitoring messages may include sleep reports, abnormal warnings, etc., which are not specifically limited here. For example, a sleep report may include: the duration of deep sleep, the duration of light sleep, the breathing rate during sleep, the heart rate during sleep, the number and amplitude of body movements during sleep, the number of awakenings during sleep and the duration of each awakening, etc. The sleep report may be a sleep report for one night's sleep, or a long-term sleep report. An abnormal warning may include an abnormal notification generated in response to abnormal heart rate and / or breathing values ​​(such as tachypnea, apnea, bradycardia, tachycardia).

[0169] Step 820: Display the sleep monitoring message on the monitoring page.

[0170] The detection page refers to the display interface of the client of the user terminal. The client can be in the form of an application or a web page. Correspondingly, the display interface of the client can be in the form of a program window or a web page, which is not limited here.

[0171] In the above process, after the sleep monitoring message is generated, it will be transmitted to the user terminal for display, so that the user can understand the sleep quality and sleep breathing problems, etc., which can help the user better understand their sleep conditions and effectively solve sleep problems.

[0172] Fig.11A This is a schematic diagram of a specific implementation of a sleep monitoring method in an application scenario.

[0173] In this application scenario, the real-time collected radar signal is first subjected to feature extraction, and the radar signal is separated and extracted to obtain respiratory rate features, heart rate features, and body movement features respectively; the separated heart rate and respiratory rate are then input into the trained neural network 1 to obtain a preliminary classification result 1, which includes: the probability of each classification label (sleep stage) corresponding to the input, for example, the probability that the human body is in the NREM-light sleep period is 0.8, the probability that the human body is in the NREM-deep sleep period is 0.7, the probability that the human body is in the REM period is 0.5, and the probability that the human body is in the WAKE period is 0.6.

[0174] The body motion signal is then separated from the radar signal and input into the trained neural network 2 to obtain a preliminary classification result 2, which includes: the probability of each type of label (sleep stage) corresponding to the input, for example, the probability of a human body being in NREM-light sleep is 0.9, the probability of a human body being in NREM-deep sleep is 0.8, the probability of a human body being in REM sleep is 0.5, and the probability of a human body being in WAKE sleep is 0.5.

[0175] Finally, the classification results 1 and 2 are calculated in a weighted manner to obtain the final classification result of sleep stages. The final classification result includes: the probability of each type of label (sleep stage) corresponding to the radar signal, for example, the probability of the human body being in NREM-light sleep is 1.52, the probability of the human body being in NREM-deep sleep is 1.34, the probability of the human body processing REM period is 0.95, and the probability of the human body being in WAKE period is 1.25. According to the order of probability, the classification label (sleep stage) with the largest probability is given priority as the output sleep stage result.

[0176] The trained neural network 1 and neural network 2 are obtained based on the training data, such as Fig. 11B , Fig. 11C , Fig.11D As shown, they are schematic diagrams showing the process of acquiring training data.

[0177] exist Fig. 11B , Fig. 11C , Fig.11D In the test, testers are simultaneously monitored by PSG equipment, ATC equipment and millimeter-wave radar data collection in the same test.

[0178] PSG can obtain indicators such as the subject's breathing volume, apnea duration, snoring loudness, blood oxygen saturation, electromyographic signals, electroencephalographic signals, and electrocardiographic signals. These indicators can be used to divide the subject's sleep during the test into stages, that is, to label the subject in the time dimension as 1 (for example: 11:00-11:15 WEAK, 12:00-12:15 NERM1).

[0179] ATC can obtain the subject's body movement data and sleep analysis parameters, such as sleep latency, total sleep time, sleep efficiency, sleep-wake rhythm and other indicators. These indicators can be used to stage the subject's sleep during the test, that is, to label the subject in the time dimension2 (for example: 11:00-11:15 WEAK, 12:00-12:15 SLEEP).

[0180] The millimeter wave radar collects radar signals from the subject and further separates them into respiratory rate, heart rate, and body movement. There are many algorithms to choose from to separate the radar signals into respiratory rate, heart rate, and body movement (amplitude and frequency of human body vibration) (for example: wavelet transform, empirical mode decomposition, Fourier transform, etc.).

[0181] After preparing the training data (respiration rate, heart rate, body movement, label 1, label 2), neural networks 1 and 2 can be trained, such as Fig.11E , Fig.11F As shown, it shows a schematic diagram of training neural networks 1 and 2.

[0182] Convolutional neural network, support vector machine (SVM), LeNet, long short-term memory (LSTM), etc. can be selected as neural network 1 and 2, without specific limitation here.

[0183] The prepared respiratory rate, heart rate and label 1 are used as input to train the neural network 1. The trained neural network 1 establishes a mapping relationship between the respiratory rate, heart rate and label 1.

[0184] The prepared body movement data and label 2 are used as input to train the neural network 2, and the trained neural network 2 establishes a mapping relationship between the body movement and the label 2.

[0185] Studies have shown that the sleep tests of ATC and PSG are accurate and reliable, and the results are well correlated. However, ATC tends to over-evaluate TST and SE and underestimate WASO time, which also leads to the tendency of neural network 2 trained based on ATC to over-evaluate TST and SE and underestimate WASO time. In order to ensure the accuracy of the final sleep staging results, when using classification results 1 and classification results 2 to obtain the final sleep staging results, the weight of classification result 1 corresponding to the WAKE period is less than the weight of classification result 2, and for REM period, NREM-light sleep, and NREM-deep sleep period, the weight of classification result 1 is greater than the weight of classification result 2.

[0186] It should be noted that the value of the weight is related to the accuracy of the training results of the neural networks 1 and 2 selected for actual training, and ATC and PSG are only possible factors that affect the accuracy of the classification results of the neural networks 1 and 2. Therefore, the above weight distribution is only an example. In fact, all factors that affect the accuracy of the training results of the neural networks 1 and 2 must be comprehensively considered.

[0187] In this application scenario, PSG and ATC devices are only used to construct training data to facilitate training of neural networks. After the neural network is trained, it is only necessary to separate the respiratory rate, heart rate, and body movement from the radar signal to perform sleep monitoring. In other words, sleep monitoring can be achieved based on non-contact equipment (radar equipment), which has little interference with the user's sleep and is friendly to patients with burns and scalds, as well as children. It does not require the troublesome operation of professional equipment such as PSG and ATC. It is simple and easy to operate, can be performed at home, and is low cost.

[0188] The following is an embodiment of the device of the present application, which can be used to perform the sleep monitoring method involved in the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of the sleep monitoring method involved in the present application.

[0189] See also Fig.12 In an embodiment of the present application, a sleep monitoring device 900 is provided, including but not limited to: a data acquisition module 910, a prediction module 930, and a result acquisition module 950.

[0190] Among them, the data acquisition module 910 is used to obtain the first type of sleep data and the second type of sleep data of the target object; the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; the first type of sleep data and the second type of sleep data are both obtained based on non-contact devices.

[0191] The prediction module 930 is used to predict the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and to predict the sleep stage of the target object using the second type of sleep data to obtain a second prediction result.

[0192] The result acquisition module 950 is used to obtain the sleep monitoring result of the target object according to the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

[0193] In an exemplary embodiment, the first prediction result is obtained by calling a trained first sleep staging model; the device 900 also includes a first model training module 960, which is used to train the first sleep staging model; the first model training module 960 includes: a first data acquisition unit 961, which is used to acquire first training data carrying a first label; the first label is generated based on first indicator data collected from a training object using a first device, and is used to indicate the sleep stage of the training object; the first training data is the first type of sleep data of the training object; a first training unit 962, which is used to input the first training data carrying the first label into a first neural network to train the first neural network to obtain the first sleep staging model.

[0194] In an exemplary embodiment, the prediction module 930 includes: a first feature extraction unit 931, used to extract features from the first type of sleep data to obtain a first feature; a first probability calculation unit 932, used to calculate the probability that the target object belongs to each sleep stage based on the first feature, and obtain the first prediction result based on the probability.

[0195] In an exemplary embodiment, the second prediction result is obtained by calling a trained second sleep staging model; the device 900 also includes a second model training module 970, used to train the second sleep staging model; the second model training module 970 includes: a second data acquisition unit 971, used to acquire second training data carrying a second label; the second label is generated based on second indicator data collected from a training object using a second device, and is used to indicate the sleep stage of the training object; the second training data is the second type of sleep data of the training object; a second training unit 972, used to input the second training data carrying the second label into a second neural network to train the second neural network to obtain the second sleep staging model.

[0196] In an exemplary embodiment, the prediction module 930 includes: a second feature extraction unit 933, used to extract features from the second type of sleep data to obtain a second feature; a second probability calculation unit 934, used to calculate the probability that the target object belongs to each sleep stage based on the second feature, and obtain the second prediction result based on the probability.

[0197] In an exemplary embodiment, the result acquisition module 950 includes: a weight determination unit 951, used to determine the weights corresponding to the first prediction result and the second prediction result respectively; a probability acquisition unit 952, used to obtain the probability of the target object being in each sleep stage according to the probabilities of the target object being in each sleep stage respectively indicated by the first prediction result and the second prediction result, and the weights corresponding to the first prediction result and the second prediction result respectively; a result acquisition unit 953, used to obtain the sleep monitoring result according to the probability of the target object being in each sleep stage.

[0198] In an exemplary embodiment, the result acquisition module 950 includes: a difference detection unit 954, which is used to perform difference detection on the first prediction result and the second prediction result to obtain a difference detection result, and the difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large; a deletion unit 955, which is used to delete the sleep stages and their probabilities with large differences from the first prediction result if the difference detection result indicates that the difference between the first prediction result and the second prediction result is large; a detection result acquisition unit 956, which is used to obtain the sleep monitoring result based on the deleted first prediction result and the second prediction result.

[0199] In an exemplary embodiment, the result acquisition module 950 also includes: a first configuration unit 957, used to configure the weight of the first prediction result for the first sleep stage to be smaller than the weight of the second prediction result; or a second configuration unit 958, used to configure the weight of the first prediction result for the second sleep stage to be greater than the weight of the second prediction result.

[0200] In an exemplary embodiment, the data acquisition module 910 includes: a signal receiving unit 911, used to receive a radar signal; the radar signal is obtained by detecting the target object; a signal separation unit 912, used to separate the radar signal to obtain the first category of sleep data and the second category of sleep data; the first category of sleep data includes breathing data and heartbeat data; the second category of sleep data includes body movement data.

[0201] See also Fig.13 Another sleep monitoring device 1000 is provided in an embodiment of the present application, including but not limited to: a monitoring message receiving module 1010 and a monitoring message display module 1030.

[0202] The monitoring message receiving module 1010 is used to receive sleep monitoring messages.

[0203] The monitoring message display module 1030 is used to display the sleep monitoring message in the monitoring page.

[0204] It should be noted that the sleep monitoring device and the sleep monitoring message display device provided in the above embodiments only use the division of the above-mentioned functional modules as examples when performing sleep monitoring and sleep monitoring message display. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the sleep monitoring device and the sleep monitoring message display device will be divided into different functional modules to complete all or part of the functions described above.

[0205] In addition, the sleep monitoring device, sleep monitoring message display device, sleep monitoring method, and sleep monitoring message display method embodiments provided in the above embodiments belong to the same concept, and the specific manner in which each module performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0206] See also Fig.14 , Fig.14 is a schematic diagram of a terminal structure according to an exemplary embodiment. The terminal is suitable for Figure 1 A user terminal 110 is shown in an implementation environment.

[0207] It should be noted that the terminal is only an example adapted to the present application and cannot be considered as any limitation on the scope of use of the present application. The terminal cannot be interpreted as requiring dependence on or having Fig.14 One or more components of the exemplary terminal 1100 shown in FIG.

[0208] like Fig.14 As shown, the terminal 1100 includes a memory 101, a storage controller 103, one or more ( Fig.14 The processor 105 (only one is shown in the figure), the peripheral interface 107, the radio frequency module 109, the positioning module 111, the camera module 113, the audio module 115, the touch screen 117 and the key module 119. These components communicate with each other via one or more communication buses / signal lines 121.

[0209] Among them, the memory 101 can be used to store program instructions or codes, such as the program instructions or codes corresponding to the sleep monitoring method, sleep monitoring message display method and device in the exemplary embodiments of the present application. The processor 105 executes various functions and data processing by running the program instructions or codes stored in the memory 101, that is, completes the sleep monitoring method and the sleep monitoring message display method.

[0210] The memory 101 is a carrier for storing resources, and may be a random access memory, such as a high-speed random access memory, a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other solid-state memory. The storage method may be temporary storage or permanent storage.

[0211] The peripheral interface 107 may include at least one wired or wireless network interface, at least one serial-parallel conversion interface, at least one input / output interface, and at least one USB interface, etc., for coupling various external input / output devices to the memory 101 and the processor 105 to achieve communication with various external input / output devices.

[0212] The radio frequency module 109 is used to receive and transmit electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with other devices through a communication network. The communication network includes a cellular telephone network, a wireless local area network or a metropolitan area network, and the above communication network can use various communication standards, protocols and technologies.

[0213] The positioning module 111 is used to obtain the current geographical location of the terminal 1100. Examples of the positioning module 111 include, but are not limited to, a global satellite positioning system (GPS) and positioning technology based on a wireless local area network or a mobile communication network.

[0214] The camera module 113 is a camera for taking pictures or videos. The taken pictures or videos can be stored in the memory 101 and can also be sent to the host computer through the radio frequency module 109.

[0215] The audio module 115 provides an audio interface to the user, which may include one or more microphone interfaces, one or more speaker interfaces, and one or more headphone interfaces. The audio data is interacted with other devices through the audio interface. The audio data can be stored in the memory 101 and can also be sent through the radio frequency module 109.

[0216] The touch screen 117 provides an input / output interface between the terminal 1100 and the user. Specifically, the user can perform input operations through the touch screen 117, such as gesture operations such as clicking, touching, and sliding, so that the terminal 1100 responds to the input operation. The terminal 1100 displays the output content formed by any form or combination of text, pictures, or videos to the user through the touch screen 117.

[0217] The key module 119 includes at least one key to provide an interface for the user to input to the terminal 1100. The user can press different keys to make the terminal 1100 perform different functions. For example, the sound adjustment key allows the user to adjust the volume of the sound played by the terminal 1100.

[0218] Understandably, Fig.14 The structure shown is for illustration only. The terminal 1100 may also include Fig.14 More or fewer components as shown, or with Fig.14 The different components are shown. Fig.14Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0219] See also Fig.15 In an embodiment of the present application, an electronic device 4000 is provided, and the electronic device 4000 may include: a desktop computer, a laptop computer, a server, a smart phone, etc.

[0220] exist Fig.15 In the embodiment, the electronic device 4000 includes at least one processor 4001 and at least one memory 4003.

[0221] The data interaction between the processor 4001 and the memory 4003 can be realized through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0222] Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0223] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0224] The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 4000, but is not limited to this.

[0225] The memory 4003 stores program instructions or codes, and the processor 4001 can read the program instructions or codes stored in the memory 4003 through the communication bus 4002 .

[0226] When the program instructions or codes are executed by the processor 4001, the sleep monitoring method and the sleep monitoring message display method in the above-mentioned embodiments are implemented.

[0227] In addition, a storage medium is provided in an embodiment of the present application, on which program instructions or codes are stored, and the program instructions or codes are loaded and executed by a processor to implement the sleep monitoring method and sleep monitoring message display method as described above.

[0228] In an embodiment of the present application, an application product is provided, which includes program instructions or codes, and the program instructions or codes are stored in a storage medium. A processor of an electronic device reads the program instructions or codes from the storage medium, loads and executes the program instructions or codes, so that the electronic device implements the sleep monitoring method and sleep monitoring message display method as described above.

[0229] Compared with the related art, this application has the following beneficial effects:

[0230] 1. Obtain the first type of sleep data and the second type of sleep data of the target object, wherein the first type of sleep data is used to describe the physiological condition of the target object, and the second type of sleep data is used to describe the body movement of the target object; then use the first type of sleep data and the second type of sleep data to predict the sleep stage of the target object, respectively, to obtain a first prediction result and a second prediction result, and finally combine the first prediction result and the second prediction result to obtain the sleep monitoring result of the target object. This application divides the sleep data into the first type of sleep data and the second type of sleep data, and predicts them separately, which can make fuller use of the characteristic information of each sleep data, thereby improving the prediction accuracy.

[0231] Both the first and second types of sleep data can be obtained using non-contact devices (such as radar), without the need for expensive and complex professional equipment. They are convenient, highly feasible, and low-cost, and can meet the needs of daily family sleep monitoring. They do not require contact with the user, have little interference with the user's normal sleep, are friendly to patients with burns and scalds, and children, and have a better user experience.

[0232] In summary, the present application effectively solves the problems of high cost, inconvenience and inaccuracy of sleep monitoring existing in the related art.

[0233] 2. Construct training data based on the PSG device and the ATC device, and the obtained labeled training data contains the input and the corresponding true value label. During the training process, the neural network uses the input as the data of the input layer, and obtains the training result after calculation and processing by the neural network. Compare this training result with the true value label, calculate the loss value, and adjust the parameters of the neural network according to the loss value, so that the neural network can predict the output result more accurately. Through repeated iterative training, the prediction ability of the neural network is continuously improved, and finally a trained sleep staging model is obtained. The trained first sleep staging model includes the mapping relationship between the first training data and the first label, and the trained second sleep staging model includes the mapping relationship between the second training data and the second label. Training the first sleep staging model and the second sleep staging model separately can improve the prediction accuracy of the neural network, while also reducing the risk of overfitting of the neural network and improving the generalization ability of the neural network.

[0234] 3. Determine the weights corresponding to the first prediction result and the second prediction result, and obtain the sleep monitoring result in a weighted manner, so as to obtain a more accurate sleep monitoring result. When the probability of each sleep stage in the first prediction result and the second prediction result is not accurately predicted, by assigning weights to the first prediction result and the second prediction result, the inaccuracy can be corrected, which can further improve the accuracy of sleep monitoring.

[0235] 4. After the sleep monitoring message is generated, it will be transmitted to the user terminal for display, so that the user can understand the sleep quality and sleep breathing problems, etc., which can help the user better understand their sleep conditions and effectively solve sleep problems.

[0236] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0237] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A sleep monitoring method, characterized in that: The method comprises: Acquire first-category sleep data and second-category sleep data of a target object; the first-category sleep data is used to describe the physiological condition of the target object, and the second-category sleep data is used to describe the body movement of the target object; the first-category sleep data and the second-category sleep data are both obtained based on a non-contact device; Predicting the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and predicting the sleep stage of the target object using the second type of sleep data to obtain a second prediction result; A sleep monitoring result of the target object is obtained according to the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

2. The method according to claim 1, characterized in that The first prediction result is obtained by calling a trained first sleep stage model; The training process of the first sleep staging model includes: Acquire first training data carrying a first label; the first label is generated based on first indicator data collected from a training subject using a first device, and is used to indicate the sleep stage of the training subject; the first training data is first type of sleep data of the training subject; The first training data carrying the first label is input into a first neural network to train the first neural network and obtain the first sleep stage model.

3. The method according to claim 1, characterized in that The step of using the first type of sleep data to predict the sleep stage of the target object to obtain a first prediction result includes: Extracting features from the first type of sleep data to obtain a first feature; The probability that the target object belongs to each sleep stage is calculated according to the first feature, and the first prediction result is obtained according to the probability.

4. The method according to claim 1, characterized in that The second prediction result is obtained by calling a trained second sleep stage model; The training process of the second sleep staging model includes: Acquire second training data carrying a second label; the second label is generated based on second indicator data collected from the training object using a second device, and is used to indicate the sleep stage of the training object; the second training data is second type of sleep data of the training object; The second training data carrying the second label is input into a second neural network to train the second neural network and obtain the second sleep stage model.

5. The method according to claim 1, characterized in that The step of using the second type of sleep data to predict the sleep stage of the target object to obtain a second prediction result includes: performing feature extraction on the second type of sleep data to obtain a second feature; The probability that the target object belongs to each sleep stage is calculated according to the second feature, and the second prediction result is obtained according to the probability.

6. The method according to claim 1, characterized in that The first prediction result and the second prediction result are respectively used to indicate the probability that the target object is in each sleep stage; The step of obtaining the sleep monitoring result of the target object according to the first prediction result and the second prediction result includes: Determine weights corresponding to the first prediction result and the second prediction result respectively; Obtaining the probability that the target object is in each sleep stage according to the probabilities of the target object being in each sleep stage respectively indicated by the first prediction result and the second prediction result, and the weights respectively corresponding to the first prediction result and the second prediction result; The sleep monitoring result is obtained according to the probability that the target object is in each sleep stage.

7. The method according to claim 6, characterized in that Before determining the weights respectively corresponding to the first prediction result and the second prediction result, the method further includes: configuring the weight of the first prediction result for the first sleep stage to be smaller than the weight of the second prediction result; or The weight of the first prediction result configured for the second sleep stage is greater than the weight of the second prediction result.

8. The method according to claim 1, characterized in that The step of obtaining the sleep monitoring result of the target object according to the first prediction result and the second prediction result includes: Performing a difference detection on the first prediction result and the second prediction result to obtain a difference detection result; the difference detection result is used to indicate whether the difference between the first prediction result and the second prediction result is large; If the difference detection result indicates that the difference between the first prediction result and the second prediction result is large, the sleep stage with the large difference and its probability are deleted from the first prediction result; The sleep monitoring result is obtained based on the deleted first prediction result and the second prediction result.

9. The method according to any one of claims 1 to 8, characterized in that The step of obtaining the first type of sleep data and the second type of sleep data of the target object includes: Receiving a radar signal; the radar signal is obtained by detecting the target object; performing signal separation on the radar signal to obtain the first category of sleep data and the second category of sleep data; The first type of sleep data includes breathing data and heartbeat data; The second type of sleep data includes body movement data.

10. A sleep monitoring method, characterized in that: The method comprises: receiving a sleep monitoring message; the sleep monitoring message is obtained by analyzing the first type of sleep data, the second type of sleep data, and / or the sleep monitoring result of the target object, and the sleep monitoring result is obtained by predicting the sleep stage of the target object using the first type of sleep data and the second type of sleep data; The sleep monitoring message is displayed on a monitoring page.

11. A sleep monitoring device, characterized in that: The device comprises: A data acquisition module, used to acquire first-category sleep data and second-category sleep data of a target object; the first-category sleep data is used to describe the physiological condition of the target object, and the second-category sleep data is used to describe the body movement of the target object; the first-category sleep data and the second-category sleep data are both obtained based on a non-contact device; a prediction module, configured to predict the sleep stage of the target object using the first type of sleep data to obtain a first prediction result, and to predict the sleep stage of the target object using the second type of sleep data to obtain a second prediction result; A result acquisition module is used to obtain the sleep monitoring result of the target object according to the first prediction result and the second prediction result; the sleep monitoring result is used to indicate the sleep stage of the target object.

12. A sleep monitoring device, characterized in that: The device comprises: A monitoring message receiving module, used to receive a sleep monitoring message; the sleep monitoring message is obtained by analyzing the first type of sleep data, the second type of sleep data, and / or the sleep monitoring result of the target object, and the sleep monitoring result is obtained by predicting the sleep stage of the target object using the first type of sleep data and the second type of sleep data; The monitoring message display module is used to display the sleep monitoring message on the monitoring page.

13. An electronic device, characterized in that: include: at least one processor and at least one memory, wherein: The memory stores program instructions or codes; The program instructions or codes are loaded and executed by the processor, so that the electronic device implements the sleep monitoring method according to any one of claims 1 to 10.

14. A storage medium having program instructions or codes stored thereon, characterized in that: The program instructions or codes are loaded and executed by a processor to implement the sleep monitoring method according to any one of claims 1 to 10.

Images