Method, device, equipment, medium and program product for detecting a motor state during sleep

Abstract

The present disclosure relates to a sleep motion state detection method, device, equipment, medium and program product. According to an example of the present disclosure, the method comprises: in response to detecting that a user is in a sleep state, acquiring a pulse signal and an acceleration signal of the user; segmenting the pulse signal and the acceleration signal based on a preset time window to obtain a plurality of pulse signal segments and a plurality of acceleration signal segments; combining the pulse signal segments and the acceleration signal segments in the same time window to obtain a plurality of signal segment combinations, wherein each signal segment combination corresponds to a time window; and based on each signal segment combination and a pre-trained sleep motion state recognition model, obtaining a sleep motion state recognition result corresponding to the time window. The present disclosure can accurately detect the sleep motion state, and the scheme has a low hardware cost and a wide range of applications.

Description

Technical Field

[0001] This disclosure relates to the field of mobile communication technology, and in particular to a method, apparatus, device, medium, and program product for detecting motion state during sleep. Background Technology

[0002] With the continuous advancement of technology, wearable devices have become an indispensable part of our daily lives. These devices not only track our exercise and health data but are also beginning to venture into the field of sleep monitoring. Among the many sleep monitoring products, they provide users with detailed analyses of their sleep cycles by analyzing physiological indicators such as heart rate, respiratory rate, and body movement, helping people better understand and improve their sleep quality. However, despite the achievements of these wearable devices in sleep monitoring, their ability to identify and monitor specific movement states during sleep remains limited. Summary of the Invention

[0003] To overcome the problems existing in the related technologies, the present disclosure provides a method, apparatus, device, medium and program product for detecting motion state during sleep, in order to solve the defects in the related technologies.

[0004] According to a first aspect of the present disclosure, a method for detecting motion states during sleep is provided, the method comprising:

[0005] In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired;

[0006] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0007] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0008] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0009] In some embodiments, obtaining the sleep motion state recognition result for a corresponding time window based on each signal segment combination and a pre-trained sleep motion state recognition model includes:

[0010] Feature extraction is performed on the combination of the signal segments to obtain signal features;

[0011] The signal features are input into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result.

[0012] In some embodiments, feature extraction from the combination of signal segments includes:

[0013] Extract the time-domain and / or frequency-domain features of the signal segment combination.

[0014] In some embodiments, obtaining the sleep motion state recognition result for a corresponding time window based on each signal segment combination and a pre-trained sleep motion state recognition model includes:

[0015] Obtain the type of limb entered in the user interface, where the limb type includes wrist or ankle;

[0016] Based on the combination of the signal segments and the pre-trained sleep motion state recognition model that matches the type of the limb, the sleep motion state recognition result is obtained.

[0017] In some embodiments, the method further includes:

[0018] The results of motion state recognition during sleep are processed to obtain the processing result;

[0019] The processing results are presented in the user interface.

[0020] In some embodiments, processing the sleep motion state recognition result to obtain a processing result includes:

[0021] The number of periodic limb movements detected within a unit of time is counted, where the unit of time is longer than the preset time.

[0022] Based on the statistical results of the quantity, a preset graphic is generated, which includes at least one of column chart, bar chart, pie chart, and histogram.

[0023] In some embodiments, the motion state recognition result during sleep is processed to obtain the following processing result:

[0024] Count the total number of sleep-related movements of the user during this sleep period;

[0025] The Periodic Limb Movement Index (PLMI) is determined based on the total number and the user's current sleep duration.

[0026] In some embodiments, the method further includes:

[0027] The PLMI is matched with a preset threshold range to obtain the matching result;

[0028] Based on the pre-built correspondence, find suggestions corresponding to the matching results;

[0029] In response to the detection that the preset suggestion presentation conditions are met, the suggestion is presented in the user interface.

[0030] In some embodiments, the sleep motion state recognition model is trained based on the following method:

[0031] Acquire sample pulse signals and sample acceleration signals from sample users;

[0032] The sample pulse signal and the sample acceleration signal are segmented based on a preset time window to obtain multiple sample pulse signal segments and multiple sample acceleration signal segments.

[0033] Combining sample pulse signal segments and sample acceleration signal segments within the same time window yields multiple sample signal segment combinations, where each sample signal segment combination corresponds to a time window;

[0034] Feature extraction is performed on the combination of the sample signal segments to obtain the sample signal features;

[0035] The sleep motion state recognition model is obtained by training the constructed model based on the features of multiple sample signals and their corresponding sleep motion state recognition labels.

[0036] In some embodiments, the sleep motion state identification label is determined based on the polysomnography (PSG) monitoring results of the sample users.

[0037] In some embodiments, the sleep motion state recognition model includes at least one of the following models:

[0038] Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) network model.

[0039] According to a second aspect of the present disclosure, a sleep motion state detection device is provided, the device comprising:

[0040] The signal acquisition module is used to acquire the user's pulse signal and acceleration signal in response to detecting that the user is in a sleep state;

[0041] The signal segmentation module is used to segment the pulse signal and the acceleration signal based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments.

[0042] The segment combination module is used to combine pulse signal segments and acceleration signal segments within the same time window to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0043] The result acquisition module is used to obtain the sleep motion state recognition result for the corresponding time window based on each combination of signal segments and the pre-trained sleep motion state recognition model.

[0044] In some embodiments, the result acquisition module includes:

[0045] A feature extraction unit is used to extract features from the combination of signal segments to obtain signal features;

[0046] The feature input unit is used to input the signal features into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result.

[0047] In some embodiments, the feature extraction unit is further configured to:

[0048] Extract the time-domain and / or frequency-domain features of the signal segment combination.

[0049] In some embodiments, the result acquisition module includes:

[0050] A type acquisition unit is used to acquire the type of a limb input in the user interface, wherein the type of the limb includes wrist or ankle;

[0051] The result acquisition unit is used to obtain the sleep motion state recognition result based on the combination of signal segments and a pre-trained sleep motion state recognition model that matches the type of the limb.

[0052] In some embodiments, the apparatus further includes: a result presentation module;

[0053] The result presentation module includes:

[0054] The result processing unit is used to process the sleep motion state recognition results to obtain the processing results;

[0055] The result presentation unit is used to present the processing results in the user interface.

[0056] In some embodiments, the result processing unit is further configured to:

[0057] The number of periodic limb movements detected within a unit of time is counted, where the unit of time is longer than the preset time.

[0058] Based on the statistical results of the quantity, a preset graphic is generated, which includes at least one of column chart, bar chart, pie chart, and histogram.

[0059] In some embodiments, the result processing unit is further configured to:

[0060] Count the total number of sleep-related movements of the user during this sleep period;

[0061] The Periodic Limb Movement Index (PLMI) is determined based on the total number and the user's current sleep duration.

[0062] In some embodiments, the result processing unit is further configured to:

[0063] The PLMI is matched with a preset threshold range to obtain the matching result;

[0064] Based on the pre-built correspondence, find suggestions corresponding to the matching results;

[0065] In response to the detection that the preset suggestion presentation conditions are met, the suggestion is presented in the user interface.

[0066] In some embodiments, the sleep motion state recognition model is trained based on the following method:

[0067] Acquire sample pulse signals and sample acceleration signals from sample users;

[0068] The sample pulse signal and the sample acceleration signal are segmented based on a preset time window to obtain multiple sample pulse signal segments and multiple sample acceleration signal segments.

[0069] Combining sample pulse signal segments and sample acceleration signal segments within the same time window yields multiple sample signal segment combinations, where each sample signal segment combination corresponds to a time window;

[0070] Feature extraction is performed on the combination of the sample signal segments to obtain the sample signal features;

[0071] The sleep motion state recognition model is obtained by training the constructed model based on the features of multiple sample signals and their corresponding sleep motion state recognition labels.

[0072] In some embodiments, the sleep motion state identification label is determined based on the polysomnography (PSG) monitoring results of the sample users.

[0073] In some embodiments, the sleep motion state recognition model includes at least one of the following models:

[0074] Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) network model.

[0075] According to a third aspect of the present disclosure, a sleep motion detection device is provided, the device comprising:

[0076] Pulse signal sensor, acceleration signal sensor, processor, and memory for storing computer programs;

[0077] The pulse signal sensor is configured to collect the user's pulse signal;

[0078] The acceleration signal sensor is configured to acquire the user's acceleration signal;

[0079] The processor is configured to perform the following when executing the computer program:

[0080] In response to detecting that the user is in a sleep state, the pulse signal and the acceleration signal are acquired;

[0081] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0082] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0083] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0084] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, the program being implemented when executed by a processor:

[0085] In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired;

[0086] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0087] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0088] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0089] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements:

[0090] In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired;

[0091] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0092] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0093] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0094] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0095] The disclosed method for detecting motion states during sleep acquires the user's pulse and acceleration signals in response to detecting that the user is asleep. The pulse and acceleration signals are then segmented into multiple pulse and acceleration signal segments based on a preset time window. These segments are then combined within the same time window to obtain multiple signal segment combinations. Based on each signal segment combination and a pre-trained motion state recognition model, the method obtains the motion state recognition result for the corresponding time window. This method can accurately detect motion states during sleep, is applicable to sleep monitoring in environments such as homes, has a wide range of applications, reduces hardware costs, and improves the user experience.

[0096] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0097] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0098] Figure 1A This is a block diagram illustrating a sleep motion detection device according to an exemplary embodiment of the present disclosure;

[0099] Figure 1B This is a schematic diagram illustrating a method for detecting motion states during sleep according to an exemplary embodiment of the present disclosure;

[0100] Figure 1C This is a schematic diagram illustrating a motion state detection scenario during sleep according to an exemplary embodiment of the present disclosure;

[0101] Figure 2 This is a flowchart illustrating how to obtain the sleep motion state recognition result corresponding to the preset duration, according to an exemplary embodiment of this disclosure;

[0102] Figure 3 This is a flowchart illustrating how to obtain the sleep motion state recognition result corresponding to the corresponding preset duration, according to yet another exemplary embodiment of this disclosure;

[0103] Figure 4 This is a flowchart illustrating how to post-process the results of motion state recognition during sleep according to an exemplary embodiment of the present disclosure;

[0104] Figure 5 This is a flowchart illustrating how to train a sleep motion state recognition model according to an exemplary embodiment of the present disclosure;

[0105] Figure 6 This is a schematic diagram of a sleep motion detection device according to an exemplary embodiment of the present disclosure;

[0106] Figure 7 This is a schematic diagram of a motion detection device during sleep, according to yet another exemplary embodiment of this disclosure. Detailed Implementation

[0107] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0108] With the continuous advancement of technology, wearable devices have become an indispensable part of our daily lives. These devices not only track our exercise and health data but are also venturing into the field of sleep monitoring. Among the many sleep monitoring products, they provide users with detailed analyses of their sleep cycles by examining physiological indicators such as heart rate, respiratory rate, and body movement, helping people better understand and improve their sleep quality. However, despite the achievements of these wearable devices in sleep monitoring, their ability to identify and monitor specific movement states during sleep remains limited. In particular, for Periodic Limb Movement Disorder (PLMD), the electronic products in related technologies cannot provide effective detection methods.

[0109] PLMD is a periodic behavioral disorder that occurs during sleep, characterized by periodic, stereotyped movements of the lower limbs during sleep. This disorder not only affects sleep quality but can also lead to daytime fatigue, poor concentration, and other consequences, severely impacting daily life and work efficiency. Therefore, recognizing PLMD is of great significance for early warning of individual sleep patterns.

[0110] In view of this, the present disclosure provides the following methods, devices, equipment, media and program products for detecting movement during sleep, in order to solve the above-mentioned drawbacks in the related technologies.

[0111] In an exemplary embodiment, this disclosure provides a sleep motion state detection device, which includes a pulse signal sensor, an acceleration signal sensor, a processor, and a memory for storing a computer program.

[0112] The pulse signal sensor is configured to acquire the user's pulse signal; the acceleration signal sensor is configured to acquire the user's acceleration signal; and the processor is configured to, when executing the computer program, implement the following: Figure 1B Methods for detecting movement during sleep.

[0113] For example, the pulse signal sensor described above may include any one of photoelectric pulse sensors (such as photoplethysmography PPG sensors), piezoelectric pulse sensors, ultrasonic sensors, and piezoresistive pulse sensors, and this embodiment does not limit it.

[0114] The aforementioned acceleration signal sensor, namely the ACC (acceleration) sensor, may include any one of piezoelectric acceleration sensors, capacitive acceleration sensors, and piezoresistive acceleration sensors; this embodiment does not limit the type of sensor.

[0115] For example, Figure 1AThis is a block diagram illustrating a sleep motion detection device according to an exemplary embodiment. For example, device 100 may be a smart bracelet, smartwatch, mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0116] Reference Figure 1A The device 100 may include one or more of the following components: processing component 102, memory 104, power supply component 106, multimedia component 108, audio component 110, input / output (I / O) interface 112, sensor component 114, and communication component 116.

[0117] Processing component 102 typically controls the overall operation of device 100, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 102 may include one or more processors 120 to execute instructions to implement the following when executing a computer program. Figure 1B The method for detecting movement during sleep. Furthermore, the processing component 102 may include one or more modules to facilitate interaction between the processing component 102 and other components. For example, the processing component 102 may include a multimedia module to facilitate interaction between the multimedia component 108 and the processing component 102.

[0118] Memory 104 is configured to store various types of data to support operation on device 100. Examples of this data include instructions for any application or method operating on device 100, contact data, phonebook data, messages, pictures, videos, etc., wherein the application is executed by the processor to implement the following Figure 1B A method for detecting movement during sleep. The memory 104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0119] Power supply component 106 provides power to various components of device 100. Power supply component 106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 100.

[0120] Multimedia component 108 includes a screen that provides an output interface between the device 100 and the user. In some embodiments, the screen may include a liquid crystal display panel and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 108 includes a front-facing camera and / or a rear-facing camera. When the device 100 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0121] Audio component 110 is configured to output and / or input audio signals. For example, audio component 110 includes a microphone (MIC) configured to receive external audio signals when device 100 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 104 or transmitted via communication component 116. In some embodiments, audio component 110 also includes a speaker for outputting audio signals.

[0122] I / O interface 112 provides an interface between processing component 102 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0123] Sensor assembly 114 includes one or more sensors for providing status assessments of various aspects of device 100. For example, sensor assembly 114 can detect the on / off state of device 100, the relative positioning of components such as the display panel and keypad of device 100, changes in the position of device 100 or a component of device 100, the presence or absence of user contact with device 100, the orientation or acceleration / deceleration of device 100, and temperature changes of device 100. Sensor assembly 114 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 114 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 114 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0124] Communication component 116 is configured to facilitate wired or wireless communication between device 100 and other devices. Device 100 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G or 5G, or combinations thereof. In one exemplary embodiment, communication component 116 receives broadcast signals or broadcast time-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 116 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0125] In an exemplary embodiment, device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the following Figure 1B The motion state detection method shown.

[0126] Figure 1B This is a schematic diagram illustrating a method for detecting movement during sleep according to an exemplary embodiment. Figure 1B As shown, the method for detecting movement during sleep includes the following steps S101-S104:

[0127] In step S101, in response to detecting that the user is in a sleep state, the pulse signal and the acceleration signal are acquired.

[0128] In this embodiment, the sleep motion state detection device (hereinafter referred to as the "device") can acquire the pulse signal and the acceleration signal in response to detecting that the user is in a sleep state.

[0129] For example, Figure 1C This is a schematic diagram illustrating a motion detection scenario during sleep according to an exemplary embodiment of this disclosure; as shown below. Figure 1CAs shown, the sleep motion detection device involved in this application scenario is a smartwatch 100. Users can wear the smartwatch 100 in daily life (at least before falling asleep), for example, on their wrist or ankle, and activate the "sleep motion detection" function. Based on this, the smartwatch 100 can detect whether the user is asleep using sleep detection algorithms in related technologies. Furthermore, when it detects that the user is asleep, it acquires pulse signals collected by a pulse signal sensor and acceleration signals collected by an acceleration signal sensor, and then performs subsequent detection steps. For example, it can acquire continuous PPG signals collected by a PPG sensor and continuous ACC signals collected by an ACC sensor.

[0130] In some embodiments, the pulse signal sensor and the acceleration signal sensor described above can acquire signals at the same frequency to facilitate subsequent alignment of the timestamps of the two signals. For example, a frequency of 25Hz can be used, but this embodiment is not limited to this.

[0131] It is worth noting that the above sleep detection algorithm can be selected based on business needs, and this embodiment does not limit it.

[0132] In step S102, the pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0133] In this embodiment, after acquiring the pulse signal and the acceleration signal, the device can segment the pulse signal and the acceleration signal based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments.

[0134] It is worth noting that the above preset duration can be set according to the typical duration of PLM in a single sleep state. For example, considering that the duration of a single PLM is usually 1 to 5 seconds, the above preset duration can be set to 5 seconds, etc.

[0135] For example, after acquiring continuous PPG and ACC signals, these two signals can be divided into multiple PPG signal segments and multiple ACC signal segments according to a preset duration (i.e., the 5 seconds set above).

[0136] In step S103, pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations.

[0137] In this embodiment, after obtaining multiple pulse signal segments and multiple acceleration signal segments, the pulse signal segments and acceleration signal segments within the same time window can be combined to obtain multiple signal segment combinations.

[0138] For example, after obtaining 15 seconds of continuous PPG and ACC signals, these two signals can be divided into signal segments of 1–5 seconds (first time window), 6–10 seconds (second time window), and 11–15 seconds (third time window) based on a 5-second time window. Then, pulse and acceleration signal segments within the same time window can be combined. For instance, the PPG and ACC signal segments from the first time window can be combined to obtain the signal segment combination corresponding to the first time window; similarly, the PPG and ACC signal segments from the second time window can be combined to obtain the signal segment combination corresponding to the second time window, and so on. These are just a few examples.

[0139] In step S104, based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0140] In this embodiment, after obtaining the signal segment combination corresponding to the above-mentioned preset duration, the sleep motion state recognition result for the corresponding time window can be obtained based on each signal segment combination and the pre-trained sleep motion state recognition model.

[0141] Taking the detection of specific motion states during sleep as PLMD as an example, after obtaining the signal segment combination corresponding to the first time window, the signal segment combination corresponding to the first time window can be preprocessed, and then the preprocessed data can be input into the above-mentioned motion state recognition model during sleep to obtain the PLMD recognition result corresponding to the first time window. This result is, for example, the probability of the existence of PLMD, or whether PLMD is detected.

[0142] In some embodiments, the PLMD identification result may also include the occurrence time of the PLMD (e.g., start time and / or end time), and / or the Periodic Limb Movement Index (PLMI). PLMI stands for Periodic Limb Movement Index, which is the ratio of the total number of periodic limb movement PLMS to the sleep duration (in hours, h).

[0143] The aforementioned sleep motion state recognition model may include at least one of the following models: Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) model, and this embodiment does not limit the specific model.

[0144] Taking the Transformer model as an example, the sleep motion state recognition model in this embodiment can be an Encoder-Only model within the Transformer class, such as BERT. This is because both PPG and ACC signals are sequential data, and the Transformer model has significant advantages in processing sequential data. It can convert PPG and ACC signals into a sequential data format suitable for Transformer model processing and can utilize self-attention mechanisms to capture long-term dependencies and time-series features in PPG and ACC signals, which is beneficial for recognizing complex patterns in PLMD. Furthermore, the Transformer model has advantages in end-to-end learning, which can significantly improve training efficiency and parallel computing capabilities.

[0145] In some embodiments, the above-described preprocessing method for combining signal segments may include feature extraction, for example, see the following: Figure 2 The embodiments shown will not be described in detail here.

[0146] Still with Figure 1C For example, after obtaining the user's PLMD test results, the smartwatch 100 can display test results such as PLMI on the screen, such as "PLMI=5". In addition, it can also provide a "View Suggestions" option for selection.

[0147] As described above, the sleep motion state detection method of this embodiment acquires pulse signals (including PPG signals) and acceleration signals (including acceleration signals) in response to detecting that the user is in a sleep state. The pulse signals and acceleration signals are then segmented based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments. The pulse signal segments and acceleration signal segments within the same time window are then combined to obtain multiple signal segment combinations. Based on each signal segment combination and a pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained. This method can accurately detect sleep motion states and is suitable for sleep monitoring in environments such as homes, with a wide range of applications.

[0148] Furthermore, in implementing the embodiments of this disclosure, the inventors considered that if the PLMD detection device adopts a split design, such as a structure including a main unit (to be worn on the user's wrist) and a secondary unit (to be placed on the side of the user's sternum, clavicle, or calf), then the main unit could include a data acquisition module, a data transmission module, and a data processing module, while the secondary unit could include a data acquisition module and a data transmission module. The data acquisition module is used to collect various relevant data during the user's sleep, the data transmission module is used to transmit the electromyography (EMG) data collected by the secondary unit to the main unit via Bluetooth, and the data processing module is used to process and analyze the acquired data to predict the probability of PLMD occurrence. However, this split design results in a complex device structure, which is inconvenient for users to carry and use, leads to higher overall power consumption, and unstable Bluetooth connections affect data transmission efficiency, thus impacting PLMD screening efficiency. Moreover, this design also results in higher hardware costs. Therefore, the device in this embodiment can be designed as a single, integrated structure, i.e., an integrated structure that contrasts with the split structure including a main unit and a secondary unit. For example, the device described above can have a single housing, within which various components such as sensors, memory, and processor can be housed. By designing the device as a single, integrated structure, the drawbacks of a split design, such as high power consumption, structural complexity, and reliance on Bluetooth for data transmission, can be avoided. This effectively reduces the overall power consumption of the device, improves ease of use, ensures the stability and efficiency of data transmission, and the pulse signal sensor and accelerometer are relatively inexpensive (compared to ECG or EMG sensors), thus saving on hardware costs.

[0149] Figure 2 This is a flowchart illustrating how to obtain the sleep motion state recognition result corresponding to the corresponding preset duration, according to an exemplary embodiment of the present disclosure. This embodiment is based on the above embodiment and takes how to obtain the sleep motion state recognition result corresponding to the corresponding preset duration as an example for illustrative explanation.

[0150] like Figure 2 As shown, the step S104 above, which involves obtaining the sleep motion state recognition result for the corresponding time window based on the combination of each signal segment and the pre-trained sleep motion state recognition model, may include the following steps S201-S202:

[0151] In step S201, feature extraction is performed on the combination of signal segments to obtain signal features.

[0152] In this embodiment, when pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, feature extraction can be performed on the signal segment combinations corresponding to the same time window to obtain signal features.

[0153] In some embodiments, when performing feature extraction on the combination of signal segments, time-domain features and / or frequency-domain features of the combination of signal segments can be extracted.

[0154] The time-domain features mentioned above may include at least one of the following: mean, standard deviation, maximum value, and minimum value.

[0155] The aforementioned frequency domain characteristics may include, but are not limited to, the power spectral density after Fourier transform.

[0156] In step S202, the signal features are input into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result.

[0157] In this embodiment, after feature extraction is performed on the combination of signal segments to obtain signal features, the signal features can be input into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result for the corresponding time window.

[0158] By employing a pre-trained sleep motion state recognition model to identify sleep motion states, compared to using simple probability calculation empirical formulas, it can more accurately capture the complex physiological and behavioral patterns of users during sleep, thereby more accurately handling individual differences and improving the accuracy of sleep motion state detection.

[0159] In other embodiments, after obtaining the signal features, the extracted features can be standardized, for example, to have a zero mean (i.e., the mean of the data is adjusted to 0) and a unit variance (i.e., the variance of the data is adjusted to 1), and then the standardized features are input into the model.

[0160] As described above, this embodiment extracts signal features from the combination of signal segments, and inputs these signal features into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result. This allows for accurate recognition of the sleep motion state within the corresponding time window based on each combination of signal segments and the pre-trained sleep motion state recognition model, meeting users' needs for daily detection and monitoring of sleep motion states. It provides users with accurate reference information and helps them improve their sleep conditions.

[0161] Figure 3This is a flowchart illustrating how to obtain the sleep motion state recognition result corresponding to the corresponding preset duration, according to another exemplary embodiment of this disclosure; this embodiment is based on the above embodiment and takes how to obtain the sleep motion state recognition result corresponding to the corresponding preset duration as an example for illustrative explanation.

[0162] like Figure 3 As shown, the step S104 above, which obtains the sleep motion state recognition result for the corresponding time window based on each signal segment combination and the pre-trained sleep motion state recognition model, may include the following steps S301-S302:

[0163] In step S301, the type of limb entered in the user interface is obtained.

[0164] In this embodiment, when the device performs detection, it can obtain the type of limb entered in the user interface.

[0165] For example, the device's user interface can include an interface for specifying the type of limb the user is wearing the device with. The device can then obtain the input limb type, such as wrist or ankle, through this interface.

[0166] In step S302, the sleep movement state recognition result is obtained based on the combination of signal segments and a pre-trained sleep movement state recognition model that matches the type of the limb.

[0167] In this embodiment, after obtaining the type of limb input in the user interface, the sleep motion state recognition result can be obtained based on the combination of signal segments and a pre-trained sleep motion state recognition model that matches the type of limb.

[0168] The aforementioned sleep motion state recognition model matching the type of the limb can be a sleep motion state recognition model pre-trained based on sample data from the limb. For example, the sleep motion state recognition model matching the wrist can be a model trained based on sample PPG and sample ACC signals collected from the wrist of the sample user (e.g., PPG and ACC signals collected when the device is worn on the wrist of the sample user); the sleep motion state recognition model matching the ankle can be a model trained based on sample PPG and sample ACC signals collected from the ankle of the sample user (e.g., PPG and ACC signals collected when the device is worn on the ankle of the sample user).

[0169] Taking the "wrist" as an example, after obtaining multiple signal segment combinations based on the PPG and ACC signals at the user's wrist, the signal segment combinations can be preprocessed (e.g., feature extraction and standardization), and then the preprocessed data can be input into the sleep motion state recognition model matched with the wrist to obtain the sleep motion state recognition result.

[0170] As described above, this embodiment obtains the type of limb input in the user interface, including the wrist or ankle, and obtains the sleep motion state recognition result based on the combination of signal segments and a pre-trained sleep motion state recognition model that matches the type of limb. Since the motion state recognition model based on wrist data and the motion state recognition model based on ankle data are pre-trained, the appropriate model can be selected for prediction based on the data source when performing motion state recognition in the future. This not only improves the accuracy and efficiency of detection, but also enhances the adaptability and robustness of the model, thereby providing users with more reliable and efficient support for detecting sleep motion state in daily life.

[0171] Figure 4 This is a flowchart illustrating how to post-process the results of motion state recognition during sleep according to an exemplary embodiment of the present disclosure; this embodiment is based on the above embodiment and takes how to post-process the results of motion state recognition during sleep as an example for illustrative explanation.

[0172] like Figure 4 As shown, the sleep motion state detection method of this embodiment may further include post-processing of the sleep motion state recognition results based on the following steps S401-S402:

[0173] In step S401, the results of the motion state recognition during sleep are processed to obtain the processing result.

[0174] In this embodiment, after obtaining the sleep motion state recognition result for the corresponding time window based on each signal segment combination and the pre-trained sleep motion state recognition model, the sleep motion state recognition result can be processed to obtain the processing result.

[0175] The methods for processing the results of motion state recognition during sleep can include graphical processing and / or calculation of relevant indicators.

[0176] In step S402, the processing result is presented in the user interface.

[0177] In this embodiment, after processing the motion state recognition results during sleep and obtaining the processing results, the processing results can be presented in the user interface.

[0178] Taking the detection of specific movement states during sleep as an example, when performing graphical processing, the number of periodic limb movements (PLMs) detected within a unit of time can be counted, and then a preset graph can be generated based on the statistical results of the count. The preset graph includes at least one of bar charts, pie charts, and histograms. It is understood that the aforementioned unit of time is longer than the preset time.

[0179] For example, the unit duration can be a natural hour (e.g., UTC-0 time zone, 1:00-2:00, etc.), and the preset duration can be 5 seconds. After obtaining the PLMD recognition results corresponding to each 5-second interval, the number of PLMs detected within each hour can be counted, and then a preset graph can be generated based on the statistical results of the number. The preset graph includes at least one of bar charts, pie charts, and histograms. Based on this, the preset graph can be displayed on the user interface to help users better understand the detection results and improve the user experience.

[0180] Taking the calculation of relevant indicators as an example, the total number of movement states during the user's sleep (e.g., the user's sleep throughout the night) can be counted, and then the Periodic Limb Movement Index (PLMI) can be determined based on the total number and the duration of the user's sleep. PLMI stands for Periodic Limb Movement Index, which is the ratio of the total number of periodic limb movements (PLMS) to the sleep duration (in hours, hours).

[0181] For example, assuming 120 PLMs are recorded during a user's sleep throughout the night, and the total sleep duration is 7.5 hours, the PLMI can be calculated based on the following formula (4-1):

[0182] PLMI=120 / 7.5=16; (4-1)

[0183] The calculated PLMI can then be displayed on the user interface for the user to view.

[0184] Based on this, the PLMI can be matched with a preset threshold range to obtain a matching result. Based on the pre-built correspondence, suggestions corresponding to the matching result can be found. Then, in response to the detection that the preset suggestion presentation conditions are met, the suggestions can be presented in the user interface.

[0185] For example, the correspondence between the different pre-built matching results and suggestions can be shown in Table 1 below:

[0186] Table 1

[0187]

[0188] For example, when a user's PLMI is 25 times / hour, their matching result can be determined as "mild PLMD". Then, according to Table 1 above, the corresponding suggestions are "maintain a regular schedule, ensure sufficient sleep, practice deep breathing or meditation before bed, read or listen to light music".

[0189] It should be noted that the contents shown in Table 1 above are for illustrative purposes only and can be adjusted according to the actual scenario or business needs when applied. This embodiment does not limit this.

[0190] Based on this, the suggestions can also be presented in response to the fulfillment of preset suggestion presentation conditions.

[0191] For example, once the matching result of the above PLMI is determined, the suggestion can be directly presented in the user interface (e.g., the user interface of a device's sleep motion detection app), or it can be displayed in the "Query Suggestions" function option (e.g., ...). Figure 1C As shown in the figure, when the user clicks on the "Query Suggestion" function option, the system determines the current presentation conditions that meet the suggestion, and then presents the suggestion.

[0192] As described above, this embodiment processes the motion state recognition results during sleep to obtain processing results, and presents the processing results in the user interface. This can improve the understandability of the PLMD recognition results, thereby enhancing the functionality of the device and improving the user experience.

[0193] Figure 5 This is a flowchart illustrating how to train a sleep motion state recognition model according to an exemplary embodiment of the present disclosure. This embodiment is based on the above embodiment and uses the training of a sleep motion state recognition model as an example for illustrative explanation.

[0194] like Figure 5 As shown, the sleep motion state detection method in this embodiment may further include training a sleep motion state recognition model based on the following steps S501-S505.

[0195] In this embodiment, when training the above-mentioned sleep motion state recognition model, the user's device wearing location, such as on the wrist or ankle, can be obtained.

[0196] The following explanation will still use PLMD, which detects specific movement states during sleep, as an example.

[0197] PLMD is characterized by periodic, stereotyped limb movements during sleep. These movements primarily affect the lower limbs, but may also involve the upper limbs. The movement patterns in PLMD differ somewhat between the wrists (upper limbs) and ankles (lower limbs), mainly in the frequency, amplitude, and pattern of the movements:

[0198] 1) Ankle (lower limb) symptoms:

[0199] Frequency: Periodic movements of the lower limbs are usually more frequent, especially in the second stage of non-rapid eye movement (NREM) sleep.

[0200] Range of motion: The range of motion of the lower limbs is usually large, which may include flexion and extension of the toes, dorsiflexion and plantarflexion of the ankle, and flexion and extension of the knee joint.

[0201] Pattern: The periodic movements of the lower limbs are usually characterized by an "extension-flexion" pattern, that is, dorsiflexion of the foot accompanied by knee extension, followed by plantar flexion of the foot accompanied by knee flexion.

[0202] 2) Wrist (upper limb) manifestations:

[0203] Frequency: The frequency of periodic movements of the upper limbs is usually lower than that of the lower limbs, and they are not as regular as the movements of the lower limbs.

[0204] Range of motion: The range of motion of the upper limbs is usually small and may include finger flexion and extension, wrist dorsiflexion and palmar flexion.

[0205] Pattern: The periodic movement patterns of the upper limbs are not as typical as those of the lower limbs and may include slight extension and flexion of the arms, as well as flexion and extension of the wrists and fingers.

[0206] 3) Other differences:

[0207] Impact: Periodic movements of the lower limbs may have a greater impact on sleep because they are usually accompanied by greater amplitude and more frequent occurrence.

[0208] Diagnosis: In diagnosing PLMD, polysomnography (PSG) monitoring typically records lower limb movements as they are more diagnostically valuable. Upper limb movements can also be recorded to provide a more comprehensive sleep assessment.

[0209] 4) Commonalities:

[0210] Whether in the upper or lower limbs, the periodic movements of PLMD typically occur during specific stages of sleep, especially the second stage of NREM sleep.

[0211] These movements typically decrease or disappear during rapid eye movement (REM) sleep.

[0212] By acquiring the user's device wearing location and training sleep motion state recognition models based on wrist data and ankle data respectively, it is possible to select the appropriate model for prediction based on the data source during subsequent PLMD detection. This not only improves the accuracy and efficiency of detection but also enhances the adaptability and robustness of the model, thus providing users with more reliable and efficient support for daily PLMD detection and monitoring.

[0213] In step S501, the sample pulse signal and sample acceleration signal of the sample user are acquired.

[0214] In this embodiment, sample pulse signals and sample acceleration signals from a sample user can be acquired by the pulse signal sensor and acceleration signal sensor in the device, respectively. For example, sample PPG signals and sample ACC signals can be acquired. The sample PPG signals and sample ACC signals can be acquired at the same frequency, such as 25Hz, to facilitate subsequent timestamp alignment.

[0215] Simultaneously, polysomnography (PSG) can be used to monitor the number and distribution of periodic limb movements (PLMs) of the aforementioned sample users, serving as a gold standard record. This gold standard record can be used to determine the training labels for the model. For example, PSG detection can determine how many PLMs occurred in each time period. For instance, 3 PLMs occurred between 1-2 AM, 0 between 2-3 AM, and 10 between 3-4 AM, etc.

[0216] If an individual experiences 120 periodic limb movement events (PLMs) during a night's sleep, and the total sleep duration is 7.5 hours, the Periodic Limb Movement Index (PLMI) can be calculated as 120 / 7.5 = 16 (times / hour). Generally, a PLMI ≥ 15 is considered a diagnostic criterion for PLMD, while a PLMI ≥ 30 may indicate more severe PLMD.

[0217] In some embodiments, the start and end times of each PLM based on PSG detection can be recorded. It is worth noting that the duration of most PLMs is between 1 and 5 seconds.

[0218] Based on this, once the wrist and ankle data of the sample users are obtained, data preprocessing can be performed according to the following steps, which will then allow for the training of wrist and ankle models (the distinction between wrist and ankle will not be made hereafter). The data preprocessing process includes signal segmentation, combination, feature extraction, feature standardization, and label confirmation (i.e., steps S502-S504).

[0219] In step S502, the sample pulse signal and the sample acceleration signal are segmented based on a preset time window to obtain multiple sample pulse signal segments and multiple sample acceleration signal segments.

[0220] For example, after acquiring continuous sample PPG signals and sample ACC signals, these two signals can be divided into multiple PPG signal segments and multiple ACC signal segments according to a preset duration (e.g., 5 seconds). As mentioned earlier, the duration of a single PLM is usually 1 to 5 seconds, so the preset duration can be set to 5 seconds.

[0221] In step S503, sample pulse signal segments and sample acceleration signal segments within the same time window are combined to obtain multiple sample signal segment combinations.

[0222] Each combination of sample signal segments corresponds to a time window.

[0223] For example, after obtaining 15 seconds of sample PPG and sample ACC signals, these two signals can be divided into sample signal segments of 1–5 seconds (first time window), 6–10 seconds (second time window), and 11–15 seconds (third time window), respectively. Then, sample pulse signal segments and sample acceleration signal segments within the same time window can be combined. For instance, the sample PPG signal segment and sample ACC signal segment from the first time window can be combined to obtain the sample signal segment combination corresponding to the first time window; and the sample PPG signal segment and sample ACC signal segment from the second time window can be combined to obtain the sample signal segment combination corresponding to the second time window, and so on. These are not all listed here.

[0224] In step S504, feature extraction is performed on the combination of sample signal segments to obtain sample signal features.

[0225] In this embodiment, after combining sample pulse signal segments and sample acceleration signal segments within the same time window to obtain the sample signal segment combination corresponding to the preset duration, feature extraction can be performed on the sample signal segment combination to obtain sample signal features.

[0226] In some embodiments, when performing feature extraction on the combination of sample signal segments, time-domain features and / or frequency-domain features of the combination of sample signal segments can be extracted.

[0227] The time-domain features mentioned above may include at least one of the following: mean, standard deviation, maximum value, and minimum value.

[0228] The aforementioned frequency domain characteristics may include, but are not limited to, the power spectral density after Fourier transform.

[0229] In other embodiments, after obtaining the sample signal features, the extracted sample signal features can be standardized, for example, to have a zero mean (i.e., the mean of the data is adjusted to 0) and a unit variance (i.e., the variance of the data is adjusted to 1), in order to improve the generalization ability of the model.

[0230] It is worth noting that zero mean and unit variance processing can significantly improve the generalization ability of the model by eliminating the bias between features, unifying the feature scale, and accelerating the optimization algorithm, so that the model can also perform well on unseen data.

[0231] In this embodiment, when determining the training label (i.e., the sleep motion state recognition label), if more than 2.5 seconds of PLM occur within any 5-second sample signal segment combination, then that 5-second sample signal segment combination can be marked as a positive label (e.g., PLM occurred); otherwise, it is marked as negative (e.g., PLM did not occur). The purpose of this processing is to avoid the problem of duplicate counting caused by the same PLM being segmented into different time segments, which can improve the accuracy of the training label, and thus improve the accuracy of model training.

[0232] In step S505, the constructed model to be trained is trained based on multiple sample signal features and their corresponding sleep motion state recognition labels as training sample data to obtain the sleep motion state recognition model.

[0233] In this embodiment, after feature extraction is performed on the combination of sample signal segments to obtain sample signal features, the constructed model to be trained can be trained based on multiple sample signal features and their corresponding sleep motion state recognition labels as training sample data to obtain the sleep motion state recognition model.

[0234] For example, the model to be trained constructed above may include at least one of the following models:

[0235] Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) model are not limited to this embodiment.

[0236] Taking the Transformer model as an example, the model to be trained can be an Encoder-Only model within the Transformer class, such as BERT. Since PPG and ACC signals are sequential data, and the Transformer model has significant advantages in processing sequential data, it can convert PPG and ACC signals into a suitable sequential data format for Transformer processing. Furthermore, it can utilize self-attention mechanisms to capture long-term dependencies and time-series features in PPG and ACC signals, which is beneficial for identifying complex patterns in PLMD. Moreover, the Transformer model has advantages in end-to-end learning, significantly improving training efficiency and parallel computing capabilities.

[0237] Taking the Encoder-Only model as an example, its training process is summarized as follows:

[0238] 1) Embedding Layer:

[0239] The input sample signal feature sequence is converted into a word vector representation. Each word vector is mapped to a high-dimensional space through an embedding matrix.

[0240] Suppose the input sample signal feature sequence is x = {x1, x2, ..., x...} n}, where x n It is the index of the nth word vector, and the embedding matrix W emb Mapping the index to the word vector E is shown in equation (5-1):

[0241] E=W emb (x); (5-1)

[0242] 2) Positional Encoding:

[0243] To preserve the order of the sequence, positional encoding needs to be added to each word vector.

[0244] Positional encoding can be generated using sine and cosine functions and added to the embedding vector, as shown in equations (5-2) and (5-3) below:

[0245]

[0246]

[0247] In the above formula, pos is the position, i is the dimension index, and d is the dimension index. model This refers to the dimension of the model.

[0248] 3) Transformer Encoder:

[0249] In this embodiment, the Transformer encoder can be composed of multiple identical layers stacked together, each layer containing a multi-head self-attention mechanism and a feedforward neural network (FFN). The input to each layer is the output of the previous layer or the initial output (for the first layer).

[0250] The self-attention mechanism is used to calculate the attention score of each word to other words and generate new word representations.

[0251] For the multi-head self-attention mechanism of layer l, the input is H (l) The output is H (l+1) The formula for the self-attention mechanism can be shown in equation (5-3):

[0252]

[0253] In the above formula, Q, K, and V are the query, key, and value matrices, respectively, and d k is the dimension of the key vector.

[0254] 4) Output layer

[0255] The output of the final Transformer encoder layer is pooled (e.g., averaged or maximized) to obtain a fixed-length vector representation. This vector can be classified by a fully connected layer, as shown in equation (5-4):

[0256] y = W o h last +b o (5-4)

[0257] Among them, h last W is the pooled vector. o and b o These are the weights and biases of the fully connected layer, respectively.

[0258] The final output can be converted into a probability distribution using the softmax function. As shown in equation (5-5):

[0259]

[0260] 5) Loss Function

[0261] In this embodiment, the cross-entropy loss function can be used to optimize the model to correctly classify whether PLMs events occur within a preset duration of 5 seconds. For example, the cross-entropy loss function can be shown in equation (5-6):

[0262]

[0263] In the above formula, N is the sample size, and y i For real labels, To predict probabilities.

[0264] 6) Training strategies

[0265] The gradient can be calculated using the backpropagation algorithm, and the model parameters can be updated using an optimizer (such as the Adam optimizer) to minimize the loss function mentioned above.

[0266] Based on this, algorithms such as early stopping can be used to avoid model overfitting.

[0267] 7) Model Evaluation

[0268] In this embodiment, metrics such as accuracy, recall, and F1 score can be used to evaluate model performance.

[0269] Accuracy refers to the proportion of samples correctly predicted by the model out of the total number of samples. It reflects the consistency between the model's prediction results and the actual labels and is a basic indicator for evaluating the overall performance of the model.

[0270] Recall is the proportion of positive samples correctly predicted by the model out of all actual positive samples. It measures the model's ability to find all relevant samples. A higher recall means that the model can identify positive samples better and reduce false negatives.

[0271] The F1 score is the harmonic mean of precision and recall, which balances the relationship between precision and recall, allowing the model to maintain high sensitivity to positive samples while pursuing high precision.

[0272] Through continuous iterative optimization of the above process, the model can learn effective feature representations, thereby enabling it to perform well in PLMD detection tasks.

[0273] In summary, the sleep motion state detection device and method proposed in this disclosure can be used to detect and analyze sleep motion states (e.g., PLMD). By utilizing pulse and acceleration signals, and combining advanced Transformer models for data analysis and prediction, it can perform independent detection, making it suitable for home sleep monitoring (i.e., users can detect and monitor PLMD at home) and also for use in mobile environments such as travel (e.g., convenient for business people and travelers to monitor limb movements during sleep). Compared to detection devices using ECG or EMG sensors, it reduces hardware costs. Compared to devices with a split structure, it does not require an additional Bluetooth connection, saving power and enhancing data transmission stability. Compared to devices using simple probability calculation empirical formulas, it can improve the accuracy of sleep motion state detection based on models, thereby improving the user experience.

[0274] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 104 including instructions, which can be executed by the processor 120 of the device 100 to implement the following method for detecting motion states during sleep:

[0275] In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired;

[0276] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0277] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0278] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0279] For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0280] In an exemplary embodiment, a computer program product including instructions is also provided, which can be executed by the processor 120 of the device 100 to implement the following method for detecting motion states during sleep:

[0281] In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired;

[0282] The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window.

[0283] Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window;

[0284] Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

[0285] Regarding the sleep motion state detection method in the above embodiments, the specific manner in which each step is executed has been described in detail in the embodiments of the sleep motion state detection device, and will not be elaborated here.

[0286] Figure 6 This is a block diagram illustrating a multi-party calling device according to an exemplary embodiment of the present disclosure; the device of this embodiment can be configured in an electronic device, such as a smart bracelet, smartwatch, mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc. Figure 6 As shown, the device may include: a signal acquisition module 110, a signal segmentation module 120, a segment combination module 130, and a result acquisition module 140, wherein:

[0287] The signal acquisition module 110 is used to acquire the user's pulse signal and acceleration signal in response to detecting that the user is in a sleep state;

[0288] The signal segmentation module 120 is used to segment the pulse signal and the acceleration signal based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments.

[0289] The segment combination module 130 is used to combine pulse signal segments and acceleration signal segments within the same time window to obtain multiple signal segment combinations, wherein each signal segment combination corresponds to a time window;

[0290] The result acquisition module 140 is used to obtain the sleep motion state recognition result for the corresponding time window based on each signal segment combination and the pre-trained sleep motion state recognition model.

[0291] As described above, the sleep motion state detection device of this embodiment acquires pulse signals (including PPG signals) and acceleration signals (including acceleration signals) in response to detecting that the user is in a sleep state. The pulse signals and acceleration signals are segmented based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments. Then, the pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations. Based on each signal segment combination and a pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained. This device can accurately detect sleep motion state and is suitable for sleep monitoring in environments such as homes, with a wide range of applications.

[0292] Figure 7 This is a schematic diagram of a sleep motion detection device according to another exemplary embodiment of the present disclosure. The device of this embodiment can be configured in an electronic device, such as a smart bracelet, smartwatch, mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc. The signal acquisition module 210, signal segmentation module 220, segment combination module 230, and result acquisition module 240 are as described above. Figure 6 The signal acquisition module 110, signal segmentation module 120, segment combination module 130 and result acquisition module 140 in the illustrated embodiment have the same function, and will not be described in detail here.

[0293] In some embodiments, the result acquisition module 240 may include:

[0294] Feature extraction unit 241 is used to extract features from the combination of signal segments to obtain signal features;

[0295] The feature input unit 242 is used to input the signal features into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result.

[0296] In some embodiments, the feature extraction unit 241 can also be used for:

[0297] Extract the time-domain and / or frequency-domain features of the signal segment combination.

[0298] In other embodiments, the result acquisition module 240 may include:

[0299] The type acquisition unit 243 is used to acquire the type of limb input in the user interface, wherein the type of limb includes wrist or ankle;

[0300] The result acquisition unit 244 is used to obtain the sleep motion state recognition result based on the combination of signal segments and a pre-trained sleep motion state recognition model that matches the type of the limb.

[0301] In some embodiments, the above-described apparatus may further include: a result presentation module 240;

[0302] The result presentation module 240 may include:

[0303] The result processing unit 251 is used to process the sleep motion state recognition result to obtain the processing result;

[0304] The result presentation unit 252 is used to present the processing result in the user interface.

[0305] In some embodiments, the result processing unit 251 may also be used for:

[0306] The number of periodic limb movements detected within a unit of time is counted, where the unit of time is longer than the preset time.

[0307] Based on the statistical results of the quantity, a preset graphic is generated, which includes at least one of column chart, bar chart, pie chart, and histogram.

[0308] In some embodiments, the result processing unit 251 may also be used for:

[0309] Count the total number of sleep-related movements of the user during this sleep period;

[0310] The Periodic Limb Movement Index (PLMI) is determined based on the total number and the user's current sleep duration.

[0311] In some embodiments, the result processing unit 251 may also be used for:

[0312] The PLMI is matched with a preset threshold range to obtain the matching result;

[0313] Based on the pre-built correspondence, find suggestions corresponding to the matching results;

[0314] In response to the detection that the preset suggestion presentation conditions are met, the suggestion is presented in the user interface.

[0315] In some embodiments, the above-described sleep motion state recognition model can be trained in the following manner:

[0316] Acquire sample pulse signals and sample acceleration signals from sample users;

[0317] The sample pulse signal and the sample acceleration signal are segmented based on a preset time window to obtain multiple sample pulse signal segments and multiple sample acceleration signal segments.

[0318] Combining sample pulse signal segments and sample acceleration signal segments within the same time window yields multiple sample signal segment combinations, where each sample signal segment combination corresponds to a time window;

[0319] Feature extraction is performed on the combination of the sample signal segments to obtain the sample signal features;

[0320] The sleep motion state recognition model is obtained by training the constructed model based on the features of multiple sample signals and their corresponding sleep motion state recognition labels.

[0321] In some embodiments, the aforementioned sleep motion state identification label can be determined based on the polysomnography (PSG) monitoring results of the sample users.

[0322] In some embodiments, the sleep motion state recognition model may include at least one of the following models:

[0323] Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) network model.

[0324] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0325] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the foregoing claims.

[0326] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for detecting motor states during sleep, characterized in that, The method includes: In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired; The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window. Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window; Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

2. The method according to claim 1, characterized in that, The method for obtaining sleep motion state recognition results for corresponding time windows based on each signal segment combination and a pre-trained sleep motion state recognition model includes: Feature extraction is performed on the combination of the signal segments to obtain signal features; The signal features are input into a pre-trained sleep motion state recognition model to obtain the sleep motion state recognition result.

3. The method according to claim 2, characterized in that, The feature extraction of the signal segment combination includes: Extract the time-domain and / or frequency-domain features of the signal segment combination.

4. The method according to claim 1, wherein obtaining the sleep motion state recognition result for the corresponding time window based on each signal segment combination and a pre-trained sleep motion state recognition model includes: Obtain the type of limb entered in the user interface, where the limb type includes wrist or ankle; Based on the combination of the signal segments and the pre-trained sleep motion state recognition model that matches the type of the limb, the sleep motion state recognition result is obtained.

5. The method according to claim 1, characterized in that, The method further includes: The results of motion state recognition during sleep are processed to obtain the processing result; The processing results are presented in the user interface.

6. The method according to claim 5, characterized in that, The process of processing the motion state recognition results during sleep to obtain the processing result includes: The number of periodic limb movements detected within a unit of time is counted, where the unit of time is longer than the preset time. Based on the statistical results of the quantity, a preset graphic is generated, which includes at least one of column chart, bar chart, pie chart, and histogram.

7. The method according to claim 5, characterized in that, The results of the motion state recognition during sleep are processed to obtain the following processing result: Count the total number of sleep-related movements of the user during this sleep period; The Periodic Limb Movement Index (PLMI) is determined based on the total number and the user's current sleep duration.

8. The method according to claim 7, characterized in that, The method further includes: The PLMI is matched with a preset threshold range to obtain the matching result; Based on the pre-built correspondence, find suggestions corresponding to the matching results; In response to the detection that the preset suggestion presentation conditions are met, the suggestion is presented in the user interface.

9. The method according to claim 1, characterized in that, The sleep motion state recognition model is trained based on the following method: Acquire sample pulse signals and sample acceleration signals from sample users; The sample pulse signal and the sample acceleration signal are segmented based on a preset time window to obtain multiple sample pulse signal segments and multiple sample acceleration signal segments. Combining sample pulse signal segments and sample acceleration signal segments within the same time window yields multiple sample signal segment combinations, where each sample signal segment combination corresponds to a time window; Feature extraction is performed on the combination of the sample signal segments to obtain the sample signal features; The sleep motion state recognition model is obtained by training the constructed model based on the features of multiple sample signals and their corresponding sleep motion state recognition labels.

10. The method according to claim 9, characterized in that, The sleep motion state identification label is determined based on the polysomnography (PSG) monitoring results of the sample users.

11. The method according to claim 1, characterized in that, The sleep motion state recognition model includes at least one of the following models: Transformer model, Convolutional Neural Network (CNN) model, Recurrent Neural Network (RNN) model, Long Short-Term Memory (LSTM) network model.

12. A device for detecting movement during sleep, characterized in that, The device includes: The signal acquisition module is used to acquire the user's pulse signal and acceleration signal in response to detecting that the user is in a sleep state; The signal segmentation module is used to segment the pulse signal and the acceleration signal based on a preset time window to obtain multiple pulse signal segments and multiple acceleration signal segments. The segment combination module is used to combine pulse signal segments and acceleration signal segments within the same time window to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window; The result acquisition module is used to obtain the sleep motion state recognition result for the corresponding time window based on each combination of signal segments and the pre-trained sleep motion state recognition model.

13. A device for detecting movement during sleep, characterized in that, The device includes: Pulse signal sensor, acceleration signal sensor, processor, and memory for storing computer programs; The pulse signal sensor is configured to collect the user's pulse signal; The acceleration signal sensor is configured to acquire the user's acceleration signal; The processor is configured to perform the following when executing the computer program: In response to detecting that the user is in a sleep state, the pulse signal and the acceleration signal are acquired; The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window. Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window; Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the following is achieved: In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired; The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window. Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window; Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

15. A computer program product, comprising a computer program, characterized in that, The computer program is executed by the processor to achieve the following: In response to detecting that the user is in a sleep state, the user's pulse signal and acceleration signal are acquired; The pulse signal and the acceleration signal are divided into multiple pulse signal segments and multiple acceleration signal segments based on a preset time window. Pulse signal segments and acceleration signal segments within the same time window are combined to obtain multiple signal segment combinations, where each signal segment combination corresponds to a time window; Based on each combination of signal segments and the pre-trained sleep motion state recognition model, the sleep motion state recognition result for the corresponding time window is obtained.

Images