Sleep detection method and system based on time sequence continuity optimization

Abstract

The invention belongs to the technical field of sleep detection, and particularly discloses a sleep detection method and system based on time sequence continuity optimization, and the method comprises the following steps: collecting original wrist strap accelerometer data through a triaxial accelerometer, and carrying out the preprocessing, thereby obtaining an input tensor; inputting the input tensor into a feature extraction module, and extracting local features and global features of the input tensor to obtain an advanced feature tensor; inputting the high-level feature tensor into a feature reconstruction module, and performing up-sampling and feature jump connection on the high-level feature tensor by adopting a linear interpolation reconstruction strategy to obtain a final data tensor; and carrying out secondary fine adjustment and time sequence smooth correction on the data tensor output by the feature reconstruction module to form complete sleep state prediction of one day per minute and corresponding confidence score output. By adopting the technical scheme, the residual unit is improved by utilizing the multi-scale cavity convolution, and the prediction result of the sleep event is more accurate through a bidirectional time sequence information processing mechanism.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of sleep detection, and relates to a sleep detection method and system based on time sequence continuity optimization. BACKGROUND

[0002] In the modern fast-paced life, sleep problems (such as insomnia and sleep apnea) are becoming more and more common, and there is an increasing demand for effective sleep state detection methods. Existing sleep detection algorithms include body motion-based sleep monitoring methods, machine learning-based sleep monitoring methods, and deep learning-based sleep monitoring methods.

[0003] Body motion-based sleep monitoring method: the accuracy of body motion sleep detection can reach 75% to 95%, meeting the basic sleep evaluation requirements. This method is simple in algorithm, easy to implement, fast in data processing, low in hardware requirements, and suitable for real-time monitoring, but the precision is not as good as polysomnography (PSG). It is commonly used in portable wearable devices (such as bracelets, watches.

[0004] Machine learning-based sleep monitoring method: a device using a pulse oximeter and an accelerometer to monitor sleep stages is worn on the wrist to collect heart rate and acceleration signals, features are extracted through fast Fourier transform, and support vector machine (SVM) classification is used, and the results show high efficiency and accuracy.

[0005] Deep learning-based sleep monitoring method: deep learning technology has shown great potential and effectiveness in sleep analysis, gradually surpassing traditional machine learning models. In particular, long short-term memory (LSTM) and bidirectional LSTM (Bi-LSTM) can capture time dependencies and process inputs of various lengths.

[0006] Although existing body motion detection, machine learning and deep learning methods have improved the portability and automation of sleep monitoring to some extent, there are still common problems such as dependence on human-designed feature extraction, insufficient dynamic time series modeling capability, limited capture of local details, large demand for training data, and high model complexity, and an innovative method that is more efficient, more accurate and suitable for daily environment is urgently needed to effectively improve the accuracy and practicality of sleep quality monitoring. SUMMARY

[0007] The purpose of the present application is to provide a sleep detection method and system based on time sequence continuity optimization, which effectively improves the accuracy of sleep quality monitoring.

[0008] In order to achieve the above purpose, the basic scheme of the present application is: a sleep detection method based on time sequence continuity optimization, comprising the following steps: Collecting original wristband accelerometer data through a three-axis accelerometer and preprocessing to obtain an input tensor; The input tensor is input into the feature extraction module, fine-grained convolution operation is adopted, local features and global features of the input tensor are extracted, and a high-level feature tensor is obtained; The high-level feature tensor is input into the feature reconstruction module, a linear interpolation reconstruction strategy is adopted, the high-level feature tensor is up-sampled and feature jump connection is performed, and a final data tensor is obtained; The data tensor output by the feature reconstruction module is subjected to secondary fine-tuning and timing smoothing correction to form a complete one-day per minute sleep state prediction and corresponding confidence score output.

[0009] The working principle and beneficial effects of the basic scheme are that in the data preprocessing stage, the original data is collected by the three-axis accelerometer and preprocessed for subsequent use. Based on the feature extraction module, the details and global features of the three-axis accelerometer under different time windows are extracted to improve the accuracy of sleep state prediction.

[0010] The feature reconstruction module is used for feature reconstruction, which aims to gradually reconstruct and fine-grained recover the high-level features compressed by the feature extraction module. The timing prediction smoothing method is used to perform secondary optimization and smoothing processing on the preliminary prediction results through the bidirectional long short-term memory network structure, significantly reducing the short-term prediction fluctuation and misjudgment phenomenon, more comprehensively and efficiently capturing the timing continuity of sleep state prediction, and improving the overall prediction performance and actual application stability.

[0011] Further, the method for pre-processing the original data to obtain the input tensor is: The standard deviations of the z angle and ENMO activity in the original data are calculated respectively, and the natural logarithms of the two are taken to obtain log_anglez and log_enmo; The valid_flag label corresponding to the time step is introduced, and when duplicate records or invalid data are detected, the valid_flag of the corresponding time step is set to 0, and the normal record is set to 1; Therefore, each time step contains three-dimensional features log_anglez, log_enmo and valid_flag, which together constitute three-channel input data.

[0012] For the z angle signal in the original feature, the standard deviation is calculated and the natural logarithm is taken to enhance the perception ability of small angle changes. For the ENMO activity feature, the logarithmic transformation is also taken to compress the extreme value range and highlight the medium and low intensity motion features.

[0013] Further, there is also a cross-day data splicing strategy: for each target day, the feature sequence of the last 3 hours of the previous day, the whole day data, and the first 3 hours of the next day is extracted, and spliced to form a single 3x21600 feature matrix sample; The batch size is set to 16, and an input tensor with a shape of (16, 3, 21600) is formed; The input tensor is input into the preliminary feature extraction unit, and the number of feature channels is expanded from the original 3 to 72, to obtain an input tensor with a shape of (16, 72, 21600).

[0014] The cross-day data splicing strategy can improve the continuity recognition ability of sleep state transition and more fully integrate the context information of the before and after time segments.

[0015] Further, the input tensor is input into the feature extraction module, and a fine-grained convolution operation is used to extract local features and global features of the input tensor. Specifically, the feature extraction module includes a standard feature extraction unit and a multi-scale hollow residual enhancement unit. The standard feature extraction unit includes a one-dimensional convolution layer, a batch normalization layer, and an activation function layer. The standard feature extraction unit uses convolution kernel sizes of 12, 15, and 15, and strides of 12, 9, and 5, respectively, to preliminarily extract features of the input tensor. The multi-scale hollow residual enhancement unit includes three feature extraction sub-units. Each feature extraction sub-unit includes a one-dimensional hollow convolution layer, a one-dimensional convolution layer, and a residual connection layer. The dilation parameters of the multi-scale hollow residual enhancement unit are sequentially increased by 1, 3, and 9, respectively. The hollow convolution kernel length is 7. The input tensor is sequentially input into the three feature extraction sub-units, so that the time length of the input tensor is sequentially reduced from the initial 21600 to 1800, 200, and 40, and the output data x[i] is obtained. In the first residual unit R1 of the feature extraction sub-unit, the length of the convolution kernel k d [j] of the one-dimensional hollow convolution layer D1 is 7, the hollow rate d is 1, and the output y D1 [i] of the one-dimensional hollow convolution layer D1 is: ; wherein, represents the length of the convolution kernel, represents the index in the convolution kernel, which is traversed from 0 to k-1, corresponding to each convolution kernel weight; represents the value of the i+jth position in the input signal x, and represents sampling of the input by the sliding window.

[0016] The output of the first one-dimensional hollow convolution is further input into a one-dimensional convolution layer with a convolution kernel length k c [j] of 1, to obtain the output y c1 [i] of the one-dimensional convolution layer. ; The residual connection is designed to connect the initial input with the output of the output of the one-dimensional convolution layer after the hollow convolution, and the output of the final first residual unit is y R1 [i] is represented as: ; wherein, The value of the i-th position is used for direct addition in the residual connection. is the output feature of the i+j-th position obtained by the previous layer of the hollow convolution layer D1; is the weight of the ordinary convolution kernel, and the value at the j-th position; The dilation parameters of the three improved multi-scale hollow convolution residual units in each feature extraction sub-unit are sequentially increased, which are 1, 3 and 9 respectively, and the hollow convolution kernel length k d The length of [j] is set to 7; the output y D2 [i] and y R2 [i] are respectively: ; ; The output y D3 [i] and y R3 [i] are respectively: ; ; The high-level feature tensor output by the feature extraction module is finally shaped as (16, 72, 40).

[0017] The fine-grained convolution operation is adopted to capture local detail dynamics and global sequence trends at the same time, and the number of feature channels is expanded from the original 3 to 72. While keeping the length of the time sequence (21600), the feature space expression ability is greatly enriched, laying a high-quality input foundation for subsequent multi-scale feature extraction and time sequence continuity optimization.

[0018] Further, the high-level feature tensor is input into two feature reconstruction modules to obtain the final data tensor, and the specific steps are as follows: Through linear interpolation, the high-level feature tensor is up-sampled, linear interpolation is performed between two known feature points, and the up-sampling ratio factor is set. The up-sampling ratio factors of the two feature reconstruction modules are 5 and 9 respectively, and the output y u1 of the up-sampling part is: ; wherein b = , represents the interpolation point yu1 [a] subscript corresponding to y; output data dimension is (16, 72, 200); representing the output of the feature extraction module; The scale alignment jump connection mechanism is adopted to cascade and splice the feature tensor after upsampling and the feature tensor before downsampling at the corresponding scale in the feature extraction module along the channel dimension, so as to realize efficient fusion of local detail information and high-level abstract features. If the data dimensions of the corresponding layers in the feature extraction module and the feature reconstruction module are (N, C enc , L) and (N, C dec , L), then after the jump connection, the data dimension becomes (N, C enc +C dec , L); wherein N represents the batch size, i.e. the number of samples input at a time; C enc represents the number of feature channels output by the feature extraction module; C dec represents the number of feature channels output by the feature extraction module; L indicates the sequence length, representing the time step or the spatial resolution; In the first jump connection, the data dimensions of the feature extraction module and the feature reconstruction module are both (16, 72, 200), and the output data dimension becomes (16, 144, 200); The features after the jump connection are compressed in the channel, and the number of channels is restored to 72. A one-dimensional convolution structure with a convolution kernel size of 15 and a stride of 1 is adopted, and batch normalization and ReLU activation functions are used to extract and fuse the fused feature information. The second feature reconstruction module processes in the same way as the first feature reconstruction module, and the final output data tensor dimension is (16, 72, 1800), that is, the sleep state prediction and the corresponding confidence score are generated every minute for the entire data obtained by splicing.

[0019] The feature reconstruction module gradually reconstructs and restores the high-level features compressed by the feature extraction module.

[0020] Further, the data tensor output by the feature reconstruction module is further fine-tuned and time series smoothed, forming a complete sleep state prediction and corresponding confidence score output every minute in a day, and the specific steps are as follows: The feature tensor (16, 72, 1800) output by the feature reconstruction module is taken as input, i.e. the preliminary prediction result at each time step is taken as input, forming a time series predict1-predict1800; let the initial input X1, the outputs f1, i1, o1 of the forget gate, the input gate and the output gate be: ; ; ; wherein, represents the input feature vector at the current time step; represents the initial hidden state vector; 、 、 represents the weight matrix of the forget gate, the input gate and the output gate; 、 、 represents the bias term corresponding to the forget gate, the input gate and the output gate; The memory cell state is represented as: ; wherein, represents the weight matrix of the memory cell state; represents the bias term corresponding to the memory cell state; The final memory cell state is determined by the memory cell state at the previous time step, the tentative memory cell state at the current time step, the forget gate output at the current time step and the input gate output at the current time step, and is represented as: ; The hidden state at the current time step is determined by the output gate output at the current time step and the final memory cell state at the current time step, and is represented as: ; The forward LSTM uses the complete historical information from the 1st minute to the tth minute to adjust the preliminary prediction at the current time step based on the global context; The backward LSTM is symmetrical to the forward structure, processes the input sequence in reverse order of time, and also fine-tunes using future context information. The output tensor dimension is also (16, 36, 1800). The outputs of the forward and backward LSTMs are spliced in the channel dimension to fuse the bidirectional time series information, obtaining a comprehensive feature representation with a shape of (16, 72, 1800); After an output layer, the 72-dimensional features are compressed to 1-dimensional, obtaining the final sleep state prediction score for each minute; To ensure the consistency of the prediction sequence and the original input data, the first and last 180 prediction points are cut off, and the middle 1440 consecutive time steps are retained to form a complete day of sleep state prediction and corresponding confidence score output every minute.

[0021] The sleep state prediction of each minute output by the feature reconstruction module is fine-tuned and time series smoothed again to improve the stability and continuity of the prediction sequence.

[0022] The application also provides a sleep detection system based on the method. The data preprocessing module is used for obtaining original data and performing preprocessing. The feature extraction module is used for extracting local features and global features of the input tensor output by the data preprocessing module, to obtain a high-level feature tensor. The feature reconstruction module receives the high-level feature tensor and adopts a linear interpolation reconstruction strategy to perform upsampling and feature skip connection on the high-level feature tensor, to obtain a final data tensor. The sequence continuity optimization module performs secondary fine-tuning and timing smoothing correction on the data tensor output by the feature reconstruction module, to form complete one-day per-minute sleep state prediction and corresponding confidence score output.

[0023] The system is based on an improved multi-scale hollow residual feature extraction and timing continuity optimization structure, and realizes more accurate wristband sleep detection.

[0024] Further, the data preprocessing module is connected with a wristband accelerometer, and the wristband accelerometer is worn on the arm of a user.

[0025] The data preprocessing module obtains original data and performs preprocessing, which is beneficial to use.

[0026] Further, the feature extraction module comprises a standard feature extraction unit and a multi-scale hollow residual enhancement unit. The standard feature extraction unit comprises a first one-dimensional convolution layer, a batch normalization layer and a RELU activation function layer connected in sequence, and the standard feature extraction unit uses convolution kernel sizes of 12, 15 and 15, and strides of 12, 9 and 5, respectively. The multi-scale hollow residual enhancement unit comprises three feature extraction subunits, each of which comprises a one-dimensional hollow convolution layer, a second one-dimensional convolution layer and a residual connection layer connected in sequence, the dilation parameters of the multi-scale hollow residual enhancement unit are sequentially increased and are 1, 3 and 9, respectively, and the hollow convolution kernel length is 7.

[0027] The feature extraction module adopts fine-grained convolution operation to simultaneously capture local detail dynamics and global sequence trends.

[0028] Further, the feature reconstruction module comprises an upsampling layer, a skip connection layer and a feature fusion layer connected in sequence, the upsampling layer is used for receiving the high-level feature tensor output by the feature extraction module and performing upsampling, and the feature fusion layer outputs a final data tensor to the sequence continuity optimization module.

[0029] The feature reconstruction module gradually reconstructs and finely restores the high-level features compressed by the feature extraction module, and accurately restores the timing information based on up-sampling with low computational complexity. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of the sleep detection method based on timing continuity optimization of the present application; Figure 2 is a structural diagram of the feature extraction module of the sleep detection method based on timing continuity optimization of the present application; Figure 3 is a structural diagram of the feature reconstruction module of the sleep detection method based on timing continuity optimization of the present application; Figure 4 is a structural diagram of the sequence continuity optimization module of the sleep detection method based on timing continuity optimization of the present application. DETAILED DESCRIPTION

[0031] The embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary, only for explaining the present application, and cannot be understood as a limitation of the present application.

[0032] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0033] In the description of the present application, unless otherwise specified and limited, it should be noted that the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be mechanical connection or electrical connection, or the communication between two elements, or direct connection, or indirect connection through intermediate medium, and the specific meaning of the above terms can be understood by those skilled in the art according to the specific circumstances.

[0034] This invention discloses a sleep detection method based on temporal continuity optimization. Based on improved multi-scale cavitation residual feature extraction and temporal continuity optimization, it is designed for wrist-worn accelerometer data, abandoning the complex medical-grade staging methods that rely on electroencephalogram (EEG) signals, and focusing on meeting the needs of portable, high-precision sleep state monitoring in daily life. Figure 1 As shown, the sleep detection method based on temporal continuity optimization includes the following steps: Raw wristband accelerometer data is acquired using a triaxial accelerometer and preprocessed to obtain the input tensor. The input tensor is fed into the feature extraction module, and fine-grained convolution operation is used to extract the local and global features of the input tensor to obtain a high-level feature tensor. The high-level feature tensor is input into the feature reconstruction module. A linear interpolation reconstruction strategy is used to upsample and perform feature skip connections on the high-level feature tensor to obtain the final data tensor. The data tensor output by the feature reconstruction module is fine-tuned and temporally smoothed to form a complete prediction of sleep state for each minute of the day and the corresponding confidence score output.

[0035] In a preferred embodiment of the present invention, the method for preprocessing the original data to obtain the input tensor is as follows: For the z-angle and ENMO activity levels in the original data, their standard deviations were calculated separately, and their natural logarithms were taken to obtain log_anglez and log_enmo; z-angle and ENMO activity levels. For the z-angle signal in the original features, its standard deviation was calculated and its natural logarithm was taken to enhance the perception of subtle angle changes; for the ENMO activity level feature, a logarithmic transformation was also taken to compress the extreme value range and highlight the characteristics of low- to medium-intensity motion. The z-angle specifically represents the change in the relative angle between the arm wearing the wristband and the vertical axis of the body, while the ENMO activity is a specially processed value of the Euclidean norm of the triaxial accelerometer data. The data is recorded every 5 seconds to ensure the temporal continuity of the feature data and efficient processing capabilities.

[0036] To identify signal quality, a valid_flag label corresponding to each time step is introduced. When duplicate records or invalid data are detected, the valid_flag of the corresponding time step is set to 0, and the valid_flag of normal records is set to 1, thereby effectively improving the robustness of the subsequent modeling stage. Thus, each time step contains three-dimensional features log_anglez, log_enmo, and valid_flag, which together form three-channel input data.

[0037] Preferably, in order to more fully fuse the context information of the front and rear time segments, improve the continuity recognition ability of sleep state transition, a cross-day data splicing strategy is also provided: For each target day, the feature sequence of the last 3 hours (2160 time steps) of the previous day, the whole-day data, and the first 3 hours (2160 time steps) of the next day is extracted, and a single 3x21600 feature matrix sample is spliced; each sample has a long-time span of continuous context characteristics, greatly enhancing the modeling ability of the sleep mode change across the night.

[0038] The batch size is set to 16, forming an input tensor with a shape of (16, 3, 21600); The input tensor is input into the preliminary feature extraction unit, and the number of feature channels is expanded from the original 3 to 72, obtaining an input tensor with a shape of (16, 72, 21600).

[0039] In a preferred embodiment of the application, the input tensor is input into the feature extraction module, and fine-grained convolution operation is used to extract local features and global features of the input tensor, and the specific method is: As shown in Figure 2 The feature extraction module includes a standard feature extraction unit (ConvBNReLU module) and a multi-scale hollow residual enhancement unit (improved residual structure), and the number of feature channels is expanded from the original 3 to 72 in this stage, while keeping the time sequence length (21600), greatly enriching the feature space expression ability, and laying a high-quality input foundation for subsequent multi-scale feature extraction and time sequence continuity optimization.

[0040] The standard feature extraction unit includes a one-dimensional convolution layer, a batch normalization (BatchNorm) layer, and an activation function layer, which preliminarily extracts the features of the input tensor, and the standard feature extraction unit uses convolution kernel sizes of 12, 15, and 15, with strides of 12, 9, and 5, respectively; The multi-scale hollow residual enhancement unit includes three feature extraction subunits, each of which includes a one-dimensional hollow convolution layer, a one-dimensional convolution layer, and a residual connection layer, and the dilation parameters of the multi-scale hollow residual enhancement unit are sequentially increased, being 1, 3, and 9, respectively, and the hollow convolution kernel length is 7, the input tensor is sequentially input into the three feature extraction subunits, so that the time length of the input tensor is sequentially reduced from the initial 21600 to 1800, 200, and 40, and the output data x[i] is obtained; the design of the stride and the convolution kernel not only realizes feature compression, but also effectively extracts local change characteristics, while maintaining the stable flow of sequence information.

[0041] To break through the limitation of traditional convolution units in modeling long-range dependence characteristics, the application innovatively introduces an improved multi-scale dilated convolution residual unit (MSDR-Unit) to capture fine local features and global long-range dependence features in wristband data in multiple levels and multiple scales.

[0042] The multi-scale dilated convolution residual unit is composed of a one-dimensional dilated convolution layer and a one-dimensional convolution layer and a residual connection, the one-dimensional dilated convolution is realized by inserting a dilated (dilation) between the convolution kernel elements on the basis of standard convolution, for a given dilated rate d, the convolution kernel skips d-1 elements on the input signal to perform convolution operation.

[0043] In the first residual unit R1 of the feature extraction subunit, the dilated (dilation) convolution kernel k d The length of [j] is 7, the dilated rate d is 1, the output y D1 [i] of the one-dimensional dilated convolution layer D1 is: ; Wherein, represents the length of the convolution kernel, represents the index in the convolution kernel, from 0 to k-1, corresponding to each convolution kernel weight; represents the value of the i+jth position in the input signal x, and represents the sampling of the input by the sliding window; The output of the first one-dimensional dilated convolution is further subjected to a one-dimensional convolution layer with a convolution kernel length k c The length of [j] is 1, and the output y c1 [i] is obtained. ; In order to avoid the occurrence of gradient vanishing problem in training, a residual connection is designed to connect the initial input and the output of the dilated convolution and the one-dimensional convolution layer, and the output of the final first residual unit is y R1 [i], which is expressed as: ; Wherein, is the value of the i-th position, which is used for direct addition in the residual connection; is the output feature of the i+jth position obtained by the dilated convolution layer D1 of the previous layer; is the weight of the ordinary convolution kernel, the value at the jth position; The dilation parameters of the three improved multi-scale dilated convolution residual units in each feature extraction subunit are sequentially increased by 1, 3 and 9, so as to systematically cover time-dependent features of different scales, and balance fine-grained dynamic changes and long-term trend modeling. The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y d The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y D2 The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y R2 The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y ; ; The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y D3 The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y R3 The lengths of the dilated convolution kernels in each improved multi-scale dilated convolution residual unit are set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction subunit output y ; ; The advanced feature tensor output by the feature extraction module finally has a shape of (16, 72, 40).

[0044] The residual unit design uniquely fuses information flows of different scales, and the dilated convolution part captures long-distance acceleration change patterns, such as posture change trends throughout the night and movement change amplitudes; the standard one-dimensional convolution part focuses on local details, such as angle changes, high activity or stability of short-time static state. This multi-level and multi-scale feature fusion enables the model to comprehensively perceive sleep dynamic changes from local micro changes to global trends, greatly improving the classification accuracy and robustness.

[0045] The MSDR-Unit multi-scale dilated convolution residual unit designed in the application greatly enriches the expression of the feature space while keeping the length of the input and output time series consistent. The dimension reduction operation in the feature extraction module mainly depends on the stride control in the standard feature extraction unit, thereby ensuring the sequence integrity in the dilated feature extraction process.

[0046] The application uniquely adopts the innovative multi-scale dilated convolution improved residual unit to significantly enhance the model's ability to capture features of different scales and retain local details, overcoming the precision deficiency of existing portable methods.

[0047] The multi-scale dilated convolution residual unit can capture local detail features and long-distance dependent features at the same time, and fuse information of different scales through jump connection, greatly improving the depth and expression ability of feature learning, and fully identifying and utilizing multi-level features in the time series.

[0048] In a preferred scheme of the present application, as shown in Figure 3 the high-level feature tensor is input into two feature reconstruction modules to obtain the final data tensor, and the specific steps are as follows: Through linear interpolation, the high-level feature tensor (shape (16, 72, 40)) is upsampled. Unlike the traditional upsample method based on deconvolution, the present application adopts an efficient linear interpolation reconstruction strategy to perform linear interpolation between two known feature points, thereby accurately recovering the timing information with low computational complexity. For the stride setting in the feature extraction stage, the present application sets the upsample ratio factor in the feature reconstruction module one by one. The upsample ratio factors of the two feature reconstruction modules are 5 and 9 respectively, ensuring that the upsample-downsample process is strictly corresponding in structure and scale matched. The output y u1 of the upsample part is: where b = represents the subscript of y corresponding to the interpolation point y u1 [a]; the output data dimension is (16, 72, 200); represents the output of the feature extraction module; The scale-aligned skip connection mechanism is adopted to concatenate and splice the upsampled feature tensor and the feature tensor before downsample at the corresponding scale in the feature extraction module along the channel dimension, realizing efficient fusion of local detail information and high-level abstract features; If the data dimensions of the corresponding layers in the feature extraction module and the feature reconstruction module are (N, C enc , L) and (N, C dec , L), respectively, then after the skip connection, the data dimension becomes (N, C enc +C dec , L); where N represents the batch size, i.e. the number of samples input at a time; C enc represents the number of feature channels output by the feature extraction module; C dec represents the number of feature channels output by the feature extraction module; L represents the sequence length, representing the time step or spatial resolution In the first skip connection, the data dimensions of the feature extraction module and the feature reconstruction module are both (16, 72, 200), and the output data dimension becomes (16, 144, 200); The features after the skip connection are compressed in the channel, and the number of channels is restored to 72 to maintain the consistency of the number of features and the stability of the network structure. A one-dimensional convolution structure with a convolution kernel size of 15 and a stride of 1 is adopted, combined with batch normalization and ReLU activation function, to refine the fused feature information; ​The second feature reconstruction module is processed in the same way as the first feature reconstruction module, and the final output data tensor has a dimension of (16, 72, 1800), that is, the sleep state prediction and the corresponding confidence score are generated every minute for the entire data segment (the last three hours of the previous day, the entire day, and the first three hours of the next day).

[0049] Preferably, the feature reconstruction module designed in the application has flexible and scalable multi-scale output capability. If two feature reconstruction modules are set, the prediction results can be output once every minute, which effectively reduces the computational complexity while ensuring sufficient time resolution to capture key changes in sleep state. If three feature reconstruction modules are added, fine-grained prediction for each original time step can be achieved, which is suitable for data sets with higher accuracy requirements or higher complexity.

[0050] This multi-scale flexibility design fully embodies the optimization balance between performance and computational efficiency of the application, which is different from the traditional fixed-scale reconstruction strategy, and has significant application potential.

[0051] In a preferred embodiment of the application, the data tensor output by the feature reconstruction module is fine-tuned and time series smoothed to form a complete sleep state prediction and corresponding confidence score output every minute of the day. The specific steps are as follows: The feature tensor (16, 72, 1800) output by the feature reconstruction module is used as input, that is, the preliminary prediction result of each time step (corresponding to one minute) is used as input to form a time series predict1-predict1800. Let the initial input X1, the outputs f1, i1, o1 of the forget gate, input gate, and output gate be: ; ; ; Among them, represents the input feature vector at the current time; represents the initial hidden state vector; , , represent the weight matrices of the forget gate, input gate, and output gate; , , represent the bias terms corresponding to the forget gate, input gate, and output gate; The memory cell state is represented as: ; Among them, represents the weight matrix of the memory cell state; bias term corresponding to the memory cell state; final memory cell state determined by the memory cell state of the previous time step, the tentative memory cell state of the current time step, the forget gate output of the current time step, and the input gate output of the current time step, and represented as: ; hidden state of the current time step determined by the output gate output of the current time step and the final memory cell state of the current time step, and represented as: ; The forward LSTM uses the complete historical information from the 1st minute to the tth minute to adjust the preliminary prediction of the current time step based on the global context; for example, when 95% of the predict predictions in a certain time period are "sleep" state, but there are 5% abnormal fluctuations locally, the forward LSTM can use the long-term memory mechanism to smooth correct these abnormalities, and improve the consistency and reliability of the overall prediction. The output tensor dimension of the forward LSTM is (16, 36, 1800).

[0052] The backward LSTM is symmetrical to the forward structure, processes the input sequence in reverse order (from 1800 to 1), and also fine-tunes using future context information. The output tensor dimension is also (16, 36, 1800); the outputs of the forward and backward LSTM are spliced in the channel dimension to fuse the bidirectional time series information, and a comprehensive feature representation with a shape of (16, 72, 1800) is obtained; After an output layer (fully connected layer), the 72-dimensional features are compressed to 1-dimensional, and the final sleep state prediction score of each minute is obtained; To ensure the consistency of the prediction sequence and the original input data, the first and last 180 prediction points (corresponding to the 3-hour part supplemented when the data is spliced) are cut off, and the remaining 1440 consecutive time steps are retained to form a complete day (24 hours) of sleep state prediction and corresponding confidence score output.

[0053] The present application significantly improves the time series modeling capability through a specially designed sequence continuity optimization module, and performs secondary smoothing on the prediction results from the overall sequence perspective, and comprehensively optimizes the stability and continuity of the prediction.

[0054] In the present application, the design of the sequence continuity optimization module fully utilizes the historical and future information, and does not directly generate a preliminary prediction, but optimizes and calibrates the preliminary prediction result output by the feature reconstruction module based on the overall time series evolution characteristics, significantly improves the coherence and overall accuracy of the final sleep state sequence, and is different from the traditional method which only predicts based on local convolution.

[0055] The application also provides a sleep detection system based on the method.

[0056] The data preprocessing module is used for obtaining raw data and performing preprocessing. The data preprocessing module is connected with a wristband type accelerometer which is worn on the arm of a user. The overall architecture of the application clearly takes the wristband type accelerometer signal as the only input, which is different from the traditional method which relies on multiple complex physiological signals such as electroencephalogram (EEG) and electrooculogram (EOG). The input data source selection, network architecture design and optimization strategy are systematically adapted to the resource-limited portable device environment.

[0057] The feature extraction module is used for extracting local features and global features of the input tensor output by the data preprocessing module to obtain a high-level feature tensor.

[0058] The feature reconstruction module receives the high-level feature tensor and adopts a linear interpolation reconstruction strategy to perform upsampling and feature skip connection on the high-level feature tensor to obtain a final data tensor. The feature extraction module of the application can systematically capture weak and scattered pattern changes in acceleration data during sleep without increasing too much computational load, ensuring the sensitivity and robustness of multi-scale dynamic features.

[0059] As shown in Figure 4 The sequence continuity optimization module performs secondary fine-tuning and timing smoothing correction on the data tensor output by the feature reconstruction module to form complete one-day per-minute sleep state prediction and corresponding confidence score output. The sequence continuity optimization module with a specific design utilizes the context information before and after the sleep sequence through a bidirectional timing information processing mechanism (bidirectional long short-term memory network structure), realizes the smooth output of the prediction result, and effectively solves the problem of insufficient sequence continuity in the existing method.

[0060] Unlike the traditional method of directly classifying and predicting based on convolution features, the sequence continuity optimization module designed in the application is not used to generate preliminary prediction results, but as a post-processing unit, which is specially used for secondary fine-tuning and timing smoothing correction of the per-minute sleep state prediction output by the feature reconstruction module, further improving the stability and continuity of the prediction sequence.

[0061] The application can guarantee high-precision sleep detection while significantly reducing model complexity and inference delay, and has the characteristics of low power consumption, good real-time performance and friendly deployment. The design not only meets the needs of consumer-level applications such as daily health monitoring and continuous sleep evaluation, but also has the potential to support a portable sleep health management system for a large number of users, and has wide practical application prospects and commercial value.

[0062] In a preferred scheme of the application, the feature extraction module includes a standard feature extraction unit and a multi-scale hollow residual enhancement unit.

[0063] The standard feature extraction unit includes a first one-dimensional convolution layer, a batch normalization layer and a RELU activation function layer connected in sequence, and the standard feature extraction unit uses a convolution kernel size of 12, 15 and 15, and a stride of 12, 9 and 5.

[0064] The multi-scale hollow residual enhancement unit includes three feature extraction sub-units, each of which includes a one-dimensional hollow convolution layer, a second one-dimensional convolution layer and a residual connection layer connected in sequence, the dilation parameters of the multi-scale hollow residual enhancement unit are sequentially increased, and are 1, 3 and 9 respectively, and the hollow convolution kernel length is 7. The application introduces a multi-scale hollow convolution to improve the residual unit, so as to capture more rich and fine temporal features through multi-scale receptive field and hollow convolution operation, and realize more accurate sleep state prediction at each time step.

[0065] The application innovatively proposes a multi-scale hollow convolution enhanced residual unit structure suitable for one-dimensional wristband type accelerometer time series data, discovers and effectively utilizes the great potential of the encoding-decoding architecture in time series data feature extraction and reconstruction. The introduction of one-dimensional hollow convolution with different dilation rates not only expands the receptive field, but also maintains the integrity of the time series details, effectively breaks through the local feature splicing method based on the traditional U-Net structure only based on the jump connection, and greatly improves the hierarchy and expression ability of feature extraction. Through the special multi-scale hollow convolution residual structure, the details and global features of the three-axis accelerometer under different time windows are systematically extracted, and the accuracy of sleep state prediction is significantly improved.

[0066] In a preferred scheme of the application, the feature reconstruction module includes an up-sampling layer, a jump connection layer and a feature fusion layer connected in sequence, the up-sampling layer is used to receive the high-level feature tensor output by the feature extraction module and perform up-sampling, and the feature fusion layer outputs the final data tensor to the sequence continuity optimization module.

[0067] After the feature reconstruction output, the preliminary prediction result is fine-tuned and smoothed. Unlike the traditional way of directly embedding the recurrent neural network into the encoder path, the present application introduces an independent post-processing stage to optimize the global consistency of the complete prediction sequence. This design makes full use of the forward and backward time-dependent information, and has a significant effect on eliminating short-term prediction noise and correcting local abnormal fluctuations, thereby significantly improving the stability, continuity and confidence of the prediction result, and is particularly suitable for long-time span continuous sleep monitoring tasks. This mechanism enables the model to maintain excellent continuous prediction performance when facing signal fluctuations in real wearing environments, thereby improving the reliability of practical applications.

[0068] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0069] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made thereto without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A sleep detection method based on timing continuity optimization, characterized in that, The method comprises the following steps: Collecting raw wristband accelerometer data through a three-axis accelerometer and preprocessing to obtain an input tensor; Inputting the input tensor into a feature extraction module, using fine-grained convolution operation to extract local features and global features of the input tensor, and obtaining a high-level feature tensor; Inputting the high-level feature tensor into a feature reconstruction module, using linear interpolation reconstruction strategy to upsample and feature skip connection on the high-level feature tensor, and obtaining a final data tensor; The data tensor output by the feature reconstruction module is subjected to secondary fine-tuning and timing smoothing correction to form a complete one-day per minute sleep state prediction and corresponding confidence score output.

2. The sleep detection method based on timing continuity optimization as claimed in claim 1, wherein, The method for preprocessing the original data to obtain the input tensor is: For the z angle and ENMO activity in the original data, the standard deviation of each is calculated separately, and the natural logarithm is taken to obtain log_anglez and log_enmo; Introducing a valid_flag label corresponding to the time step, when detecting duplicate records or invalid data, setting the valid_flag of the corresponding time step to 0, and setting the normal record to 1; Therefore, each time step contains three-dimensional features log_anglez, log_enmo and valid_flag, which together form a three-channel input data.

3. The sleep detection method based on timing continuity optimization as claimed in claim 2, wherein, There is also a cross-day data splicing strategy: for each target day, the feature sequence of the last 3 hours of the previous day, the whole day, and the first 3 hours of the next day is extracted, and a single 3*21600 feature matrix sample is spliced; The batch size is set to 16 to form an input tensor with a shape of (16, 3, 21600); The input tensor is input into the preliminary feature extraction unit, and the number of feature channels is expanded from the original 3 to 72 to obtain an input tensor with a shape of (16, 72, 21600).

4. The sleep detection method based on timing continuity optimization as claimed in claim 1, wherein, The input tensor is input into the feature extraction module, and fine-grained convolution operation is used to extract local features and global features of the input tensor. The specific method is that the feature extraction module includes a standard feature extraction unit and a multi-scale hollow residual enhancement unit; The standard feature extraction unit includes a one-dimensional convolution layer, a batch normalization layer and an activation function layer, and the standard feature extraction unit uses convolution kernel sizes of 12, 15 and 15, with strides of 12, 9 and 5, respectively, to preliminarily extract the features of the input tensor; The multi-scale hollow residual enhancement unit includes three feature extraction subunits, each of which includes a one-dimensional hollow convolution layer, a one-dimensional convolution layer and a residual connection layer. The dilation parameters of the multi-scale hollow residual enhancement unit increase in turn, which are 1, 3 and 9 respectively. The hollow convolution kernel length is 7. The input tensor is sequentially input into the three feature extraction subunits, so that the time length of the input tensor is sequentially reduced from the initial 21600 to 1800, 200 and 40, and the output data x[i] is obtained. In the first residual unit R1 of the feature extraction subunit, the convolution kernel k of the hole convolution d The length of [j] is 7, the hole rate d is 1, and the output y of the one-dimensional hole convolution layer D1 D1 [i] is: ； wherein, denotes the length of the convolution kernel, denotes the index in the convolution kernel, iterating from 0 to k-1, corresponding to each convolution kernel weight; denotes the value of the i+jth position in the input signal x, denoting the sampling of the input by the sliding window; The output of the first one-dimensional empty convolution is further processed by a convolution kernel length k c [j] is a one-dimensional convolution layer with 1, obtaining output y c1 [i] is: ； The residual connection is designed to connect the initial input with the output after the output of the one-dimensional convolution layer is subjected to the cavity convolution, and the output of the final first residual unit is y R1 [i] is represented as: ； wherein, a value of the i-th position for direct addition in the residual connection; is an output feature of the i+j-th position obtained by the previous layer of the dilated convolution layer D1; is a weight of the ordinary convolution kernel, a value at the j-th position; The dilation parameters of the three improved multi-scale dilated convolution residual units in each feature extraction sub-unit are sequentially increased by 1, 3, and 9, respectively, and the length k of the dilated convolution kernel in each improved multi-scale dilated convolution residual unit is 3 d The lengths of j and y are both set to 7; the dilated convolution D2 and the residual unit R2 of the second feature extraction sub-unit output y D2 The lengths of i and y R2 The lengths of i and y ； ； The third feature extraction subunit outputs y D3 [i] and y R3 [i] are respectively: ； ； The high-level feature tensor output by the feature extraction module has a shape of (16, 72, 40).

5. The sleep detection method based on timing continuity optimization as claimed in claim 1, wherein, The high-level feature tensor is input into two feature reconstruction modules to obtain a final data tensor. The specific steps are as follows: By linear interpolation, the high-level feature tensor is up-sampled, linear interpolation is performed between two known feature points, the up-sampling scale factor is set, the up-sampling scale factors of the two feature reconstruction modules are 5 and 9 respectively, and the output y of the up-sampling part is u1 is: )； wherein b = 1 , represents the interpolation point y u1 [a] corresponding to the index of y; the output data dimension is (16, 72, 200); represents the output of the feature extraction module; The scale-aligned skip connection mechanism is adopted to concatenate the feature tensor after upsampling and the feature tensor before downsampling at the corresponding scale in the feature extraction module along the channel dimension, so as to realize efficient fusion of local detailed information and high-level abstract features. If the data dimensions of the corresponding layers in the feature extraction module and the feature reconstruction module are (N, C enc , L) and (N, C dec , L), then after the jump connection, the data dimensions become (N, C enc +C dec , L); wherein N represents the batch size, i.e. the number of samples input at one time; C enc represents the number of feature channels output by the feature extraction module; C dec represents the number of feature channels output by the feature extraction module; and L represents the sequence length, i.e. the time step or the spatial resolution; In the first skip connection, the data dimensions of the feature extraction module and the feature reconstruction module are both (16, 72, 200), and the output data dimension becomes (16, 144, 200); The feature after the skip connection is compressed in the channel, and the channel number is restored to 72. A one-dimensional convolution structure with a convolution kernel size of 15 and a stride of 1 is adopted to refine the fused feature information, together with batch normalization and ReLU activation function. The second feature reconstruction module processes in the same way as the first feature reconstruction module, and the final output data tensor dimension is (16, 72, 1800), that is, the sleep state prediction and the corresponding confidence score are generated every minute for the entire data segment.

6. The sleep detection method based on timing continuity optimization as claimed in claim 1, wherein, The data tensor output by the feature reconstruction module is fine-tuned and time-series smoothed to form a complete sleep state prediction and corresponding confidence score output every minute for a day, and the specific steps are as follows: The feature tensor (16, 72, 1800) output by the feature reconstruction module is taken as input, that is, the preliminary prediction result at each time step is taken as input to form a time series predict1-predict1800; let the initial input X1, the outputs f1, i1, o1 of the forget gate, input gate and output gate be respectively: ； ； ； wherein, represents an input feature vector at a current time point; represents an initial hidden state vector; , , represents a weight matrix of a forget gate, an input gate, and an output gate; , , represents a bias term corresponding to the forget gate, the input gate, and the output gate; Memory cell state is represented as: ； wherein, a weight matrix representing a memory cell state; a bias term corresponding to the memory cell state. Final memory cell state is determined by the memory cell state of the previous time step, the tentative memory cell state of the current time step, the forget gate output of the current time step, and the input gate output of the current time step, and is expressed as: ； the hidden state of the current time step is determined by the output gate output of the current time step and the final memory cell state of the current time step, denoted as: ； The forward LSTM uses the complete historical information from the 1st minute to the tth minute to adjust the preliminary prediction of the current time step based on the global context; The backward LSTM is symmetrical to the forward structure and processes the input sequence in reverse order, and also fine-tunes using future context information. The output tensor dimension is also (16, 36, 1800); the outputs of the forward and backward LSTMs are concatenated in the channel dimension to fuse the bidirectional time-series information and obtain a comprehensive feature representation with a shape of (16, 72, 1800); After an output layer, the 72-dimensional feature is compressed to 1-dimensional to obtain the final sleep state prediction score every minute; To ensure the consistency of the prediction sequence and the original input data, the first and last 180 prediction points are cut off, and the middle 1440 consecutive time steps are retained to form a complete sleep state prediction and corresponding confidence score output every minute for a day.

7. A sleep detection system based on the method of any one of claims 1 to 6, characterized by The data preprocessing module, the feature extraction module, the feature reconstruction module and the sequence continuity optimization module are sequentially connected. The data preprocessing module is used to obtain and preprocess the original data; The feature extraction module is used to extract local features and global features of the input tensor output by the data preprocessing module to obtain a high-level feature tensor; The feature reconstruction module receives the high-level feature tensor and adopts a linear interpolation reconstruction strategy to upsample and perform feature skip connection on the high-level feature tensor to obtain a final data tensor; The sequence continuity optimization module performs secondary fine-tuning and timing smoothing correction on the data tensor output by the feature reconstruction module, forming complete one-day per minute sleep state prediction and corresponding confidence score output.

8. The sleep detection system of claim 7, wherein, The data preprocessing module is connected with a wristband accelerometer, and the wristband accelerometer is worn on the arm of the user.

9. The sleep detection system of claim 7, wherein, The feature extraction module includes a standard feature extraction unit and a multi-scale hollow residual enhancement unit. The standard feature extraction unit includes a first one-dimensional convolution layer, a batch normalization layer and a RELU activation function layer connected in sequence, and the standard feature extraction unit uses convolution kernel sizes of 12, 15 and 15, and strides of 12, 9 and 5. The multi-scale hollow residual enhancement unit includes three feature extraction sub-units, each of which includes a one-dimensional hollow convolution layer, a second one-dimensional convolution layer and a residual connection layer connected in sequence, and the dilation parameters of the multi-scale hollow residual enhancement unit are sequentially increased by 1, 3 and 9, and the hollow convolution kernel length is 7.

10. The sleep detection system of claim 7, wherein, The feature reconstruction module includes an upsampling layer, a jump connection layer and a feature fusion layer connected in sequence, the upsampling layer is used to receive the high-level feature tensor output by the feature extraction module and perform upsampling, and the feature fusion layer outputs the final data tensor to the sequence continuity optimization module.

Images