Sleep state remote monitoring system and method based on telemedicine

Abstract

The invention relates to the field of intelligent monitoring of sleep states, and particularly discloses a sleep state remote monitoring system and method based on telemedicine. Firstly, a wireless respiration sensor is used for collecting human body respiration frequencies of a to-be-monitored user at a plurality of preset time points, and an infrared sensor is used for collecting a micro-motion signal caused by heartbeat of the to-be-monitored user and a body motion signal of the to-be-monitored user as input data; and then performing deep convolutional coding and analysis on the input data by using a machine learning technology to obtain a classification result, wherein the classification result is used for representing whether the sleep state of the to-be-monitored user has an abnormal condition or not. Therefore, the sleep state of the human body can be remotely monitored and analyzed. In this way, the sleep quality and health of the user can be better managed.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring of sleep status, and more specifically, to a remote monitoring system and method for sleep status based on telemedicine. Background Art

[0002] Sleep is a vital bodily need that is essential for restoring energy, promoting memory consolidation, and brain integration. About one-third of human life is spent sleeping, which is enough to show the importance of sleep in our lives. If you don't sleep for five consecutive days, it will have a fatal effect on the human body, which shows that sleep is an indispensable need for our physiology.

[0003] Good sleep helps eliminate fatigue in the body and brain. On the contrary, if sleep is deprived, people will become irritable and have difficulty concentrating, which will have a negative impact on health in the long run. With the continuous development of society, people's pace of life and work has accelerated, external competition is fierce, and internal pressure is increasing, which has led to many people facing problems such as insomnia and sleep disorders. These sleep problems have caused a series of sub-health conditions and related complications, which have attracted great attention.

[0004] Currently, some sleep monitoring devices have been launched on the market, such as polysomnography and sleep monitoring bracelets. Polysomnography obtains EEG data by fixing electrode patches on the human body in order to analyze the sleep state. However, users need to wear these electrode patches all the time when sleeping, which is not only uncomfortable but also affects movements such as turning over. The sleep monitoring bracelet uses built-in accelerometers and gyroscopes to monitor sleep movements and analyze the sleep state. However, "sleep detection" based on this principle can actually only be regarded as a "sleep speculation", which is not very accurate and has limited reference value.

[0005] Therefore, we look forward to a remote monitoring system and method for sleep status based on telemedicine, which can remotely monitor and analyze the sleep status of the human body based on machine learning technology, so as to achieve better management of the user's sleep quality and health. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a remote monitoring system and method for sleep status based on telemedicine, which first uses a wireless respiratory sensor to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points and uses an infrared sensor to collect the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored as input data, and then uses machine learning technology to perform deep convolution encoding and analysis on these input data to obtain classification results, and the classification results are used to indicate whether there is an abnormality in the sleep state of the user to be monitored. In this way, remote monitoring and analysis of the sleep state of the human body can be achieved. In this way, it is helpful to better manage the sleep quality and health of the user.

[0007] According to a first aspect of the present application, a sleep state remote monitoring system based on telemedicine is provided, which comprises:

[0008] A sleep state data acquisition module, used to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body movement signal of the user to be monitored;

[0009] A sleep state feature extraction module, used to extract and analyze the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body movement signal of the user to be monitored, so as to obtain a human respiratory frequency feature vector and a micro-motion body movement associated feature vector of the user to be monitored;

[0010] A sleep state feature fusion module, used for combining the human respiratory frequency feature vector and the micro-motion associated feature vector of the user to be monitored using a joint encoder to obtain a sleep feature matrix of the user to be monitored;

[0011] The sleep state classification result generating module is used to obtain a classification result based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is any abnormality in the sleep state of the user to be monitored.

[0012] In combination with the first aspect of the present application, in a sleep state remote monitoring system based on telemedicine in the first aspect of the present application, the sleep state feature extraction module includes: a human respiratory frequency feature extraction unit, which is used to arrange the human respiratory frequencies of multiple predetermined time points of the user to be monitored into a human respiratory frequency input vector according to the time dimension and then obtain the human respiratory frequency feature vector through a human respiratory frequency feature extraction module based on a timing encoder; a user micro-motion body movement associated feature extraction unit, which is used to perform feature encoding on the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored to obtain the micro-motion body movement associated feature vector of the user to be monitored.

[0013] According to a second aspect of the present application, a method for remote monitoring of sleep status based on telemedicine is provided, which comprises:

[0014] Collecting the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored;

[0015] Extracting and analyzing the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored;

[0016] Using a joint encoder, the human respiratory frequency feature vector and the micro-motion associated feature vector of the user to be monitored are combined to obtain a sleep feature matrix of the user to be monitored;

[0017] A classification result is obtained based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is an abnormality in the sleep state of the user to be monitored.

[0018] In combination with the second aspect of the present application, in a remote monitoring method of sleep status based on telemedicine in the second aspect of the present application, a classification result is obtained based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is an abnormality in the sleep state of the user to be monitored, including: a sleep feature extraction unit for the user to be monitored, used to pass the sleep feature matrix of the user to be monitored through a sleep feature extraction module for the user to be monitored based on a convolutional neural network model to obtain a sleep classification feature vector of the user to be monitored; a sleep feature optimization unit for the user to be monitored, used to perform feature screening optimization on the sleep classification feature vector of the user to be monitored based on set aggregation bounded deviation to obtain an optimized sleep classification feature vector of the user to be monitored; and a sleep feature classification unit for the user to be monitored, used to pass the optimized sleep classification feature vector of the user to be monitored through a classifier to obtain the classification result.

[0019] Compared with the prior art, the present application provides a remote monitoring system and method for sleep status based on telemedicine, which first uses a wireless respiratory sensor to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points and uses an infrared sensor to collect the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored as input data, and then uses machine learning technology to perform deep convolution encoding and analysis on these input data to obtain classification results, and the classification results are used to indicate whether there is an abnormality in the sleep state of the user to be monitored. In this way, remote monitoring and analysis of the human body's sleep state can be achieved. In this way, it is helpful to better manage the user's sleep quality and health. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 The figure illustrates a schematic block diagram of a remote monitoring system of sleep status based on telemedicine according to an embodiment of the present application.

[0022] Figure 2 The figure illustrates a schematic block diagram of a sleep state feature extraction module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application.

[0023] Figure 3 The diagram illustrates a schematic block diagram of a user micro-motion body motion associated feature extraction unit in a sleep state feature extraction module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application.

[0024] Figure 4 The figure illustrates a schematic block diagram of a sleep state classification result generating module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application.

[0025] Figure 5 The figure illustrates a flow chart of a remote monitoring method of sleep status based on telemedicine according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0027] Figure 1 FIG. 1 is a schematic block diagram of a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application. Figure 1As shown, according to the embodiment of the present application, the remote monitoring system 100 of the sleep state based on telemedicine includes: a sleep state data acquisition module 110, which is used to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored; a sleep state feature extraction module 120, which is used to extract and analyze the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored; a sleep state feature fusion module 130, which is used to use a joint encoder to perform feature combination on the human respiratory frequency feature vector and the micro-motion body motion associated feature vector of the user to be monitored to obtain a sleep feature matrix of the user to be monitored; a sleep state classification result generation module 140, which is used to obtain a classification result based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is an abnormality in the sleep state of the user to be monitored.

[0028] Adequate sleep is essential for restoring strength, consolidating memory and boosting brain function. Humans spend nearly a third of their lives sleeping, highlighting the critical role sleep plays in our lives. Five consecutive days of sleep deprivation can cause serious harm to the body, highlighting the indispensability of sleep to our physiological health.

[0029] Good sleep helps eliminate physical and mental fatigue. On the contrary, lack of sleep can lead to irritability and lack of concentration, which can have a negative impact on health in the long run. With the rapid development of society, people's life and work pressures are increasing, and external competition is fierce, which has led to many people facing problems such as sleep disorders and insomnia. These sleep problems have caused a series of sub-health conditions and related complications, which have attracted widespread attention.

[0030] At present, some sleep monitoring devices have been launched on the market, such as polysomnography and sleep monitoring bracelets. Polysomnography obtains EEG data by fixing electrode patches on the human body in order to analyze the sleep state. However, users need to wear these electrode patches all the time when they sleep, which is not only uncomfortable, but also affects movements such as turning over. The sleep monitoring bracelet uses built-in accelerometers and gyroscopes to monitor sleep movements and analyze sleep states. However, "sleep detection" based on this principle can actually only be regarded as a kind of "sleep speculation", which is not accurate and has limited reference value. Therefore, a remote monitoring system and method for sleep status based on telemedicine is expected to remotely monitor and analyze the sleep status of the human body based on machine learning technology to achieve better management of the user's sleep quality and health.

[0031] In an embodiment of the present application, the sleep state data acquisition module 110 is used to collect the human respiratory frequency at multiple predetermined time points of the user to be monitored, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body movement signal of the user to be monitored. It should be understood that the human respiratory frequency usually changes during sleep. Under normal circumstances, the respiratory frequency will be different in different sleep stages and sleep states. By monitoring the changes in the respiratory frequency, the user's sleep quality can be evaluated, such as the ratio of deep sleep and light sleep, whether there are abnormal conditions during sleep, etc. Therefore, in order to better monitor the user's sleep state, the human respiratory frequency at multiple predetermined time points of the user to be monitored is first collected. Next, considering that both heartbeat and body movement are physiological activities generated by the human body during sleep. Some studies have shown that there may be a certain correlation between the rhythm of the heartbeat and body movement during sleep. For example, some body movements may be related to changes in the heartbeat frequency, or some characteristics of the heartbeat may affect the pattern of body movement. Therefore, in sleep monitoring and analysis, considering the micro-motion signal and body movement signal caused by the heartbeat at the same time can provide more comprehensive information to help evaluate the user's sleep state and health status. Therefore, in order to monitor and analyze the user's sleep state more comprehensively, the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored are further collected as input data.

[0032] In particular, when collecting these input data, a wireless breathing sensor can be placed under the mattress or sheets to monitor the human body's breathing, and an infrared sensor can be installed beside the bed or in the room to monitor the human body's micro-motion signals and the body movement signals of the user to be monitored.

[0033] In the embodiment of the present application, the sleep state feature extraction module 120 is used to extract and analyze the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored. It should be understood that after collecting these input data, the input data are further subjected to feature extraction and analysis tasks.

[0034] Specifically, Figure 2 FIG. 1 is a schematic block diagram of a sleep state feature extraction module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application. Figure 2As shown, the sleep state feature extraction module 120 includes: a human respiratory frequency feature extraction unit 121, which is used to arrange the human respiratory frequencies of multiple predetermined time points of the user to be monitored into a human respiratory frequency input vector according to the time dimension and then obtain the human respiratory frequency feature vector through a human respiratory frequency feature extraction module based on a timing encoder; a user micro-motion body movement associated feature extraction unit 122, which is used to perform feature encoding on the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored respectively to obtain the micro-motion body movement associated feature vector of the user to be monitored.

[0035] First, the human respiratory frequency at multiple predetermined time points of the user to be monitored is subjected to feature extraction and analysis operations. It should be understood that the human respiratory frequency is time series data, that is, the human respiratory frequency changes dynamically over time. Based on this, in order to better retain the time series information of the respiratory frequency data, the human respiratory frequency at multiple predetermined time points of the user to be monitored is first arranged as a human respiratory frequency input vector according to the time dimension. Further, it is considered that the time series encoder has advantages in processing time series data. The time series encoder is a neural network structure specially designed for processing sequence data, which can effectively capture the time correlation and sequence pattern in the data. Through the time series encoder, the system can learn the complex patterns and regularities in the respiratory frequency data, extract more representative and useful respiratory frequency features, and thus more accurately analyze the user's sleep state. Therefore, in order to understand the user's breathing situation more comprehensively and accurately, the time series encoder is used to perform deep convolution encoding on the human respiratory frequency input vector. Specifically, in an embodiment of the present application, the human respiratory frequency input vector is deeply convolutionally encoded by a human respiratory frequency feature extraction module based on a time series encoder to obtain the human respiratory frequency feature vector.

[0036] In a specific embodiment of the present application, the human respiratory frequency feature extraction unit 121 is used to: use the fully connected layer of the time series encoder to fully connect the human respiratory frequency input vector to extract the high-dimensional implicit features of the eigenvalues ​​at each position in the human respiratory frequency input vector; and use the one-dimensional convolution layer of the time series encoder to perform one-dimensional convolution encoding on the human respiratory frequency input vector to extract the high-dimensional implicit correlation features of the correlation between the eigenvalues ​​at each position in the human respiratory frequency input vector.

[0037] Then, feature extraction and analysis are performed on the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored. Specifically, Figure 3 The figure shows a schematic block diagram of a user micro-motion body motion associated feature extraction unit in a sleep state feature extraction module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application. Figure 3 As shown, the user micro-motion body motion associated feature extraction unit 122 includes: a signal denoising unit 122-1, used for respectively passing the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored through a signal denoising module based on an autoencoder to obtain a denoised micro-motion signal of the user to be monitored and a denoised body motion signal of the user to be monitored; a micro-motion feature extraction unit 122-2, used for passing the waveform of the denoised micro-motion signal of the user to be monitored through a micro-motion feature extraction module based on a residual double attention mechanism model to obtain a micro-motion feature map of the user to be monitored; a body motion feature extraction unit 122-3, used for passing the waveform of the denoised micro-motion signal of the user to be monitored through a micro-motion feature extraction module based on a residual double attention mechanism model to obtain a micro-motion feature map of the user to be monitored; The feature extraction unit 122-3 is used to pass the waveform of the noise-reduced body motion signal of the user to be monitored through a body motion feature extraction module based on convolution kernels of different scales to obtain a body motion feature map of the user to be monitored; the micro-motion body motion association unit 122-4 is used to perform feature fusion on the micro-motion feature map of the user to be monitored and the body motion feature map of the user to be monitored to obtain a micro-motion body motion association feature map of the user to be monitored; the association feature map pooling unit 122-5 is used to perform a pooling operation on the micro-motion body motion association feature map of the user to be monitored to obtain a micro-motion body motion association feature vector of the user to be monitored.

[0038] It should be understood that when collecting the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored, the noise of the surrounding environment (such as electromagnetic interference, sound interference, etc.) may affect the clarity and accuracy of the signal, so these signals need to be denoised first. Further, considering that the autoencoder is an unsupervised learning model, it is possible to learn the potential representation of the data, that is, to learn the most important features from the input data. This characteristic enables the autoencoder to perform well in the noise reduction task and can effectively remove the noise in the signal. Therefore, the autoencoder is used to perform noise reduction on the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored. Specifically, in an embodiment of the present application, the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored are respectively passed through a signal denoising module based on an autoencoder to obtain the micro-motion signal of the user to be monitored after noise reduction and the body motion signal of the user to be monitored after noise reduction.

[0039] In a specific embodiment of the present application, the signal noise reduction unit 122-1 is used to: use the feature encoder of the signal noise reduction module to extract multiple signal features from the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored; and use the feature decoder of the signal noise reduction module to decode the multiple signal features to obtain the micro-motion signal of the user to be monitored after noise reduction and the body movement signal of the user to be monitored after noise reduction.

[0040] Next, it should be understood that the micro-motion signal caused by the heartbeat refers to the tiny movement or vibration signal generated by the human heartbeat. The micro-motion signal may contain complex patterns and features, and a powerful model is required to learn these features. Considering that the residual dual attention mechanism model has a strong representation learning ability, it can effectively extract important features in the signal. Therefore, the micro-motion feature extraction module based on the residual dual attention mechanism model is used to feature encode the waveform of the micro-motion signal of the user to be monitored after noise reduction to obtain the micro-motion feature map of the user to be monitored. In this way, the user's micro-motion signal can be better described and analyzed.

[0041] In a specific embodiment of the present application, the micro-motion feature extraction unit 122-2 is used to: pass the waveform of the micro-motion signal of the user to be monitored after noise reduction through the spatial attention module of the residual dual attention mechanism model to obtain a spatial attention map; pass the waveform of the micro-motion signal of the user to be monitored after noise reduction through the channel attention module of the residual dual attention mechanism model to obtain a channel attention map; fuse the spatial attention map and the channel attention map to obtain a weighted feature map; and fuse the waveform of the micro-motion signal of the user to be monitored after noise reduction and the weighted feature map to obtain a micro-motion feature map of the user to be monitored.

[0042] Among them, the waveform of the micro-motion signal of the user to be monitored after noise reduction is passed through the spatial attention module of the residual dual attention mechanism model to obtain a spatial attention map, including: performing average pooling and maximum pooling on the waveform of the micro-motion signal of the user to be monitored after noise reduction along the channel dimension to obtain an average feature matrix and a maximum feature matrix; cascading and channel-adjusting the average feature matrix and the maximum feature matrix to obtain a channel feature matrix; using the convolution layer of the spatial attention module to convolutionally encode the channel feature matrix to obtain a convolution feature matrix; passing the convolution feature matrix through the Softmax function to obtain a spatial attention score matrix; and multiplying the spatial attention score matrix by each feature matrix of the waveform of the micro-motion signal of the user to be monitored after noise reduction along the channel dimension according to the position point to obtain the spatial attention map.

[0043] Next, it should be understood that body motion signals refer to motion or action signals generated by the human body during sleep, such as small movements such as turning over and turning legs. These body motion signals can reflect the user's activity status and movement during sleep, and provide important information on sleep quality and sleep depth. The body motion signal may contain features and patterns on different time scales. Based on this, in order to capture more comprehensive and rich body motion information, convolution kernels of different scales are used to capture features of the user's body motion signal. Specifically, in an embodiment of the present application, the waveform of the noise-reduced body motion signal of the user to be monitored is passed through a body motion feature extraction module based on convolution kernels of different scales to obtain a body motion feature map of the user to be monitored. Among them, using convolution kernels of different scales, the signal can be feature extracted at different scales, and the smaller-scale convolution kernel can capture subtle features, while the larger-scale convolution kernel can capture more macroscopic features. In this way, the comprehensive use of information at different scales can better describe the characteristics of the body motion signal.

[0044] In a specific embodiment of the present application, the body motion feature extraction unit 122-3 is used to: use the layers of the body motion feature extraction module based on convolution kernels of different scales to perform the following on the input data in the forward transfer of the layer: convolution processing on the input data based on the first convolution kernel to obtain a first convolution feature map; convolution processing on the input data based on the second convolution kernel to obtain a second convolution feature map; convolution processing on the input data based on the third convolution kernel to obtain a third convolution feature map; convolution processing on the input data based on the fourth convolution kernel to obtain a fourth convolution feature map, wherein the The first convolution kernel, the second convolution kernel, the third convolution kernel and the fourth convolution kernel have different sizes; the first convolution feature map, the second convolution feature map, the third convolution feature map and the fourth convolution feature map are cascaded to obtain a multi-scale convolution feature map; the multi-scale convolution feature map is subjected to mean pooling processing based on a local feature matrix to obtain a pooled feature map; and the pooled feature map is subjected to nonlinear activation processing to obtain an activated feature map; wherein the output of the last layer of the body motion feature extraction module based on convolution kernels of different scales is the body motion feature map of the user to be monitored.

[0045] Then, it should be understood that during sleep, there may be a certain correlation between the rhythm of the heartbeat and body movement. For example, some body movements may be related to changes in the heartbeat frequency, or some characteristics of the heartbeat may affect the pattern of body movement. Therefore, in order to capture the correlation features between the two, the micro-motion feature map of the user to be monitored and the body movement feature map of the user to be monitored are feature fused to obtain the micro-motion body movement correlation feature map of the user to be monitored.

[0046] In a specific embodiment of the present application, the micro-motion body movement association unit 122-4 is used to fuse the micro-motion feature map of the user to be monitored and the body movement feature map of the user to be monitored using the following formula to obtain the micro-motion body movement association feature map of the user to be monitored, wherein the fusion formula is:

[0047] F1=αF s +βF d

[0048] Among them, F1 is the micro-motion correlation feature map of the user to be monitored, F s is the micro-motion characteristic graph of the user to be monitored, F d is the body movement characteristic map of the user to be monitored, "+" represents the addition of the elements at corresponding positions of the micro-motion characteristic map of the user to be monitored and the body movement characteristic map of the user to be monitored, and α and β are weighted parameters for controlling the balance between the micro-motion characteristic map of the user to be monitored and the body movement characteristic map of the user to be monitored in the micro-motion body movement associated characteristic map of the user to be monitored.

[0049] Finally, considering that high-dimensional feature maps will increase the complexity of calculations, reduce the execution efficiency of the algorithm, and affect real-time performance and response speed. Therefore, in order to improve the calculation efficiency of the model, the feature map associated with the micro-motion of the user to be monitored is subjected to a dimensionality reduction operation. Specifically, in an embodiment of the present application, the feature map associated with the micro-motion of the user to be monitored is subjected to a pooling operation to obtain a feature vector associated with the micro-motion of the user to be monitored.

[0050] In a specific embodiment of the present application, the associated feature map pooling unit 122-5 is used to: perform a feature matrix-based mean pooling operation on the associated feature map of the micro-motion body movement of the user to be monitored to obtain the associated feature vector of the micro-motion body movement of the user to be monitored.

[0051] In an embodiment of the present application, the sleep state feature fusion module 130 is used to use a joint encoder to perform feature combination on the human respiratory rate feature vector and the micro-motion body movement associated feature vector of the user to be monitored to obtain a sleep feature matrix of the user to be monitored. It should be understood that the human respiratory rate feature vector and the micro-motion body movement associated feature vector represent different aspects of information. The respiratory rate reflects the respiratory activity, while the micro-motion body movement associated features reflect the movement activity. Fusion of the two can comprehensively consider the user's breathing and movement conditions and provide a more comprehensive description of the sleep state. Therefore, in order to more comprehensively analyze the user's sleep state, a joint encoder is used to perform feature combination on the human respiratory rate feature vector and the micro-motion body movement associated feature vector of the user to be monitored to obtain a sleep feature matrix of the user to be monitored. Among them, the joint encoder can automatically learn how to fuse different types of features together to obtain a feature representation with more characterization capabilities. This helps the system better understand and utilize the associations between different types of features.

[0052] In a specific embodiment of the present application, the sleep state feature fusion module 130 is used to: use the joint encoder to jointly encode the human respiratory rate feature vector and the micro-motion associated feature vector of the user to be monitored using the following formula to generate the sleep feature matrix of the user to be monitored; wherein the formula is: in Represents vector multiplication, M a represents the sleep feature matrix of the user to be monitored, V a1 represents the human respiratory frequency feature vector, V a2 represents the micro-motion associated feature vector of the user to be monitored, represents the transpose of the micro-motion associated feature vector of the user to be monitored.

[0053] In the embodiment of the present application, the sleep state classification result generating module 140 is used to obtain a classification result based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether the sleep state of the user to be monitored is abnormal. Specifically, Figure 4 FIG. 1 is a schematic block diagram of a sleep state classification result generation module in a sleep state remote monitoring system based on telemedicine according to an embodiment of the present application. Figure 4As shown, the sleep state classification result generating module 140 includes: a sleep feature extraction unit 141 for a user to be monitored, which is used to pass the sleep feature matrix of the user to be monitored through a sleep feature extraction module for a user to be monitored based on a convolutional neural network model to obtain a sleep classification feature vector for the user to be monitored; a sleep feature optimization unit 142 for a user to be monitored, which is used to perform feature screening optimization based on a set aggregated bounded deviation on the sleep classification feature vector for the user to be monitored to obtain an optimized sleep classification feature vector for the user to be monitored; and a sleep feature classification unit 143 for a user to be monitored, which is used to pass the optimized sleep classification feature vector for the user to be monitored through a classifier to obtain the classification result.

[0054] It should be understood that in the process of feature fusion using the joint encoder, information loss or feature distortion may occur, affecting the quality of the final sleep state feature representation. Therefore, in order to more accurately capture the user's sleep state, a sleep feature extraction module for the monitored user based on a convolutional neural network model is further used to perform a deeper feature extraction operation on the sleep feature matrix of the monitored user to obtain a sleep classification feature vector for the monitored user. Among them, the convolutional neural network model can make up for the information loss or feature distortion that may be introduced by the joint encoder, and improve the quality of the final sleep state feature representation.

[0055] In particular, in the technical solution of the present application, it is considered that the user's breathing rate, micro-motion signal and body motion signal are all parameters closely related to the physiological state. These signals may provide similar information in some aspects. For example, micro-motion signals and body motion signals may reflect similar physiological changes, resulting in repeated information in the feature vector. In different feature extraction modules (such as temporal encoders, autoencoders, residual dual attention mechanisms, etc.), similar features may be extracted. For example, the extraction process of micro-motion feature maps and body motion feature maps may capture the same physiological phenomenon, resulting in an increase in redundant features. In the feature fusion stage, the fusion of micro-motion feature maps and body motion feature maps may not effectively eliminate redundant information, resulting in repeated features in the micro-motion body motion associated feature map of the monitored user. Although the pooling operation can reduce the feature dimension, if there is already a large amount of redundant information in the original feature, the redundant features may still be retained after pooling, affecting the validity of subsequent features. However, the sleep classification feature vector of the user to be monitored has feature redundancy, which leads to the problem of inter-category squeeze relative to the class label of the classifier when it is classified and judged by the classifier, affecting the classification iteration effect. Therefore, in the technical solution of the present application, an optimized sleep classification feature vector of the user to be monitored is obtained by performing feature screening optimization based on set aggregation bounded deviation on the sleep classification feature vector of the user to be monitored.

[0056] Among them, the sleep classification feature vector of the user to be monitored is subjected to feature screening optimization based on set aggregate bounded deviation to obtain an optimized sleep classification feature vector of the user to be monitored, including: extracting each row vector in the prediction parameter space of the classifier to obtain a set of sleep prediction parameter row vectors of the user to be monitored; calculating the class probability entropy degradation simulation value between the sleep classification feature vector of the user to be monitored and each sleep prediction parameter row vector of the user to be monitored in the set of sleep prediction parameter row vectors of the user to be monitored to obtain a set of sleep class probability entropy degradation simulation values ​​of the user to be monitored; calculating the square root of the sum of squares of the sleep class probability entropy degradation simulation values ​​of the user to be monitored to obtain a sleep entropy degradation modulation value of the user to be monitored; based on the sleep entropy degradation modulation value of the user to be monitored, performing feature distribution modulation on the sleep classification feature vector of the user to be monitored to obtain the optimized sleep classification feature vector of the user to be monitored.

[0057] The optimization formula is:

[0058] P={p1,p2,...,p j ,...,p n}

[0059] D j =Softmax(M m v c +H m p j +b m )

[0060]

[0061] Among them, P represents the set of row vectors of sleep prediction parameters of the user to be monitored, p1, p2, p j 、p n represents the first, second, jth and nth sleep prediction parameter row vectors of the user to be monitored in the set of sleep prediction parameter row vectors of the user to be monitored, v c represents the sleep classification feature vector of the user to be monitored, Softmax represents the normalized exponential function, M m represents the first weight matrix, H m represents the second weight matrix, b m represents the bias vector, D j represents the set of the jth set of the simulated values ​​of the sleep class probability entropy degradation of the user to be monitored, t represents the modulation value of the sleep entropy degradation of the user to be monitored, v c ⊙t Indicates the calculation of the sleep classification feature vector v of the user to be monitored c Each eigenvalue of the t-th power, v cirepresents the i-th eigenvalue of the sleep classification feature vector of the user to be monitored, α represents a predetermined hyperparameter, ⊕ represents addition by position, and V′ represents the optimized sleep classification feature vector of the user to be monitored.

[0062] Specifically, in the technical solution of the present application, the entropy degradation of the sleep classification feature vector of the user to be monitored is maximized based on the prediction parameter space of the classifier. First, each row vector is extracted from the prediction parameter space of the classifier to form a row vector set. Then, the class probability entropy degradation simulation value between the sleep classification feature vector of the user to be monitored and each row vector in the above row vector set is calculated to measure the uncertainty or information entropy change of the sleep classification feature vector of the user to be monitored under given parameters. Furthermore, all the sleep class probability entropy degradation simulation values ​​of the user to be monitored are squared and summed, and then the square root is taken to obtain a comprehensive sleep entropy degradation modulation value of the user to be monitored. The sleep entropy degradation modulation value of the user to be monitored reflects the degree of entropy change of the entire system when processing the feature vector. Finally, the sleep classification feature vector of the user to be monitored is adjusted according to the sleep entropy degradation modulation value of the user to be monitored obtained above. Specifically, the attribute set of the sleep classification feature vector of the user to be monitored is represented in a power series based on the sleep entropy degradation modulation value of the user to be monitored, and a set aggregation bounded deviation represented by an aggregation radius based on each attribute value of the sleep classification feature vector of the user to be monitored is added, so as to perform attribute selection and sparse optimization on the sleep classification feature vector of the user to be monitored under a unified limit standard of the sleep entropy degradation modulation value of the user to be monitored, so as to ensure that the sleep classification feature vector of the user to be monitored contains sufficient data but is not too cumbersome so as to introduce unnecessary interference or redundancy.

[0063] In a specific embodiment of the present application, the sleep feature classification unit 143 of the user to be monitored is used to: use the classifier to process the optimized sleep classification feature vector of the user to be monitored using the following formula to obtain the classification result; wherein the formula is:

[0064] O=softmax{(W c ,B c )|Project(M)}

[0065] Wherein Project(M) represents the optimized sleep classification feature vector of the user to be monitored, W c is the weight matrix, B c represents a bias vector, softmax represents a normalized exponential function, and O represents the classification result.

[0066] In summary, the remote monitoring system for sleep status based on telemedicine according to the embodiment of the present application is explained, which first uses a wireless respiratory sensor to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points and uses an infrared sensor to collect the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored as input data, and then uses machine learning technology to perform deep convolution encoding and analysis on these input data to obtain classification results, and the classification results are used to indicate whether there is an abnormality in the sleep state of the user to be monitored. In this way, remote monitoring and analysis of the human body's sleep state can be achieved. In this way, it is helpful to better manage the user's sleep quality and health.

[0067] As described above, the remote monitoring system 100 for sleep status based on telemedicine according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for remote monitoring of sleep status based on telemedicine. In one example, the remote monitoring system 100 for sleep status based on telemedicine according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the remote monitoring system 100 for sleep status based on telemedicine can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the remote monitoring system 100 for sleep status based on telemedicine can also be one of the many hardware modules of the wireless terminal.

[0068] Alternatively, in another example, the telemedicine-based sleep status remote monitoring system 100 and the wireless terminal may also be separate devices, and the telemedicine-based sleep status remote monitoring system 100 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0069] Figure 5 FIG. 1 is a flowchart of a remote monitoring method for sleep status based on telemedicine according to an embodiment of the present application. Figure 5As shown, the remote monitoring method of sleep status based on telemedicine according to the embodiment of the present application includes: S1, collecting the human respiratory frequency of the user to be monitored, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored at multiple predetermined time points; S2, respectively extracting and analyzing the human respiratory frequency of the user to be monitored, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored at multiple predetermined time points to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored; S3, using a joint encoder to perform feature combination of the human respiratory frequency feature vector and the micro-motion body motion associated feature vector of the user to be monitored to obtain a sleep feature matrix of the user to be monitored; S4, obtaining a classification result based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is an abnormality in the sleep state of the user to be monitored.

[0070] Here, those skilled in the art can understand that the specific functions and operations of each step in the above-mentioned sleep state remote monitoring method based on telemedicine have been referred to above. Figure 1 The invention has been introduced in detail in the description of the remote monitoring system of sleep state based on telemedicine, and therefore, its repeated description will be omitted.

[0071] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices, apparatuses and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0072] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0073] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0074] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0075] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. Secondary words are used to indicate names, but not to indicate any particular order.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit of the technical solution of the present invention.

Claims

1. A sleep status remote monitoring system based on telemedicine, characterized in that: include: A sleep state data acquisition module, used to collect the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body movement signal of the user to be monitored; A sleep state feature extraction module, used to extract and analyze the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body movement signal of the user to be monitored, so as to obtain a human respiratory frequency feature vector and a micro-motion body movement associated feature vector of the user to be monitored; A sleep state feature fusion module, used for combining the human respiratory frequency feature vector and the micro-motion associated feature vector of the user to be monitored using a joint encoder to obtain a sleep feature matrix of the user to be monitored; The sleep state classification result generating module is used to obtain a classification result based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is any abnormality in the sleep state of the user to be monitored.

2. The sleep state remote monitoring system based on telemedicine according to claim 1, characterized in that: The sleep state feature extraction module comprises: A human respiratory frequency feature extraction unit, configured to arrange the human respiratory frequencies of the user to be monitored at a plurality of predetermined time points according to the time dimension into a human respiratory frequency input vector, and then obtain the human respiratory frequency feature vector through a human respiratory frequency feature extraction module based on a time series encoder; The user micro-motion body movement associated feature extraction unit is used to perform feature encoding on the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored to obtain the micro-motion body movement associated feature vector of the user to be monitored.

3. The sleep state remote monitoring system based on telemedicine according to claim 2, characterized in that: The human body breathing frequency feature extraction unit is used to: Use the fully connected layer of the temporal encoder to perform fully connected encoding on the human respiratory frequency input vector to extract high-dimensional implicit features of the feature values ​​at each position in the human respiratory frequency input vector; as well as The one-dimensional convolution layer of the temporal encoder is used to perform one-dimensional convolution encoding on the human respiratory frequency input vector to extract high-dimensional implicit correlation features of the correlation between the feature values ​​at each position in the human respiratory frequency input vector.

4. The sleep state remote monitoring system based on telemedicine according to claim 3 is characterized in that: The user micro-motion body motion associated feature extraction unit comprises: A signal denoising unit, configured to pass the micromotion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored through a signal denoising module based on an automatic encoder to obtain a micromotion signal of the user to be monitored after denoising and a body motion signal of the user to be monitored after denoising; A micro-motion feature extraction unit, used for passing the waveform of the micro-motion signal of the user to be monitored after noise reduction through a micro-motion feature extraction module based on a residual double attention mechanism model to obtain a micro-motion feature map of the user to be monitored; A body motion feature extraction unit, used for passing the waveform of the noise-reduced body motion signal of the user to be monitored through a body motion feature extraction module based on convolution kernels of different scales to obtain a body motion feature map of the user to be monitored; A micro-motion body movement association unit, used for performing feature fusion on the micro-motion feature map of the user to be monitored and the body movement feature map of the user to be monitored to obtain a micro-motion body movement association feature map of the user to be monitored; The associated feature map pooling unit is used to perform a pooling operation on the associated feature map of the micro-motion body movement of the user to be monitored to obtain the associated feature vector of the micro-motion body movement of the user to be monitored.

5. The sleep state remote monitoring system based on telemedicine according to claim 4, characterized in that: The signal noise reduction unit is used for: Using the feature encoder of the signal noise reduction module to extract a plurality of signal features from the micro-motion signal caused by the heartbeat of the user to be monitored and the body motion signal of the user to be monitored; and The feature decoder of the signal noise reduction module is used to decode the multiple signal features to obtain the noise-reduced micro-motion signal of the user to be monitored and the noise-reduced body movement signal of the user to be monitored.

6. The sleep state remote monitoring system based on telemedicine according to claim 5, characterized in that: The sleep state feature fusion module is used to: use the joint encoder to jointly encode the human respiratory frequency feature vector and the micro-motion associated feature vector of the user to be monitored using the following formula to generate a sleep feature matrix of the user to be monitored; Wherein, the formula is: in Represents vector multiplication, M a represents the sleep feature matrix of the user to be monitored, V a1 represents the human respiratory frequency feature vector, V a2 represents the micro-motion associated feature vector of the user to be monitored, represents the transpose of the micro-motion associated feature vector of the user to be monitored.

7. The sleep state remote monitoring system based on telemedicine according to claim 6, characterized in that: The sleep state classification result generating module comprises: A sleep feature extraction unit for a user to be monitored, used for passing the sleep feature matrix of the user to be monitored through a sleep feature extraction module for the user to be monitored based on a convolutional neural network model to obtain a sleep classification feature vector for the user to be monitored; A sleep feature optimization unit for the user to be monitored, used for performing feature screening optimization based on set aggregate bounded deviation on the sleep classification feature vector of the user to be monitored to obtain an optimized sleep classification feature vector of the user to be monitored; The sleep feature classification unit of the user to be monitored is used to pass the optimized sleep classification feature vector of the user to be monitored through a classifier to obtain the classification result.

8. The sleep state remote monitoring system based on telemedicine according to claim 7, characterized in that: The sleep characteristic optimization unit of the user to be monitored includes: Extracting each row vector in the prediction parameter space of the classifier to obtain a set of row vectors of sleep prediction parameters of the user to be monitored; Calculating the class probability entropy degradation simulation value between the sleep classification feature vector of the user to be monitored and each sleep prediction parameter row vector of the user to be monitored in the set of sleep prediction parameter row vectors of the user to be monitored to obtain a set of sleep class probability entropy degradation simulation values ​​of the user to be monitored; Calculating the square root of the sum of squares of the sleep class probability entropy degradation simulation values ​​of the user to be monitored to obtain the sleep entropy degradation modulation value of the user to be monitored; Based on the sleep entropy degradation modulation value of the user to be monitored, feature distribution modulation is performed on the sleep classification feature vector of the user to be monitored to obtain the optimized sleep classification feature vector of the user to be monitored.

9. A remote monitoring method for sleep status based on telemedicine, characterized in that ,include: Collecting the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored; Extracting and analyzing the human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored; Using a joint encoder, the human respiratory frequency feature vector and the micro-motion associated feature vector of the user to be monitored are combined to obtain a sleep feature matrix of the user to be monitored; A classification result is obtained based on the sleep feature matrix of the user to be monitored, and the classification result is used to indicate whether there is an abnormality in the sleep state of the user to be monitored.

10. The sleep state remote monitoring method based on telemedicine according to claim 9, characterized in that: The human respiratory frequency of the user to be monitored at multiple predetermined time points, the micro-motion signal caused by the heartbeat of the user to be monitored, and the body motion signal of the user to be monitored are respectively extracted and analyzed to obtain a human respiratory frequency feature vector and a micro-motion body motion associated feature vector of the user to be monitored, including: A human respiratory frequency feature extraction unit, configured to arrange the human respiratory frequencies of the user to be monitored at a plurality of predetermined time points according to the time dimension into a human respiratory frequency input vector, and then obtain the human respiratory frequency feature vector through a human respiratory frequency feature extraction module based on a time series encoder; The user micro-motion body movement associated feature extraction unit is used to perform feature encoding on the micro-motion signal caused by the heartbeat of the user to be monitored and the body movement signal of the user to be monitored to obtain the micro-motion body movement associated feature vector of the user to be monitored.

Images