A healthy sleep monitoring management method and system

Abstract

The application provides a healthy sleep monitoring management method and system, and relates to the field of artificial intelligence health. The method comprises the following steps: acquiring real-time sleep image data; inputting the real-time sleep image data into a pre-trained human eye key point detection model to obtain first feature data; inputting the real-time sleep image data into a pre-trained human body skeleton detection and recognition model to obtain second feature data; aligning the first feature data and the second feature data in the time domain according to a first preset period to obtain third feature data; inputting the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information comprises wake state information, light sleep state information and deep sleep state information; and obtaining sleep posture information according to the deep sleep state information. The application can realize unobtrusive monitoring and management of sleep state and sleep posture.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence in health, and in particular to a method and system for monitoring and managing healthy sleep. Background Technology

[0002] Sleep plays a vital role in human health. In today's fast-paced society, people have higher expectations for sleep quality due to the accelerated pace of life. To improve sleep quality, new technologies can effectively help people monitor and manage their sleep. With the development of smart hardware, sleep management through wearable smart devices has become integrated into people's lives; however, this method suffers from inconvenience and limited functionality.

[0003] Therefore, how to achieve unobtrusive monitoring and management of sleep state and sleeping posture is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method for monitoring and managing sleep patterns, enabling unobtrusive monitoring and management of sleep states and postures. This application also provides a health sleep monitoring and management system with the same technical effects.

[0005] The first objective of this application is to provide a method for monitoring and managing healthy sleep.

[0006] The aforementioned objective of this application is achieved through the following technical solution:

[0007] A method for monitoring and managing healthy sleep includes:

[0008] Acquire real-time sleep image data;

[0009] The real-time sleep image data is input into a pre-trained human eye key point detection model to obtain the first feature data;

[0010] The real-time sleep image data is input into a pre-trained human skeleton detection and recognition model to obtain the second feature data.

[0011] According to the first preset period, the first feature data and the second feature data are time-domain aligned to obtain the third feature data;

[0012] The third feature data is input into a pre-trained sleep state classification model to obtain sleep state information, which includes wakefulness state information, light sleep state information and deep sleep state information.

[0013] Sleeping posture information is obtained based on the deep sleep state information.

[0014] Preferably, in the healthy sleep monitoring and management method, the pre-trained human eye key point detection model is obtained through the following steps:

[0015] Based on human eye key points, the SCRFD model is improved to obtain a face key point detection model, wherein SCRFD represents an efficient face detection algorithm based on sample and computational redistribution;

[0016] First historical sleep image data is acquired, and the facial landmark detection model is trained using the first historical sleep image data to obtain a trained facial landmark detection model.

[0017] Preferably, in the healthy sleep monitoring and management method, the pre-trained human skeleton detection and recognition model is obtained through the following steps:

[0018] Acquire second historical sleep image data, and annotate the second historical sleep image data based on key points of body parts to obtain annotated second historical sleep image data;

[0019] Using the labeled second historical sleep image data, a deep learning-based pedestrian skeleton recognition algorithm is trained to obtain a trained human skeleton detection and recognition model.

[0020] Preferably, in the healthy sleep monitoring and management method, the pre-trained sleep state classification model is obtained through the following steps:

[0021] Acquire third-generation historical sleep image data;

[0022] The third historical sleep image data is input into the pre-trained human eye key point detection model to obtain the fourth feature data;

[0023] The third historical sleep image data is input into the pre-trained human skeleton detection and recognition model to obtain the fifth feature data;

[0024] According to the first preset period, the fourth feature data and the fifth feature data are time-domain aligned to obtain the sixth feature data;

[0025] The sixth feature data is labeled based on preset rules to obtain labeled sixth feature data;

[0026] Using the labeled sixth feature data, a three-layer classification neural network based on deep learning is trained to obtain a trained sleep state classification model.

[0027] Preferably, in the healthy sleep monitoring and management method, the step of annotating the sixth feature data based on preset rules to obtain the annotated sixth feature data includes:

[0028] Calculate the aspect ratio of the human eye at the key points of the human eye in each frame of image data in the sixth feature data;

[0029] Determine whether the aspect ratio of the human eye in a consecutive preset number of image frames in the sixth feature data is less than a first preset value.

[0030] If so, the consecutive preset number of image frames will be marked as a sleep state;

[0031] If not, then the consecutive preset number of image frames will be marked as a conscious state;

[0032] Calculate the center offset of the coordinate vector of the human skeleton in a series of preset frames of image data labeled as sleep state;

[0033] Determine whether the center offset of the coordinate vector is less than a second preset value.

[0034] If so, then the consecutive preset number of image frames labeled as sleep state will be labeled as deep sleep state.

[0035] If not, then the consecutive preset number of image data frames marked as sleep state will be marked as light sleep state;

[0036] Based on a series of pre-defined frames of image data labeled as waking state, deep sleep state, and light sleep state, the sixth feature data after labeling is obtained.

[0037] Preferably, in the healthy sleep monitoring and management method, the step of aligning the first feature data and the second feature data in the time domain according to a first preset period to obtain the third feature data includes:

[0038] The preset number of frames of images are set as a period to obtain a first preset period. Based on the first preset period, the first feature data and the second feature data are fused in chronological order to obtain the third feature data.

[0039] Accordingly, the step of aligning the fourth feature data and the fifth feature data in the time domain according to the first preset period to obtain the sixth feature data includes:

[0040] The preset number of frames of images are set as a period to obtain a first preset period. Based on the first preset period, the fourth feature data and the fifth feature data are fused in chronological order to obtain the sixth feature data.

[0041] Preferably, in the healthy sleep monitoring and management method, obtaining sleeping posture information based on the deep sleep state information includes:

[0042] Based on the deep sleep state information, output the image of the deep sleep state and save the corresponding skeleton coordinate information;

[0043] Calculate the normalized variance of the skeletal coordinates of the longest deep sleep period during the second preset cycle.

[0044] The variance is compared with a preset variance threshold, and the comparison result is used to determine whether the sleeping posture is the same each time, thereby obtaining sleeping posture information.

[0045] Preferably, in the healthy sleep monitoring and management method, after obtaining the sleeping posture information based on the deep sleep state information, the method further includes:

[0046] A sleep report is generated based on the sleep state information and the sleep posture information;

[0047] The sleep report is then transmitted to the user's terminal.

[0048] Preferably, in the healthy sleep monitoring and management method, after obtaining the sleeping posture information based on the deep sleep state information, the method further includes:

[0049] Based on the sleeping posture information, recommended sleeping posture information is obtained.

[0050] The recommended sleeping position information is transmitted to the user's terminal.

[0051] The second objective of this application is to provide a health sleep monitoring and management system.

[0052] The second objective of this application is achieved through the following technical solution:

[0053] A health sleep monitoring and management system, comprising:

[0054] Image acquisition unit, used to acquire real-time sleep image data;

[0055] The first extraction unit is used to input the real-time sleep image data into a pre-trained human eye key point detection model to obtain the first feature data;

[0056] The second extraction unit is used to input the real-time sleep image data into a pre-trained human skeleton detection and recognition model to obtain the second feature data.

[0057] The feature fusion unit aligns the first feature data and the second feature data in the time domain according to a first preset period to obtain the third feature data;

[0058] The sleep classification unit is used to input the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information includes wakefulness state information, light sleep state information and deep sleep state information.

[0059] The sleeping posture analysis unit obtains sleeping posture information based on the deep sleep state information.

[0060] The above technical solution, based on real-time sleep image data, utilizes a human eye key point detection model and a human skeleton detection and recognition model to obtain first and second feature data. Through temporal alignment, the first and second feature data are fused to obtain third feature data, which is beneficial for subsequent sleep state classification. This third feature data is then input into a sleep state classification model to obtain sleep state information, and based on deep sleep state information, sleep posture information is obtained. This enables health monitoring and management of sleep states without wearable devices, effectively helping people improve their sleeping posture and sleep quality. In summary, the above technical solution enables imperceptible monitoring and management of sleep states and sleeping postures. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating a health sleep monitoring and management method provided in an embodiment of this application.

[0063] Figure 2 This is a schematic diagram of six key points of the human eye in an embodiment of this application;

[0064] Figure 3 This is a schematic diagram of 14 key points of body parts in the embodiments of this application;

[0065] Figure 4 This is a schematic diagram of the structure of a health sleep monitoring and management system provided in the embodiments of this application. Detailed Implementation

[0066] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] In the embodiments provided in this application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described below are merely illustrative. For example, the division of units and modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or modules can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0068] In addition, each functional unit in the various embodiments of this application can be integrated into a single processor, or each unit can be a separate device, or two or more units can be integrated into a single device; each functional unit in the various embodiments of this application can be implemented in hardware or in the form of hardware plus software functional units.

[0069] Those skilled in the art will understand that all or part of the steps of the following method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the following method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0070] It should be understood that the use of terms such as "system," "device," "unit," and / or "module" in this application is merely one method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0071] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" or "several" means two or more, unless otherwise explicitly specified.

[0072] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0073] It should also be noted that, in this document, terms such as “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the article or device that includes the aforementioned element.

[0074] The embodiments in this application are written in a progressive manner.

[0075] like Figure 1 As shown in the figure, this application provides a method for monitoring and managing healthy sleep, including:

[0076] S1. Acquire real-time sleep image data;

[0077] In S1, specifically, a network camera can be installed above the bed to ensure full-body monitoring and a relatively clear image of the face. One frame is extracted from the real-time monitoring video of the network camera at preset intervals to obtain real-time sleep image data. Sleep monitoring and management based on real-time sleep image data avoids the inconvenience of wearing smart hardware devices compared to this method, achieving seamless monitoring and management. In other embodiments of this application, real-time sleep image data can also be obtained through other reasonable means; this application is not limited to these.

[0078] S2. Input the real-time sleep image data into a pre-trained human eye key point detection model to obtain the first feature data;

[0079] In S2, the pre-trained eye keypoint detection model can extract eye keypoints from a face image. These eye keypoints reflect the level of eye opening, which can be used to assess a person's sleep state. The resulting first feature data contains sleep state-related information. The eye keypoint detection model can employ a deep learning-based computer face keypoint detection algorithm. To achieve better feature extraction results, the pre-trained eye keypoint detection model can be obtained through the following steps:

[0080] S21. Based on human eye key points, the SCRFD model is improved to obtain a face key point detection model, wherein the SCRFD represents an efficient face detection algorithm based on sample and computational redistribution;

[0081] In S21, the key points of the human eye can be the six key points of the human eye, which can be referred to in detail. Figure 2 In the diagram, P1 to P6 represent the 2D keypoint locations of the human eye. Based on the detection of these six keypoints, the level of eye opening can be predicted more reliably. SCRFD, proposed by InsightFace in 2021, is a face detection model. Its full name is Sample and Computation Redistribution for Effificient Face Detection. In Chinese, it refers to an efficient face detection algorithm based on sample and computational redistribution. "Sample" mainly refers to data augmentation strategies that enhance the model's detection of small faces during training by enlarging them. "Computation" refers to methods to reduce the model's search space. "Redistribution" means reallocation. "Effificient" refers to the ability to redistribute data. FaceDetection refers to efficient face detection. Based on six key points of the human eye, the SCRFD face detection algorithm is improved to obtain a face key point detection model. Specifically, in the final feature map output layer of the SCRFD algorithm, a human eye key point regression branch is added. The L2 loss function is used to regress the face key points to obtain the face key point detection model. The purpose is to modify the network output so that the algorithm only focuses on 12 key points in the eye region of the face, thereby obtaining better feature extraction results.

[0082] S22. Acquire first historical sleep image data, and use the first historical sleep image data to train the facial landmark detection model to obtain a trained facial landmark detection model;

[0083] In S22, the acquisition method of the first historical sleep image data can refer to the above S1. The specific image acquisition period and quantity can be determined according to the actual model training requirements. Based on the first historical sleep image data, the facial landmark detection model is trained to obtain the trained facial landmark detection model.

[0084] S3. Input the real-time sleep image data into the pre-trained human skeleton detection and recognition model to obtain the second feature data;

[0085] In S3, the pre-trained human skeleton detection and recognition model can extract key points of the human skeleton from a full-body image. Changes in the position of these key points reflect changes in human posture, which can be used to assess sleep depth. The resulting second feature data contains information related to sleep depth. The human skeleton detection and recognition model can employ a deep learning-based pedestrian skeleton recognition algorithm. The execution order of S3 and S2 can be interchanged without affecting the implementation of this embodiment. To obtain better feature extraction results, the pre-trained human eye key point detection model can be obtained through the following steps:

[0086] S31. Acquire second historical sleep image data, and annotate the second historical sleep image data based on key points of body parts to obtain annotated second historical sleep image data;

[0087] In S31, the method for acquiring the second historical sleep image data can refer to S1 above. The specific image acquisition period and number can be determined according to the actual model training requirements. The body part key points can use 14 body part key points, which can be specifically referred to... Figure 3 Fourteen key points on body parts, including the head, neck, shoulders, elbows, hands, hips, knees, and feet, are used to perfectly fit the human body and better reflect changes in posture. Based on these 14 key points, the second historical sleep image data is annotated to obtain an annotated second historical sleep image data. Specifically, this can be achieved using automatic annotation software, or other reasonable methods; this application is not limited to these. The annotated second historical sleep image data is used for subsequent model training.

[0088] S32. Using the labeled second historical sleep image data, train a deep learning-based pedestrian skeleton recognition algorithm to obtain a trained human skeleton detection and recognition model;

[0089] In S32, model training is performed based on the labeled second historical sleep image data, which can improve the model's feature extraction capability.

[0090] S4. According to the first preset period, the first feature data and the second feature data are time-domain aligned to obtain the third feature data;

[0091] In S4, the first preset period can be determined according to actual needs, for example, using ten frames as one period. The first feature data and the second feature data are fused in chronological order to form a sequence feature. This sequence feature is then used as a set of data to finally obtain the third feature data. The third feature data includes sleep state-related information and sleep depth-related information, which is beneficial for subsequent sleep state classification.

[0092] S5. Input the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information includes wakefulness state information, light sleep state information and deep sleep state information;

[0093] In S5, the pre-trained sleep state classification model can classify sleep states based on the third feature data, outputting three state information: awake, light sleep, and deep sleep. The sleep state classification model can employ a deep learning-based classification neural network. Based on the sleep state information, the time spent in awake, light sleep, and deep sleep states can be recorded and statistically analyzed over time, enabling tracking and statistical analysis of sleep states. To improve the model's classification and recognition capabilities, the pre-trained sleep state classification model is obtained through the following steps:

[0094] S51. Obtain third historical sleep image data;

[0095] S52. Input the third historical sleep image data into the pre-trained human eye key point detection model to obtain the fourth feature data;

[0096] S53. Input the third historical sleep image data into the pre-trained human skeleton detection and recognition model to obtain the fifth feature data;

[0097] S54. According to the first preset period, the fourth feature data and the fifth feature data are time-domain aligned to obtain the sixth feature data;

[0098] The method for acquiring the third historical sleep image data described in S51 can refer to S1 above. The specific image acquisition period and quantity can be determined according to the actual model training requirements. For the specific implementation details of S52-S54, please refer to S2-S4 above.

[0099] S55. Based on preset rules, the sixth feature data is labeled to obtain the labeled sixth feature data;

[0100] In S55, the preset rules can be formulated based on human eye key point detection and human skeleton detection. According to the preset rules, the sixth feature data can be labeled to obtain sixth feature data labeled with the three sleep states of wakefulness, light sleep, and deep sleep. The specific labeling process can be implemented using automatic labeling software, or it can be labeled using other reasonable methods; this application is not limited to these. One implementation method of this step may include the following steps:

[0101] S551. Calculate the aspect ratio of the human eye at the key points of the human eye in each frame of image data in the sixth feature data;

[0102] In S551, the aspect ratio of the human eye is used to describe the degree of eye opening in each frame image, and its specific calculation formula is as follows:

[0103]

[0104] In the formula, EAR represents the aspect ratio of the human eye, and p1 to p6 represent the coordinates of key points on the human eye. When the eyes are open, the value of EAR remains almost unchanged, but when the eyes are closed, EAR is almost zero.

[0105] S552. Determine whether the aspect ratio of the human eye in the sixth feature data of a series of preset frames is less than the first preset value. If yes, mark the series of preset frames as a sleep state; if no, mark the series of preset frames as a wakeful state.

[0106] In S552, the preset number of frames and the first preset value can be determined according to actual training needs. For example, the preset number of frames may be ten frames, and the first preset value may be 0.2. In other embodiments of this application, one implementation of the step of aligning the first feature data and the second feature data in the temporal domain according to the first preset period to obtain the third feature data specifically includes: setting the preset number of frames as one period to obtain the first preset period, and fusing the first feature data and the second feature data in chronological order according to the first preset period to obtain the third feature data. Correspondingly, one implementation of the step of aligning the fourth feature data and the fifth feature data in the temporal domain according to the first preset period to obtain the sixth feature data specifically includes: setting the preset number of frames as one period to obtain the first preset period, and fusing the fourth feature data and the fifth feature data in chronological order according to the first preset period to obtain the sixth feature data. That is, the number of image frames fused in each period during the feature data fusion process is consistent with the number of image frames continuously determined during the sleep state annotation process of the feature data, thereby obtaining better annotation results.

[0107] S553. Calculate the center offset of the coordinate vector of the human skeleton in a series of preset frames of image data labeled as sleep state;

[0108] In S553, the coordinates of the human skeleton can be represented by the coordinates of key points of the body parts; based on the offset of the coordinate vector center, the range of change in human posture can be reflected.

[0109] S554. Determine whether the center offset of the coordinate vector is less than the second preset value. If yes, mark the consecutive preset number of image data frames marked as sleep state as deep sleep state. If no, mark the consecutive preset number of image data frames marked as sleep state as light sleep state.

[0110] In S554, the second preset value can be determined according to actual training needs; by calculating the offset of the coordinate vector center and comparing it with the second preset value, the sleep depth of a person is evaluated based on the change range of human posture, thereby further subdividing and labeling the sleep state into light sleep state and deep sleep state.

[0111] S555. Based on a series of preset frames of image data labeled as waking state, deep sleep state, and light sleep state, obtain the labeled sixth feature data;

[0112] In S555, the sixth feature data after annotation is obtained based on a number of consecutive preset frames of image data that have been labeled as awake, deep sleep and light sleep states.

[0113] S56. Using the labeled sixth feature data, train a three-layer classification neural network based on deep learning to obtain a trained sleep state classification model;

[0114] In S56, each neuron in the neural network can be viewed as a logistic regression model. A three-layer neural network is a composite of three logistic regression models. Generally, the input layer and hidden layers have multiple neurons, while the output layer corresponds to a logistic regression unit, a softmax unit, or a linear regression model. The sixth feature data can include the coordinates of 12 key points of the eyes and the coordinates of 14 key points of body parts. One specific implementation of this step is as follows: the above 26 coordinate pairs can be used as input, a three-layer classification neural network can be used, Tanh can be used as the activation function, and softmax can be used as the classification output. The network is divided into three categories: awake, light sleep, and deep sleep. The model is trained in this way to obtain a trained sleep state classification model.

[0115] S6. Obtain sleeping posture information based on the deep sleep state information.

[0116] In S6, specifically, an image of the deep sleep state can be obtained based on the deep sleep state information. Based on the image data and sleep duration of the deep sleep state, various sleeping positions in the current deep sleep state can be distinguished or merged to obtain sleeping position information. One implementation of this step may include the following steps: outputting an image of the deep sleep state and saving the corresponding skeleton coordinate information based on the deep sleep state information; calculating the normalized variance of the skeleton coordinate information with the longest deep sleep duration in each period within a second preset period; comparing the variance with a preset variance threshold, and determining whether the sleeping positions are the same each time based on the comparison result, thereby obtaining sleeping position information. The preset variance threshold can be determined according to actual needs; the second preset period can be determined according to actual needs, for example, it can be calculated that the normalized variance of the skeleton coordinate information with the longest deep sleep duration in each sleep period within a one-month period; based on the above comparison results, when the sleeping positions are determined to be the same, the same sleeping positions can be merged; when the sleeping positions are determined to be different, the different sleeping positions can be saved, ultimately obtaining sleeping position information.

[0117] With the development of smart hardware, sleep management through wearable smart hardware devices has been integrated into people's lives. However, this method has problems such as inconvenience in wearing and limited functionality.

[0118] The above embodiments, based on real-time sleep image data, utilize a human eye key point detection model and a human skeleton detection and recognition model to obtain first and second feature data. Through temporal alignment, the first and second feature data are fused to obtain third feature data, which is beneficial for subsequent sleep state classification. The third feature data is input into a sleep state classification model to obtain sleep state information, and based on deep sleep state information, sleep posture information is obtained. This enables health monitoring and management of sleep states without wearable devices and in a seamless manner, effectively helping people improve their sleeping posture and sleep quality. In summary, the above embodiments can achieve seamless monitoring and management of sleep states and sleeping postures.

[0119] Based on the above embodiments, another embodiment of this application also provides another method for monitoring and managing healthy sleep, including:

[0120] S101. Acquire real-time sleep image data;

[0121] S102. Input the real-time sleep image data into a pre-trained human eye key point detection model to obtain the first feature data;

[0122] S103. Input the real-time sleep image data into a pre-trained human skeleton detection and recognition model to obtain the second feature data;

[0123] S104. According to the first preset period, the first feature data and the second feature data are time-domain aligned to obtain the third feature data;

[0124] S105. Input the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information includes wakefulness state information, light sleep state information and deep sleep state information;

[0125] S106. Obtain sleeping posture information based on the deep sleep state information.

[0126] For details regarding S101-S106, please refer to the above-mentioned S1-S6.

[0127] S107. Generate a sleep report based on the sleep state information and the sleep posture information;

[0128] In S107, specifically, based on the sleep state information and the sleeping posture information, the time of wakefulness, light sleep, and deep sleep, as well as the sleep time and images of each sleeping posture, can be recorded and statistically analyzed to form a statistical report as a sleep report; the sleep report can also be presented in other reasonable ways, and this application is not limited thereto.

[0129] S108. Based on the sleeping posture information, recommended sleeping posture information is obtained;

[0130] In step S108, specifically, the deep sleep duration of various sleeping positions within a third preset period can be calculated based on the sleeping position information; the sleeping position with the longest deep sleep duration is selected as the recommended sleeping position, thus obtaining recommended sleeping position information; the third preset period can be determined according to actual needs, for example, the deep sleep duration of various sleeping positions can be statistically analyzed over a one-month period. The recommended sleeping position information can be provided to the user for reference in adjusting their sleeping position. The execution order of S108 and S107 can be interchanged without affecting the implementation of this embodiment.

[0131] S109. The sleep report and the recommended sleeping position information are transmitted to the user terminal.

[0132] In S109, the user terminal can be a smartphone, and the sleep report and the recommended sleeping position information can be displayed through a smartphone app for the user to view.

[0133] In this embodiment, by analyzing and processing the sleep state information and the sleeping posture information, a sleep report and recommended sleeping posture information are obtained and pushed to the user terminal for display. This can help people track and statistically analyze their sleep state and monitor their sleeping posture, improve their understanding of their own sleep state, and guide them to improve their sleep quality.

[0134] like Figure 4As shown, in another embodiment of this application, a health sleep monitoring and management system is also provided, including:

[0135] Image acquisition unit 11 is used to acquire real-time sleep image data;

[0136] The first extraction unit 12 is used to input the real-time sleep image data into a pre-trained human eye key point detection model to obtain the first feature data;

[0137] The second extraction unit 13 is used to input the real-time sleep image data into a pre-trained human skeleton detection and recognition model to obtain second feature data.

[0138] The feature fusion unit 14 aligns the first feature data and the second feature data in the time domain according to a first preset period to obtain the third feature data;

[0139] The sleep classification unit 15 is used to input the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information includes wakefulness state information, light sleep state information and deep sleep state information.

[0140] The sleeping posture analysis unit 16 obtains sleeping posture information based on the deep sleep state information.

[0141] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A healthy sleep monitoring management method, characterized in that, The method comprises the following steps: obtaining real-time sleep image data; inputting the real-time sleep image data into a pre-trained human eye key point detection model to obtain first feature data; inputting the real-time sleep image data into a pre-trained human body skeleton detection and recognition model to obtain second feature data; aligning the first feature data and the second feature data in the time domain according to a first preset period to obtain third feature data; inputting the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information comprises wake state information, light sleep state information and deep sleep state information; obtaining sleep posture information according to the deep sleep state information; The pre-trained sleep state classification model is obtained through the following steps: obtaining third historical sleep image data; inputting the third historical sleep image data into the pre-trained human eye key point detection model to obtain fourth feature data; inputting the third historical sleep image data into the pre-trained human body skeleton detection and recognition model to obtain fifth feature data; aligning the fourth feature data and the fifth feature data in the time domain according to the first preset period to obtain sixth feature data; labeling the sixth feature data based on a preset rule to obtain labeled sixth feature data; training a three-layer classification neural network based on deep learning by using the labeled sixth feature data to obtain a trained sleep state classification model; The labeling of the sixth feature data based on the preset rule to obtain the labeled sixth feature data comprises the following steps: calculating the eye length-width ratio of the human eye key point in each frame of image data in the sixth feature data; determining whether the eye length-width ratio of the continuous preset number of frames of image data in the sixth feature data is less than a first preset value, if yes, labeling the continuous preset number of frames of image data as sleep state; if no, labeling the continuous preset number of frames of image data as wake state; calculating the coordinate vector center offset of the human body skeleton of the continuous preset number of frames of image data labeled as sleep state; determining whether the coordinate vector center offset is less than a second preset value, if yes, labeling the continuous preset number of frames of image data labeled as sleep state as deep sleep state, if no, labeling the continuous preset number of frames of image data labeled as sleep state as light sleep state; obtaining the labeled sixth feature data according to the continuous preset number of frames of image data labeled as wake state, deep sleep state and light sleep state.

2. The method as claimed in claim 1, characterized in that, The pre-trained human eye key point detection model is obtained through the following steps: improving an SCRFD model based on human eye key points to obtain a human face key point detection model, wherein the SCRFD represents a high-efficiency face detection algorithm based on sample and calculation redistribution; obtaining first historical sleep image data and training the human face key point detection model by using the first historical sleep image data to obtain a trained human face key point detection model.

3. The method as claimed in claim 1, wherein, The pre-trained human body skeleton detection and recognition model is obtained through the following steps: Obtain second historical sleep image data, and label the second historical sleep image data based on the body part key points to obtain labeled second historical sleep image data; Train a pedestrian skeleton recognition algorithm based on deep learning using the labeled second historical sleep image data to obtain a trained human skeleton detection and recognition model.

4. The method as claimed in claim 1, wherein, The first feature data and the second feature data are aligned in time domain according to the first preset period to obtain third feature data, including: The preset number of frames of images is set as one period to obtain a first preset period, and the first feature data and the second feature data are fused in time sequence according to the first preset period to obtain third feature data; Correspondingly, the fourth feature data and the fifth feature data are aligned in time domain according to the first preset period to obtain sixth feature data, including: The preset number of frames of images is set as one period to obtain a first preset period, and the fourth feature data and the fifth feature data are fused in time sequence according to the first preset period to obtain sixth feature data.

5. The method as claimed in claim 1, wherein, The deep sleep state information is used to obtain sleep posture information, including: According to the deep sleep state information, an image of the deep sleep state is outputted and the corresponding skeleton coordinate information is saved; The normalized variance of the skeleton coordinate information with the longest deep sleep time in a second preset period is calculated; The variance is compared with a preset variance threshold, and whether the sleep posture is the same each time is determined according to the comparison result to obtain sleep posture information.

6. The method as claimed in claim 1, wherein, After the deep sleep state information is used to obtain sleep posture information, it further includes: According to the sleep state information and the sleep posture information, a sleep report is formed; The sleep report is delivered to a user terminal.

7. The method as claimed in claim 1, wherein, After the deep sleep state information is used to obtain sleep posture information, it further includes: According to the sleep posture information, recommended sleep posture information is obtained; The recommended sleep posture information is delivered to a user terminal.

8. A healthy sleep monitoring management system, characterized by, Applied to the health sleep monitoring and management method of claim 1, including: An image acquisition unit is configured to acquire real-time sleep image data; A first extraction unit is configured to input the real-time sleep image data into a pre-trained human eye key point detection model to obtain first feature data; A second extraction unit is configured to input the real-time sleep image data into a pre-trained human skeleton detection and recognition model to obtain second feature data; A feature fusion unit is configured to align the first feature data and the second feature data in time domain according to a first preset period to obtain third feature data; A sleep classification unit is configured to input the third feature data into a pre-trained sleep state classification model to obtain sleep state information, wherein the sleep state information includes wake state information, light sleep state information and deep sleep state information; A sleep posture analysis unit is configured to obtain sleep posture information according to the deep sleep state information.

Images