Sleep behavior monitoring method and device, storage medium and terminal

Abstract

The invention discloses a sleep behavior monitoring method and device, a storage medium and a terminal, and the method comprises the steps: obtaining a cover image corresponding to a sleep event video, the sleep event video being a video collected for a sleep event in a current scene through a sleep monitoring device; the cover image is input into a large action recognition model, sleep actions output by the large action recognition model based on the cover image are determined, and the sleep actions are normal sleep actions or quilt kicking actions; and when the sleep action is a quilt kicking action, generating first notification information corresponding to the quilt kicking event, and sending the first notification information to a target terminal bound with the sleep monitoring equipment, so that the target terminal displays the first notification information. And the server can analyze the cover image by using the action recognition large model, so that the sleep action of the user is obtained. In this way, when the user kicks the quilt, the corresponding prompt notification information can be sent to the bound target terminal in time, and the sleep experience of family members is improved.

Description

Technical Field

[0001] This application relates to the field of behavior recognition and monitoring technology, and in particular to a sleep behavior monitoring method, device, storage medium, and terminal. Background Technology

[0002] In the field of pediatric sleep monitoring, children commonly exhibit unconscious limb movements during sleep, often resulting in kicking off the covers with varying amplitude, frequency, and direction. This leads to a persistently high incidence of colds and chills among children. Traditional monitoring methods rely heavily on constant parental attention, but parents face challenges during nighttime monitoring: frequent visits to the child's room disrupt sleep continuity, and the scheduled check-in mechanism is insufficient to promptly detect the child kicking off the covers. Furthermore, this results in parents being in a state of light sleep for extended periods, causing fragmented sleep for caregivers and impacting the sleep quality of other family members. Summary of the Invention

[0003] This application provides a sleep behavior monitoring method, device, storage medium, and terminal, which can solve the technical problem in related technologies that it is difficult to pay attention to children's kicking off the blankets in a timely manner.

[0004] In a first aspect, embodiments of this application provide a sleep behavior monitoring method applied to a server, the method comprising:

[0005] Obtain the cover image corresponding to the aforementioned sleep event video, which is a video collected by a sleep monitoring device for sleep events in the current scenario;

[0006] The above cover image is input into the action recognition model to determine the sleep action output by the action recognition model based on the above cover image. The sleep action is either a normal sleep action or a kicking-off action.

[0007] When the aforementioned sleep action is the aforementioned kicking off the blanket action, a first notification message corresponding to the kicking off the blanket event is generated and the aforementioned first notification message is sent to the target terminal bound to the aforementioned sleep monitoring device, so that the aforementioned target terminal displays the aforementioned first notification message.

[0008] In one possible implementation, the above-mentioned acquisition of the cover image corresponding to the sleep event video includes: acquiring sleep event videos collected by the sleep monitoring device at preset intervals for sleep events, and acquiring the cover image corresponding to each sleep event video.

[0009] In one possible implementation, after generating the first notification information corresponding to the kicking-off-the-blanket event and sending the first notification information to the target terminal bound to the sleep monitoring device, the method further includes: when the action recognition big data model recognizes the normal sleep action based on the cover image of the newly added sleep event video, generating the second notification information corresponding to the covering-off-the-blanket event and sending the second notification information to the target terminal bound to the sleep monitoring device, so that the target terminal displays the second notification information.

[0010] In one possible implementation, the step of generating a first notification message corresponding to the blanket-kicking action when the sleep action is the blanket-kicking action and sending the first notification message to the target terminal bound to the sleep monitoring device includes: when the sleep action is the blanket-kicking action, determining whether the previous output result of the action recognition model is the blanket-kicking action; if the previous output result is the blanket-kicking action, ignoring the current output result; if the previous output result is the normal sleep action, generating a first notification message corresponding to the blanket-kicking action and sending the first notification message to the target terminal bound to the sleep monitoring device.

[0011] In one possible implementation, determining the sleep action output by the action recognition big model based on the cover image includes: controlling the action recognition big model to recognize the target human figure outline and the target quilt outline of the user based on the cover image, and determining the user's sleep action based on the target human figure outline and the target quilt outline.

[0012] In one possible implementation, the method further includes: constructing an initial action recognition model for sleep scenarios based on a base model; acquiring multiple sample cover images, wherein the multiple sample cover images are sample data with standard sleep action labels; inputting the multiple sample cover images into the initial action recognition model to train the initial action recognition model; during the training process of the initial action recognition model, controlling the initial action recognition model to output predicted sleep action labels for the multiple sample cover images, calculating a training loss value based on the predicted sleep action labels and the standard sleep action labels of the multiple sample cover images, and adjusting the parameters of the initial action recognition model based on the training loss value until the initial action recognition model converges to obtain the trained action recognition model.

[0013] Secondly, embodiments of this application provide a sleep behavior monitoring method, applied to a sleep monitoring device, the method comprising:

[0014] Collect sleep event videos for the current scenario, and extract cover images from the sleep event videos;

[0015] The aforementioned cover image is sent to the server, so that the server uses an action recognition big data model to output sleep actions based on the aforementioned cover image, and when the aforementioned sleep action is the action of kicking off the blanket, generates a first notification information corresponding to the kicking off the blanket event and sends the aforementioned first notification information to the target terminal bound to the aforementioned sleep monitoring device to display the aforementioned first notification information.

[0016] In one possible implementation, the above-mentioned extraction of the cover image from the sleep event video includes: extracting the pixel features of each frame of the sleep event video, and determining the cover image corresponding to the sleep event video based on the temporal features of each frame and the pixel features of each frame.

[0017] In one possible implementation, the above-mentioned acquisition of sleep event video for sleep events in the current scenario includes: in response to a video acquisition setting instruction triggered by a user, acquiring sleep event video for sleep events in the current scenario once every preset time interval as indicated in the video acquisition setting instruction.

[0018] Thirdly, embodiments of this application provide a sleep behavior monitoring device applied to a server, the device comprising:

[0019] The cover acquisition module is used to acquire the cover image corresponding to the aforementioned sleep event video, which is a video collected by a sleep monitoring device for sleep events in the current scenario.

[0020] The action recognition module is used to input the cover image into the action recognition big model, determine the sleep action output by the action recognition big model based on the cover image, and determine whether the sleep action is a normal sleep action or a kicking-off action.

[0021] The information notification module is used to generate a first notification message corresponding to the blanket-kicking event when the aforementioned sleep action is the aforementioned blanket-kicking action, and send the first notification message to the target terminal bound to the aforementioned sleep monitoring device, so that the target terminal displays the aforementioned first notification message.

[0022] Fourthly, embodiments of this application provide a sleep behavior monitoring device, applied to a sleep monitoring equipment, the device comprising:

[0023] The data acquisition module is used to collect sleep event videos for the current scenario and extract cover images from the sleep event videos.

[0024] The data transmission module is used to send the cover image to the server, so that the server uses an action recognition big data model to output sleep actions based on the cover image, and when the sleep action is kicking off the blanket, generates a first notification message corresponding to the kicking off the blanket event and sends the first notification message to the target terminal bound to the sleep monitoring device to display the first notification message.

[0025] Fifthly, embodiments of this application provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps of the method described above.

[0026] In a sixth aspect, embodiments of this application provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being adapted to be loaded by the processor and to execute the steps of the method described above.

[0027] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0028] This application provides a method for monitoring sleep behavior. The method involves acquiring a cover image corresponding to a sleep event video, where the sleep event video is captured by a sleep monitoring device for a sleep event in the current scenario. The cover image is input into a motion recognition model to determine the sleep action output by the model based on the cover image. The sleep action is either a normal sleep action or a kicking-off action. When the sleep action is a kicking-off action, a first notification message corresponding to the kicking-off event is generated and sent to a target terminal bound to the sleep monitoring device, so that the target terminal displays the first notification message. Because the motion recognition model can intelligently analyze the cover image of the video to accurately identify the user's actions while sleeping in the current scenario, after the sleep monitoring device captures and uploads the cover image of the video to the server, the server can use the motion recognition model to analyze the cover image to obtain the user's sleep actions, thereby determining whether the user has kicked off the covers. By accurately analyzing images based on a large model, timely notifications can be sent to the bound target terminal when a user kicks off the blanket. This allows guardians to readjust the blankets for children without constantly monitoring their sleep, preventing health risks from unconscious sleep activities and improving the sleep experience for all family members. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 An exemplary system architecture diagram of a sleep behavior monitoring method provided in this application embodiment;

[0031] Figure 2 A schematic flowchart of a sleep behavior monitoring method provided in an embodiment of this application;

[0032] Figure 3 A schematic flowchart of a sleep behavior monitoring method provided in an embodiment of this application;

[0033] Figure 4 A flowchart illustrating a model training method for a large-scale action recognition model provided in this application embodiment;

[0034] Figure 5 A schematic flowchart of a sleep behavior monitoring method provided in an embodiment of this application;

[0035] Figure 6 A structural block diagram of a sleep behavior monitoring device provided in an embodiment of this application;

[0036] Figure 7 A structural block diagram of a sleep behavior monitoring device provided in an embodiment of this application;

[0037] Figure 8 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application.

[0038] Figure 9 This is a schematic diagram of the structure of a sleep monitoring device provided in an embodiment of this application. Detailed Implementation

[0039] To make the features and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; the word "and / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, in the description of the embodiments of this application, "multiple" refers to two or more.

[0041] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0042] In children's sleep, kicking off the covers has a significant impact on their health. Because children's nervous systems are not yet fully developed, their ability to control limb movements during sleep is weak. Coupled with their high metabolism and relatively large body surface area, kicking off the covers causes rapid heat loss, easily leading to a drop in body temperature and triggering illnesses such as colds. Current monitoring methods mainly rely on parental manual checks, but this has significant drawbacks. Parents need to frequently get up at night to check on their children, disrupting their own sleep cycles and causing fragmented sleep, affecting their work and life the next day. Furthermore, relying solely on experience for checks cannot provide real-time and timely intervention. Even with traditional video surveillance equipment, only static images or basic dynamic monitoring are provided, lacking intelligent recognition capabilities for kicking off the covers. Parents still need to actively review the monitoring content, failing to fundamentally solve the problems of timeliness and continuity of monitoring.

[0043] Therefore, this application provides a sleep behavior monitoring method to solve the aforementioned technical problem of difficulty in timely monitoring children's kicking off of blankets.

[0044] Please see Figure 1 , Figure 1 This is an exemplary system architecture diagram of an event classification method provided in an embodiment of this application.

[0045] like Figure 1As shown, the system architecture may include a server 101, a network 102, and a sleep monitoring device 103. The network 102 serves as the medium for providing a communication link between the server 101 and the sleep monitoring device 103. The network 102 may include various types of wired or wireless communication links, such as wired communication links including fiber optic cables, twisted-pair cables, or coaxial cables, and wireless communication links including Bluetooth communication links, Wireless-Fidelity (Wi-Fi) communication links, or microwave communication links, etc.

[0046] Server 101 can interact with sleep monitoring device 103 via network 102 to receive or send messages to sleep monitoring device 103. Alternatively, server 101 can interact with sleep monitoring device 103 via network 102 to receive messages or data sent to sleep monitoring device 103 by other users. Server 101 can be hardware or software. When server 101 is hardware, it can be various electronic devices, including but not limited to smartwatches, smartphones, tablets, laptops, and desktop computers. When server 101 is software, it can be installed in the aforementioned electronic devices and can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module; no specific limitation is made here.

[0047] The sleep monitoring device 103 can be a business device providing various services. When the sleep monitoring device 103 is hardware, it can be implemented as a distributed device cluster composed of multiple servers, or as a single device. In this embodiment, the sleep monitoring device 103 is a device with image and video acquisition functions.

[0048] In this embodiment, when implementing the sleep behavior monitoring method, the sleep monitoring device 130 first collects sleep event videos for the current scenario and extracts a cover image from the sleep event videos; then, the sleep monitoring device 130 sends the cover image to the server 101. The server 101 obtains the cover image; further, the server 101 inputs the cover image into the action recognition big data model, determines the sleep action output by the action recognition big data model based on the cover image, and determines whether the sleep action is a normal sleep action or a kicking-off action; finally, when the sleep action is a kicking-off action, the server 101 generates a first notification information corresponding to the kicking-off event and sends the first notification information to the target terminal bound to the sleep monitoring device, so that the target terminal displays the first notification information.

[0049] It should be understood that Figure 1 The number of servers, networks, and sleep monitoring devices shown is only illustrative; the number of servers, networks, and sleep monitoring devices can be any number depending on the implementation needs.

[0050] Please see Figure 2 , Figure 2 This is a flowchart illustrating a sleep behavior monitoring method provided in an embodiment of this application. The execution entity in this embodiment can be a terminal performing sleep behavior monitoring, a processor within the terminal performing the sleep behavior monitoring method, or a sleep behavior monitoring service within the terminal performing the sleep behavior monitoring method. For ease of description, the following uses the processor within the terminal as an example to describe the specific execution process of the sleep behavior monitoring method.

[0051] like Figure 2 As shown, a sleep behavior monitoring method applied to a server may include at least:

[0052] S202. Obtain the cover image corresponding to the sleep event video. The sleep event video is a video collected by a sleep monitoring device for sleep events in the current scenario.

[0053] Optionally, to facilitate monitoring of users' sleep patterns, users can use sleep monitoring devices at home to collect videos of sleep events in the current scenario. These videos allow users to understand the events occurring in each time period. Within these videos, the sleep monitoring device considers multiple dimensions such as information content, feature quantity, importance, and representativeness to select the most suitable frame as the video cover, enabling users to quickly understand the video content from the cover. Correspondingly, to improve the reliability of sleep action recognition results, the cover image corresponding to the sleep event video can be obtained to better identify user actions within the video content.

[0054] Optionally, the cover image can be preprocessed before action recognition. In one feasible implementation, noise reduction can be performed first using a bilateral filtering algorithm to suppress Gaussian noise while preserving edge information. Then, adaptive histogram equalization can be used to enhance image contrast and improve detail performance in low-light environments. Finally, image normalization can be used to generate floating-point tensors that meet the input requirements of subsequent models.

[0055] S204. Input the cover image into the action recognition model, and determine the sleep action output by the action recognition model based on the cover image. The sleep action is either a normal sleep action or a kicking-off-the-cover action.

[0056] Optionally, when performing action recognition on the cover image, deep learning technology can be used to train a large action recognition model. The model can then be used to process various complex cover images to identify whether the target user's sleep action is a normal sleep action or an action of kicking off the blanket.

[0057] Optionally, the cover image can be input into the action recognition model. The model utilizes the powerful feature extraction and learning capabilities of deep neural networks to perform in-depth analysis of the cover image. Through multi-level, multi-scale feature extraction, the model can capture user actions within the image. The action recognition model analyzes the pixel information of the image to identify the target user's body parts and their movements, as well as the outline of the blanket, thereby determining whether the target user has kicked off the blanket and promptly sending alerts to the linked terminal.

[0058] In one feasible implementation, the action recognition big data model is built on a convolutional neural network architecture. When the cover image is input into the model, the action recognition big data model first captures key features such as the amplitude of limb swings (e.g., knee bending angle change ≥45°) and the displacement of the blanket edge contour (using the GrabCut algorithm to segment the blanket region and calculate the centroid displacement ≥10cm) through the human body contour in the image. Then, it outputs two types of probability values ​​through a fully connected layer: when the confidence score of the blanket-kicking action reaches a certain threshold, the sleep action is judged as the blanket-kicking action; otherwise, the sleep action is considered a normal sleep action. This confidence threshold is optimized through clinical testing to balance the false positive rate and the false negative rate.

[0059] S206. When the sleep action is kicking off the blanket, generate the first notification information corresponding to the blanket kicking event and send the first notification information to the target terminal bound to the sleep monitoring device so that the target terminal displays the first notification information.

[0060] Optionally, when the sleep action is kicking off the blanket, a first notification message corresponding to the blanket-kicking event is generated and sent to the target terminal bound to the sleep monitoring device, so that the target terminal displays the first notification message. The first notification message uses a structured data format and includes various information such as the event timestamp, action type code, confidence score, environmental risk level, and suggested handling measures. The notification push module supports a dual-channel transmission mechanism, including pushing to a dedicated monitoring application on the parent's mobile terminal via the MQTT protocol, and simultaneously backing up transmission via SMS.

[0061] In one feasible implementation, the target terminal can be a parent's terminal, including smartphones, smart control screens, wearable devices, etc. Upon receiving the first notification information, it is parsed and rendered on the interactive display interface. In addition to the notification information, quick operation entry points can be provided, such as "view full video" or "remotely adjust room temperature." Users can jump to the video viewing interface or the room temperature adjustment interface by triggering these components. On the other hand, the system can also automatically achieve intelligent linkage between sleep action recognition results and the sleep environment, such as automatically adjusting the bedroom temperature based on the blanket-kicking event (e.g., turning on the air conditioner to a suitable temperature), thereby providing more timely and intelligent monitoring functions and forming a closed-loop sleep health management system. This application embodiment achieves intelligent recognition and immediate response to children kicking off blankets through high-precision visual perception hardware design, anti-interference deep learning model construction, and reliable IoT communication protocols. The feature engineering methods, decision optimization mechanisms, and cross-platform interaction logic employed effectively improve the system's robustness in complex sleep scenarios, providing an engineeringable implementation path for the health monitoring functions of smart home devices.

[0062] This application provides a method for monitoring sleep behavior. The method involves acquiring a cover image corresponding to a sleep event video, where the sleep event video is a video captured by a sleep monitoring device for a sleep event in the current scenario. The cover image is input into a motion recognition model to determine the sleep action output by the model based on the cover image. The sleep action is either a normal sleep action or a kicking-off action. When the sleep action is a kicking-off action, a first notification message corresponding to the kicking-off event is generated and sent to a target terminal bound to the sleep monitoring device, so that the target terminal displays the first notification message. Since the motion recognition model can intelligently analyze the cover image of the video to accurately identify the actions of a user sleeping in the current scenario, after the sleep monitoring device captures and uploads the cover image of the video to the server, the server can use the motion recognition model to analyze the cover image to obtain the user's sleep action and determine whether the user has kicked off the covers. By accurately analyzing images based on a large model, timely notifications can be sent to the bound target terminal when a user kicks off the blanket. This allows guardians to readjust the blankets for children without constantly monitoring their sleep, preventing health risks from unconscious sleep activities and improving the sleep experience for all family members.

[0063] Please see Figure 3 , Figure 3 This is a flowchart illustrating a sleep behavior monitoring method provided in an embodiment of this application.

[0064] like Figure 3As shown, a sleep behavior monitoring method applied to a server may include at least:

[0065] S302. Acquire sleep event videos collected by the sleep monitoring device at preset intervals for each sleep event, and acquire the cover image corresponding to each sleep event video.

[0066] Optionally, when monitoring sleep events, the sleep monitoring device can capture video at regular intervals, reducing storage pressure while ensuring the data integrity of important events. The server will then retrieve the sleep event videos captured by the sleep monitoring device at preset intervals and obtain the cover image corresponding to each sleep event video.

[0067] S304. Input the cover image into the motion recognition model, control the motion recognition model to recognize the user's target human figure outline and target quilt outline based on the cover image, and determine the user's sleeping action based on the target human figure outline and target quilt outline.

[0068] In a specific implementation, the cover image is input into the action recognition model. The model identifies the user's target human figure silhouette and the target blanket silhouette based on the cover image, and determines the user's sleeping action based on these silhouettes. In the action determination logic, the model calculates the intersection-union ratio (IoU) between the human figure silhouette and the blanket silhouette. When the IoU value is below a certain threshold, it triggers the identification of kicking off the blanket. Simultaneously, it can calculate the change in limb angle using skeletal keypoint coordinates. For example, when the angle between a unilateral limb and the bed surface exceeds 45°, the action type is further subdivided by combining this with the direction of blanket silhouette deformation. To improve robustness in complex scenarios, the model also incorporates a self-supervised learning module, periodically collecting real-world scenario data authorized by the user to continuously optimize the model, ensuring stable and reliable accuracy in kicking off the blanket.

[0069] In one feasible implementation, in order to reduce the false alarm problem of the model, a multimodal verification mechanism can be introduced: when a suspected kicking of the blanket is detected, a 10-second video clip before and after the cover image is automatically retrieved from the sleep event video for secondary verification. The final judgment result is output based on the features shown by the dynamic continuous image frames.

[0070] S306. When the sleep action is kicking off the blanket, determine whether the previous output of the action recognition model was the kicking off the blanket action.

[0071] Optionally, considering that after the initial detection of the blanket-kicking action, only one blanket-kicking event needs to be reported until the blanket is properly covered, when the action recognition big model identifies the sleeping action in the current cover image as the blanket-kicking action, it can first determine whether the previous output of the action recognition big model was also the blanket-kicking action.

[0072] S308. If the previous output result is a kicking-off-the-blanket action, then ignore the current output result; if the previous output result is a normal sleep action, then generate the first notification information corresponding to the kicking-off-the-blanket event and send the first notification information to the target terminal bound to the sleep monitoring device.

[0073] Optionally, if the previous output result was a kicking-off action, it means that the blanket was not properly covered after the last kicking-off, rather than a new kicking-off event. In this case, the current output result can be ignored. Conversely, if the previous output result was a normal sleep action, it means that the blanket changed from being properly covered to being kicked off, i.e., a new kicking-off event occurred. In this case, it is necessary to generate the first notification information corresponding to the kicking-off event and send the first notification information to the target terminal bound to the sleep monitoring device.

[0074] S310. When the motion recognition big model recognizes normal sleep actions based on the cover image of the newly added sleep event video, it generates a second notification message corresponding to the blanket covering event and sends the second notification message to the target terminal bound to the sleep monitoring device so that the target terminal displays the second notification message.

[0075] Optionally, after detecting a blanket-kicking event, if the motion recognition model identifies normal sleep movements based on the cover image of the newly added sleep event video, it indicates that a blanket-covering event has occurred and the blanket-kicking event has been resolved. In this case, a second notification message corresponding to the blanket-covering event can be generated and sent to the target terminal bound to the sleep monitoring device, so that the target terminal displays the second notification message. The information type and display method of the second notification message are similar to those of the first notification message, and therefore will not be described further.

[0076] This application provides a sleep behavior monitoring method applied to a server. It acquires sleep event videos collected by a sleep monitoring device at preset intervals for each sleep event, and obtains a cover image corresponding to each sleep event video, reducing storage pressure while ensuring data integrity for important events. In the action judgment logic, the action recognition model calculates the intersection-union ratio (IU) of the human silhouette and the blanket silhouette. When the IU value is lower than a certain threshold, it triggers the identification of a kicking-the-blanket action. If the model output result of the previous round is a kicking-the-blanket action, it means that no new kicking-the-blanket event has occurred, and the output result of this round can be ignored. Conversely, if the output result of the previous round is a normal sleep action, it means that a new kicking-the-blanket event has occurred, and a first notification message corresponding to the kicking-the-blanket event needs to be generated and sent to the target terminal bound to the sleep monitoring device. When the action recognition model identifies a normal sleep action based on the cover image of the newly added sleep event video, it generates a second notification message corresponding to a covering-the-blanket event and sends the second notification message to the target terminal bound to the sleep monitoring device, so that the target terminal displays the second notification message. This application embodiment achieves intelligent recognition and immediate response to children kicking off their blankets through high-precision visual perception hardware design, anti-interference deep learning model construction, and reliable IoT communication protocol. The feature engineering methods, decision optimization mechanisms, and cross-platform interaction logic adopted effectively improve the robustness of the system in complex sleep scenarios, providing an engineeringable implementation path for the health monitoring function of smart home devices.

[0077] Please see Figure 4 , Figure 4 This is a flowchart illustrating a model training method for a large-scale action recognition model provided in an embodiment of this application.

[0078] like Figure 4 As shown, the model training methods for large-scale action recognition models can include at least:

[0079] S402. Construct an initial action recognition model for sleep scenarios based on the basic large model.

[0080] Optionally, the action recognition big model needs to classify the cover image based on multiple features of the cover image. In this case, a pre-trained basic big model can be obtained. The basic big model can output prediction results for the prediction object based on multiple different types of features of the prediction object. Therefore, by building an initial action recognition big model for the sleep scene based on the basic big model, the initial action recognition big model can also obtain the ability to "output prediction results for the prediction object based on multiple different types of features of the prediction object".

[0081] Optionally, when the basic large model is directly applied to a specific scenario, the unadjusted large model is difficult to adapt to the new scenario. Therefore, after building the initial action recognition large model for the sleep scenario, it is also necessary to train the initial action recognition large model in a targeted manner.

[0082] S404. Obtain multiple sample cover images, all of which are sample data with standard sleep action labels.

[0083] Optionally, multiple sample cover images are acquired for training the initial action recognition model. These sample cover images are sample data with standard sleep action labels. The standard sleep action label is the correct sleep action corresponding to each sample cover image, and therefore can be used as the standard label for the sample cover images. The sample cover images contain image data of different sleep actions of the user, including images of normal sleep actions, i.e., the user is in a quiet sleep with a relatively stable body posture; and images of kicking off the covers.

[0084] S406. Input multiple sample cover images into the initial action recognition model and train the initial action recognition model.

[0085] S408. During the training process of the initial action recognition model, the initial action recognition model is controlled to output predicted sleep action labels for multiple sample cover images. The training loss value is calculated based on the predicted sleep action labels and the standard sleep action labels of multiple sample cover images. The parameters of the initial action recognition model are adjusted according to the training loss value until the initial action recognition model converges, thus obtaining the trained action recognition model.

[0086] Optionally, a transfer learning strategy is employed during model training. The initial action recognition model outputs predicted sleep action labels for multiple sample cover images. These predicted sleep action labels represent the sleep action prediction results of the initial action recognition model for the multiple sample cover images. The difference between the predicted sleep action labels and the standard sleep action labels represents the difference between the current state and the expected performance of the initial action recognition model. Based on this, the training loss value of the model can be calculated using the predicted sleep action labels and the standard sleep action labels of the multiple sample cover images. The parameters in the initial action recognition model are then adjusted based on the training loss value until the initial action recognition model converges, resulting in the trained action recognition model. Furthermore, the model's training termination conditions may include conditions such as the loss function value meeting the target value condition or the number of iterations reaching a preset threshold. Specific training termination conditions can be determined based on actual circumstances and are not specifically limited here.

[0087] This application provides a method for training a large-scale action recognition model. A large-scale action recognition model is constructed using a basic large-scale model, and then specifically trained for sleep scenarios. This enables the large-scale action recognition model to perform in-depth analysis of the cover image of sleep event videos and output accurate sleep action recognition results.

[0088] Please see Figure 5 , Figure 5 This is a flowchart illustrating a sleep behavior monitoring method provided in an embodiment of this application.

[0089] like Figure 5 As shown, a sleep behavior monitoring method applied to a sleep monitoring device may include at least:

[0090] S502. Collect sleep event videos for the current scenario and extract cover images from the sleep event videos.

[0091] S504. Send the cover image to the server so that the server uses the action recognition big model to output sleep actions based on the cover image, and when the sleep action is kicking off the blanket, generate the first notification information corresponding to the kicking off the blanket event and send the first notification information to the target terminal bound to the sleep monitoring device to display the first notification information.

[0092] Alternatively, in today's increasingly intelligent living environment, smart home devices, with their superior functionality and convenience, have become an indispensable part of modern families. In the field of sleep monitoring, sleep monitoring devices, through high-precision video acquisition capabilities, can acquire videos of users' sleep events in dimly lit environments at night, capturing users' sleep behaviors in real time and using advanced video analysis technology to extract the corresponding event videos. These sleep events may include nighttime awakenings, kicking off the covers, etc. The device intelligently identifies and analyzes the continuously collected data, quickly identifying relevant segments of sleep events that require attention during the user's sleep process and extracting them as event videos.

[0093] Furthermore, users can control the device to capture sleep event videos at preset intervals according to the video capture setting instructions, thereby enabling personalized adjustments to the video capture frequency and volume.

[0094] Optionally, when the sleep monitoring device is deployed in a sleep environment, its installation position is at a suitable distance from the bed surface, and the lens axis forms a certain angle with the normal of the bed surface to ensure complete coverage of the sleep area. Specifically, the sleep monitoring device will collect sleep event videos in real time. Through the built-in dynamic region detection algorithm, it will extract the pixel features of each frame of the sleep event video. Based on the keyframe extraction algorithm built into the edge computing module, according to the temporal features and pixel features of each frame, it will automatically select video segments containing significant motion changes (such as detecting limb displacement exceeding 15 pixels in 3 consecutive frames) and extract the first frame of the segment as the cover image.

[0095] Optionally, the sleep monitoring device further sends the cover image to the server, so that the server uses a motion recognition model to output sleep actions based on the cover image. When the sleep action is kicking off the blanket, the server generates a first notification message corresponding to the kicking off the blanket event and sends the first notification message to the target terminal bound to the sleep monitoring device to display the first notification message. This improves the efficiency of children's sleep health management through intelligent means, while reducing the burden on parents and building a healthier family sleep ecosystem.

[0096] This application provides a sleep behavior monitoring method applied to a sleep monitoring device. The method involves collecting sleep event videos for a given scenario, extracting a cover image from the sleep event videos, and sending the cover image to a server. The server then uses a motion recognition model to output sleep actions based on the cover image. When the sleep action is kicking off the blanket, a first notification is generated corresponding to the blanket-kicking event and sent to a target terminal bound to the sleep monitoring device for display. Users can control the device to collect sleep event videos at preset intervals according to the video capture setting instructions, allowing for personalized adjustments to the video capture frequency and quantity. This intelligent approach improves the efficiency of children's sleep health management, reduces the burden on parents, and builds a healthier family sleep ecosystem.

[0097] Please see Figure 6 , Figure 6 This is a structural block diagram of a sleep behavior monitoring device provided in an embodiment of this application. Figure 6 As shown, the sleep behavior monitoring device 600, applied to a server, includes:

[0098] The cover acquisition module 610 is used to acquire the cover image corresponding to the sleep event video. The sleep event video is a video collected by the sleep monitoring device for sleep events in the current scene.

[0099] The action recognition module 620 is used to input the cover image into the action recognition big model and determine the sleep action output by the action recognition big model based on the cover image. The sleep action is either a normal sleep action or a kicking-off action.

[0100] The information notification module 630 is used to generate a first notification message corresponding to the blanket kicking event when the sleep action is the blanket kicking action, and send the first notification message to the target terminal bound to the sleep monitoring device so that the target terminal displays the first notification message.

[0101] Optionally, the cover acquisition module 610 is also used to acquire sleep event videos collected by the sleep monitoring device at preset intervals for each sleep event, and to acquire the cover image corresponding to each sleep event video.

[0102] Optionally, the information notification module 630 is also used to generate a second notification message corresponding to the blanket-covering event when the action recognition big model recognizes normal sleep actions based on the cover image of the newly added sleep event video, and send the second notification message to the target terminal bound to the sleep monitoring device so that the target terminal displays the second notification message.

[0103] Optionally, the information notification module 630 is further configured to determine whether the previous output result of the action recognition big model is a kicking-the-cover action when the sleep action is a kicking-the-cover action; if the previous output result is a kicking-the-cover action, then ignore the current output result; if the previous output result is a normal sleep action, then generate a first notification message corresponding to the kicking-the-cover event and send the first notification message to the target terminal bound to the sleep monitoring device.

[0104] Optionally, the motion recognition module 620 is also used to control the motion recognition big model to recognize the user's target human figure outline and target quilt outline based on the cover image, and to determine the user's sleeping motion based on the target human figure outline and target quilt outline.

[0105] Optionally, the sleep behavior monitoring device 600 further includes: a model training module, used to construct an initial action recognition model for sleep scenarios based on a basic large model; acquire multiple sample cover images, all of which are sample data with standard sleep action labels; input the multiple sample cover images into the initial action recognition model to train the initial action recognition model; during the training process of the initial action recognition model, control the initial action recognition model to output predicted sleep action labels for the multiple sample cover images, calculate the training loss value based on the predicted sleep action labels and the standard sleep action labels of the multiple sample cover images, and adjust the parameters of the initial action recognition model according to the training loss value until the initial action recognition model converges to obtain the trained action recognition model.

[0106] In this embodiment, a sleep behavior monitoring device is provided, applied to a server. The device includes a cover image acquisition module for acquiring cover images corresponding to sleep event videos, where the sleep event video is a video captured by the sleep monitoring device for a sleep event in the current scenario. An action recognition module is used to input the cover image into an action recognition model to determine the sleep action output by the model based on the cover image; the sleep action is either a normal sleep action or a kicking-off action. An information notification module is used to generate a first notification message corresponding to the kicking-off action when the sleep action is kicking off the blanket, and send the first notification message to a target terminal bound to the sleep monitoring device, so that the target terminal displays the first notification message. Since the action recognition model can intelligently analyze the cover image of the video to accurately identify the actions of a user sleeping in the current scenario, after the sleep monitoring device captures and uploads the cover image of the video to the server, the server can use the action recognition model to analyze the cover image to obtain the user's sleep action, thereby determining whether the user has kicked off the blanket. By accurately analyzing images based on a large model, timely notifications can be sent to the bound target terminal when a user kicks off the blanket. This allows guardians to readjust the blankets for children without constantly monitoring their sleep, preventing health risks from unconscious sleep activities and improving the sleep experience for all family members.

[0107] Please see Figure 7 , Figure 7 This is a structural block diagram of a sleep behavior monitoring device provided in an embodiment of this application. Figure 7 As shown, the sleep behavior monitoring device 700, used in sleep monitoring equipment, includes:

[0108] The data acquisition module 710 is used to acquire sleep event videos for sleep events in the current scenario and extract cover images from the sleep event videos;

[0109] The data transmission module 720 is used to send the cover image to the server so that the server can use the action recognition big model to output sleep actions based on the cover image, and when the sleep action is kicking off the blanket, generate the first notification information corresponding to the kicking off the blanket event and send the first notification information to the target terminal bound to the sleep monitoring device to display the first notification information.

[0110] Optionally, the data acquisition module 710 is also used to extract the pixel features of each frame of the sleep event video and determine the cover image corresponding to the sleep event video based on the temporal features of each frame and the pixel features of each frame.

[0111] Optionally, the data acquisition module 710 is also used to respond to a video acquisition setting command triggered by the user, and to acquire a sleep event video once every preset time according to the preset time indicated in the video acquisition setting command for the sleep event in the current scene.

[0112] This application provides a sleep behavior monitoring device applied to a sleep monitoring equipment. The device includes a data acquisition module for acquiring sleep event videos of sleep events in the current scenario and extracting a cover image from the videos. A data transmission module sends the cover image to a server, enabling the server to use a motion recognition model to output sleep actions based on the cover image. When the sleep action is kicking off the blanket, the module generates a first notification message corresponding to the blanket-kicking event and sends it to a target terminal bound to the sleep monitoring equipment for display. Users can control the device to acquire sleep event videos of the current scenario at preset intervals according to the video acquisition setting instructions, thereby achieving personalized adjustments to the video acquisition frequency and quantity. This intelligent approach improves the efficiency of children's sleep health management, reduces the burden on parents, and builds a healthier family sleep ecosystem.

[0113] This application also provides a computer storage medium that can store multiple instructions adapted for loading by a processor and executing the steps of any of the methods described in the above embodiments.

[0114] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Figure 8 As shown, server 800 may include: at least one server processor 801, at least one network interface 804, user interface 803, memory 805, and at least one communication bus 802.

[0115] The communication bus 802 is used to enable communication between these components.

[0116] The user interface 803 may include a display screen and a camera. Optionally, the user interface 803 may also include a standard wired interface and a wireless interface.

[0117] The network interface 804 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0118] The server processor 801 may include one or more processing cores. The server processor 801 connects to various parts within the server 800 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 805, and by calling data stored in the memory 805. Optionally, the server processor 801 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The server processor 801 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the server processor 801 and may be implemented as a separate chip.

[0119] The memory 805 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 805 may include a non-transitory computer-readable storage medium. The memory 805 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 805 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 805 may also be at least one storage device located remotely from the aforementioned server processor 801. Figure 8 As shown, the memory 805, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a sleep behavior monitoring program.

[0120] exist Figure 8In the server 800 shown, the user interface 803 is mainly used to provide an input interface for the user and to obtain the user's input data; while the server processor 801 can be used to call the sleep behavior monitoring program stored in the memory 805 and specifically perform the following operations:

[0121] Obtain the cover image corresponding to the sleep event video. The sleep event video is a video collected by a sleep monitoring device for sleep events in the current scenario.

[0122] The cover image is input into the action recognition model to determine the sleep action output by the action recognition model based on the cover image. The sleep action is either a normal sleep action or a kicking-off action.

[0123] When the sleep action is kicking off the blanket, a first notification message corresponding to the blanket kicking event is generated and sent to the target terminal bound to the sleep monitoring device so that the target terminal can display the first notification message.

[0124] In some embodiments, when the server processor 801 performs the step of acquiring the cover image corresponding to the sleep event video, it specifically performs the following steps: acquiring sleep event videos collected by the sleep monitoring device at preset intervals for sleep events, and acquiring the cover image corresponding to each sleep event video.

[0125] In some embodiments, after the server processor 801 generates the first notification information corresponding to the kicking-off-the-blanket event and sends the first notification information to the target terminal bound to the sleep monitoring device, it further performs the following steps: when the action recognition big model recognizes normal sleep actions based on the cover image of the newly added sleep event video, it generates the second notification information corresponding to the covering-off-the-blanket event and sends the second notification information to the target terminal bound to the sleep monitoring device, so that the target terminal displays the second notification information.

[0126] In some embodiments, when the server processor 801 generates a first notification message corresponding to the blanket-kicking event and sends the first notification message to the target terminal bound to the sleep monitoring device when the sleep action is the blanket-kicking action, it specifically performs the following steps: when the sleep action is the blanket-kicking action, it determines whether the previous output result of the action recognition model is the blanket-kicking action; if the previous output result is the blanket-kicking action, it ignores the current output result; if the previous output result is a normal sleep action, it generates the first notification message corresponding to the blanket-kicking event and sends the first notification message to the target terminal bound to the sleep monitoring device.

[0127] In some embodiments, when the server processor 801 executes the sleep action output by the action recognition big model based on the cover image, it specifically performs the following steps: controlling the action recognition big model to recognize the user's target human figure outline and target quilt outline based on the cover image, and determining the user's sleep action based on the target human figure outline and target quilt outline.

[0128] In some embodiments, the server processor 801 further performs the following steps: constructing an initial action recognition model for sleep scenarios based on a basic large model; acquiring multiple sample cover images, where each sample cover image is sample data with standard sleep action labels; inputting the multiple sample cover images into the initial action recognition model to train the initial action recognition model; during the training process of the initial action recognition model, controlling the initial action recognition model to output predicted sleep action labels for the multiple sample cover images, calculating the training loss value based on the predicted sleep action labels and the standard sleep action labels of the multiple sample cover images, adjusting the parameters of the initial action recognition model based on the training loss value until the initial action recognition model converges, and obtaining the trained action recognition model.

[0129] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a sleep monitoring device provided in an embodiment of this application. Figure 9 As shown, the sleep monitoring device 900 may include: at least one sleep monitoring device processor 901, at least one network interface 904, user interface 903, memory 905, and at least one communication bus 902.

[0130] The communication bus 902 is used to enable communication between these components.

[0131] The user interface 903 may include a display screen and a camera. Optionally, the user interface 903 may also include a standard wired interface and a wireless interface.

[0132] The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0133] The sleep monitoring device processor 901 may include one or more processing cores. The sleep monitoring device processor 901 connects to various parts within the sleep monitoring device 900 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 905, and by calling data stored in the memory 905. Optionally, the sleep monitoring device processor 901 may be implemented using at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The sleep monitoring device processor 901 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and Modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. Understandably, the aforementioned modem may also be implemented separately as a single chip, rather than being integrated into the sleep monitoring device processor 901.

[0134] The memory 905 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 905 may include a non-transitory computer-readable storage medium. The memory 905 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 905 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 905 may also be at least one storage device located remotely from the aforementioned sleep monitoring device processor 901. Figure 9 As shown, the memory 905, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a sleep behavior monitoring program.

[0135] exist Figure 9In the sleep monitoring device 900 shown, the user interface 903 is mainly used to provide an input interface for the user and to acquire user input data; while the sleep monitoring device processor 901 can be used to call the sleep behavior monitoring program stored in the memory 905 and specifically perform the following operations:

[0136] Collect sleep event videos for sleep events in the current scenario, and extract cover images from the sleep event videos;

[0137] The cover image is sent to the server so that the server uses a large action recognition model to output sleep actions based on the cover image. When the sleep action is kicking off the blanket, the server generates a first notification message corresponding to the kicking off the blanket event and sends the first notification message to the target terminal bound to the sleep monitoring device to display the first notification message.

[0138] In some embodiments, when the sleep monitoring device processor 901 extracts a cover image from a sleep event video, it specifically performs the following steps: extracting the pixel features of each frame of the sleep event video, and determining the cover image corresponding to the sleep event video based on the temporal features of each frame and the pixel features of each frame.

[0139] In some embodiments, when the sleep monitoring device processor 901 performs the following steps when collecting sleep event videos for sleep events in the current scenario: in response to a video collection setting command triggered by the user, it collects sleep event videos for sleep events in the current scenario once every preset time according to the preset time indicated in the video collection setting command.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0141] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0142] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0143] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0144] In addition, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0145] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0146] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0147] The above is a description of a sleep behavior monitoring method, device, storage medium, and terminal provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for monitoring sleep behavior, characterized in that, Applied to a server, the method includes: Obtain the cover image corresponding to the sleep event video, wherein the sleep event video is a video collected by a sleep monitoring device for sleep events in the current scenario; The cover image is input into the action recognition model to determine the sleep action output by the action recognition model based on the cover image. The sleep action is either a normal sleep action or a kicking-off action. When the sleep action is the kicking off the blanket action, a first notification message corresponding to the kicking off the blanket event is generated and the first notification message is sent to the target terminal bound to the sleep monitoring device so that the target terminal displays the first notification message.

2. The method according to claim 1, characterized in that, The step of obtaining the cover image corresponding to the sleep event video includes: Acquire sleep event videos collected by the sleep monitoring device at preset intervals for each sleep event, and acquire the cover image corresponding to each sleep event video.

3. The method according to claim 2, characterized in that, After generating the first notification information corresponding to the blanket-kicking event and sending the first notification information to the target terminal bound to the sleep monitoring device, the method further includes: When the motion recognition big data model recognizes the normal sleep action based on the cover image of the newly added sleep event video, it generates a second notification message corresponding to the blanket covering event and sends the second notification message to the target terminal bound to the sleep monitoring device so that the target terminal displays the second notification message.

4. The method according to claim 2, characterized in that, When the sleep action is the kicking off the blanket, generating a first notification message corresponding to the blanket-kicking event and sending the first notification message to the target terminal bound to the sleep monitoring device includes: When the sleep action is the kicking-off action, determine whether the previous output result of the action recognition model is the kicking-off action; If the output result of the previous round is the action of kicking off the blanket, then ignore the output result of the current round; If the previous output result is the normal sleep action, then a first notification message corresponding to the kicking off the blanket event is generated and the first notification message is sent to the target terminal bound to the sleep monitoring device.

5. The method according to claim 1, characterized in that, The method further includes: An initial action recognition model for sleep scenarios is built based on a fundamental large model; Acquire multiple sample cover images, all of which are sample data with standard sleep action labels; The multiple sample cover images are input into the initial action recognition model to train the initial action recognition model; During the training process of the initial action recognition model, the initial action recognition model is controlled to output predicted sleep action labels for the multiple sample cover images, and a training loss value is calculated based on the predicted sleep action labels and the standard sleep action labels of the multiple sample cover images. The parameters of the initial action recognition model are adjusted according to the training loss value until the initial action recognition model converges, thus obtaining the trained action recognition model.

6. A method for monitoring sleep behavior, characterized in that, The method, applied to a sleep monitoring device, includes: Collect sleep event videos for sleep events in the current scenario, and extract cover images from the sleep event videos; The cover image is sent to the server, so that the server uses an action recognition model to output sleep actions based on the cover image. When the sleep action is kicking off the blanket, the server generates a first notification message corresponding to the kicking off the blanket event and sends the first notification message to the target terminal bound to the sleep monitoring device to display the first notification message.

7. A sleep behavior monitoring device, characterized in that, Applied to a server, the device includes: The cover acquisition module is used to acquire the cover image corresponding to the sleep event video, wherein the sleep event video is a video collected by a sleep monitoring device for sleep events in the current scenario; An action recognition module is used to input the cover image into an action recognition model and determine the sleep action output by the action recognition model based on the cover image, wherein the sleep action is a normal sleep action or a kicking-off action. The information notification module is used to generate a first notification message corresponding to the blanket-kicking event when the sleep action is the blanket-kicking action, and send the first notification message to the target terminal bound to the sleep monitoring device so that the target terminal displays the first notification message.

8. A sleep behavior monitoring device, characterized in that, Applied to sleep monitoring devices, the device includes: The data acquisition module is used to collect sleep event videos for sleep events in the current scenario and extract cover images from the sleep event videos; The data transmission module is used to send the cover image to the server, so that the server uses an action recognition big data model to output sleep actions based on the cover image, and when the sleep action is kicking off the blanket, generates a first notification message corresponding to the kicking off the blanket event and sends the first notification message to the target terminal bound to the sleep monitoring device to display the first notification message.

9. A server, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method as described in any one of claims 1 to 5.

10. A sleep monitoring device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the method as claimed in any one of claims 6.

Images