Sleep state detection method and device, storage medium and terminal

Abstract

The invention discloses a sleep state detection method and device, a storage medium and a terminal. Sleep images collected for a sleep event of a target user are continuously obtained; for each sleep image, inputting the sleep image into a sleep state detection model, and determining the sleep state of the target user output by the sleep state detection model based on the sleep image, the sleep state being a sleep state or a getting-up state; and when the sleep state is a night getting-up state, the night getting-up auxiliary light is automatically turned on. According to the embodiment of the invention, the sleep images of the sleep event are continuously collected, and the sleep state detection model is called for each sleep image to analyze the sleep state of the target user in the image, so that bedding wrinkles and human body limb actions are effectively distinguished, and whether the user is in a sleep state or a getting-up state is judged according to the image features of the sleep images; when it is detected that the sleep state of the user is the night getting-up state, night getting-up auxiliary light is automatically turned on for the user, and the action safety of the user in the night getting-up process is protected.

Description

Technical Field

[0001] The present application relates to the field of behavior recognition detection technology, and in particular to a sleep state detection method, device, storage medium, and terminal. Background Art

[0002] Sleep quality, as a key indicator of human health, directly impacts both physiological and psychological well-being. In today's fast-paced modern world, sleep quality issues are common and diverse. Existing sleep companion solutions often focus on the physical implementation of a single functional module. For example, to address the safety needs of nighttime sleep, traditional solutions typically require users to independently follow a fixed "find the switch - manually operate" process, which can easily lead to secondary risks in dark environments and during emergencies. Existing solutions struggle to meet the safety needs of children and those with limited mobility, especially when waking up at night. Summary of the Invention

[0003] The present application provides a sleep state detection method, device, storage medium and terminal, which can solve the technical problem in related technologies that users have difficulty moving around when getting up at night.

[0004] In a first aspect, an embodiment of the present application provides a sleep state detection method, the method comprising:

[0005] Continuously acquiring sleep images collected from sleep events of a target user;

[0006] For each sleep image, input the sleep image into a sleep state detection model, and determine the sleep state of the target user output by the sleep state detection model based on the sleep image, where the sleep state is either a falling asleep state or a waking up state;

[0007] When the above sleeping state changes to the night-waking state, the night-waking auxiliary light is automatically turned on.

[0008] In a possible embodiment, the above method also includes: in response to a user's viewing instruction for the sleep state detection area, displaying the image acquisition area corresponding to the camera device in the display interface; in response to the above user's triggering of a detection area setting instruction for the target display area in the above image acquisition area, determining the above target display area as the target detection area in the above sleep image; the above determination of the sleep state of the above target user output by the above sleep state detection model based on the above sleep image includes: determining the sleep state of the above target user output by the above sleep state detection model based on the above target detection area in the above sleep image.

[0009] In a possible embodiment, the above method also includes: in response to a detection area automatic setting instruction triggered by a user, capturing a sample image of the current scene through a camera device and identifying the calibration furniture in the above sample image, the above calibration furniture being at least one of a bed, a chair, and a sofa; delineating a target detection area in the above sleep image based on the position of the above calibration furniture in the above sample image; the above determination of the sleep state of the above target user output by the above sleep state detection model based on the above sleep image includes: determining the sleep state of the above target user output by the above sleep state detection model based on the above target detection area in the above sleep image.

[0010] In a possible embodiment, the above-mentioned determination of the sleep state of the target user output by the above-mentioned sleep state detection model based on the above-mentioned target detection area in the above-mentioned sleep image includes: controlling the above-mentioned sleep state detection model to identify whether there is a target human body in the above-mentioned target detection area based on the above-mentioned sleep image, and if the above-mentioned sleep state detection model identifies the above-mentioned target human body, outputting the state of the above-mentioned target user as a waking state.

[0011] In a possible implementation, the target human body is a human outline having a preset ratio to a complete human outline; or, the target human body is a human body with a preset posture.

[0012] In a possible implementation, the method further includes: in response to a user triggering operation for the night-waking assistance function, turning on the night-waking assistance function to capture sleep images of the target user's sleep events through a camera device.

[0013] In a possible embodiment, the above method also includes: determining through a clock whether the current time meets the triggering conditions of a preset timed task, and the above preset timed task is a timed start task pre-set by the user for the night-waking assistance function; if the above current time meets the above triggering conditions, then executing the above preset timed task to turn on the above night-waking assistance function, so as to collect sleep images of the target user's sleep events through a camera device.

[0014] In a possible embodiment, when the above-mentioned sleep state is the state of waking up at night, after automatically turning on the auxiliary light for waking up at night, it also includes: generating notification information corresponding to the above-mentioned sleep event based on the above-mentioned state of waking up at night, and sending the above-mentioned notification information in the bound target terminal according to the preset notification method.

[0015] In a possible embodiment, when the above-mentioned sleeping state is a state of getting up at night, after turning on the auxiliary light for getting up at night, it also includes: when the above-mentioned sleeping state detection model recognizes that the number of times the above-mentioned sleeping state has reached a preset number of consecutive times based on the above-mentioned sleep image, the above-mentioned auxiliary light for getting up at night is automatically turned off.

[0016] In a possible embodiment, the above method also includes: constructing an initial sleep state detection model for sleep scenes based on the basic model; obtaining multiple sample sleep images, and the above multiple sample sleep images are all sample data with standard sleep state labels; inputting the above multiple sample sleep images into the above initial sleep state detection model, and training the above initial sleep state detection model; during the training process of the above initial sleep state detection model, controlling the above initial sleep state detection model to output predicted sleep state labels for the above multiple sample sleep images, and calculating a training loss value based on the above predicted sleep state labels and the standard sleep state labels of the above multiple sample sleep images, and adjusting the parameters of the above initial sleep state detection model according to the above training loss value until the above initial sleep state detection model converges, thereby obtaining a trained sleep state detection model.

[0017] In a second aspect, an embodiment of the present application provides a sleep state detection device, the device comprising:

[0018] An image acquisition module, configured to continuously acquire sleep images collected from sleep events of a target user;

[0019] a state detection module for inputting each sleep image into a sleep state detection model and determining a sleep state of the target user output by the sleep state detection model based on the sleep image, wherein the sleep state is either a falling asleep state or a waking state;

[0020] The night-waking assistance module is used to automatically turn on the night-waking assistance light when the above-mentioned sleeping state changes to the night-waking state.

[0021] In one possible embodiment, the sleep state detection device further includes: a detection area manual setting module for displaying the image acquisition area corresponding to the camera device in the display interface in response to a user's viewing instruction for the sleep state detection area; in response to a detection area setting instruction triggered by the user for the target display area in the image acquisition area, determining the target display area as the target detection area in the sleep image; and a state detection module for determining the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0022] In a possible embodiment, the sleep state detection device further includes: an automatic detection area setting module, which is used to respond to an automatic detection area setting instruction triggered by a user, capture a sample image of the current scene through a camera device and identify calibrated furniture in the sample image, where the calibrated furniture is at least one of a bed, a chair, and a sofa; delineate a target detection area in the sleep image based on the position of the calibrated furniture in the sample image; and a state detection module, which is further used to determine the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0023] In a possible embodiment, the above-mentioned state detection module is also used to control the above-mentioned sleep state detection model to identify whether there is a target human body in the above-mentioned target detection area based on the above-mentioned sleep image, and if the above-mentioned sleep state detection model identifies the above-mentioned target human body, the state of the above-mentioned target user is output as the state of waking up at night.

[0024] In a possible implementation, the target human body is a human outline having a preset ratio to a complete human outline; or, the target human body is a human body with a preset posture.

[0025] In a possible embodiment, the above-mentioned sleep state detection device also includes: a manual function activation module, which is used to activate the above-mentioned night-waking assistance function in response to the user's triggering operation on the night-waking assistance function, so as to collect sleep images of the target user's sleep events through a camera device.

[0026] In a possible embodiment, the above-mentioned sleep state detection device also includes: a function automatic start-up module, which is used to determine through a clock whether the current time meets the triggering conditions of a preset timing task, and the above-mentioned preset timing task is a timed start-up task pre-set by the user for the night-waking assistance function; if the above-mentioned current time meets the above-mentioned triggering conditions, the above-mentioned preset timing task is executed to turn on the above-mentioned night-waking assistance function, so as to collect sleep images of the target user's sleep events through a camera device.

[0027] In a possible implementation, the sleep state detection device further includes: an event notification module for generating notification information corresponding to the sleep event based on the waking state, and sending the notification information to the bound target terminal in a preset notification manner.

[0028] In a possible embodiment, the sleep state detection device further includes: a light-off module for automatically turning off the night-time auxiliary light when the sleep state detection model recognizes that the number of times the sleeping state occurs reaches a preset number of times based on the sleep image.

[0029] In one possible embodiment, the sleep state detection device further includes: a model training module for constructing an initial sleep state detection model for a sleep scene based on a basic model; obtaining a plurality of sample sleep images, wherein the plurality of sample sleep images are sample data with standard sleep state labels; inputting the plurality of sample sleep images into the initial sleep state detection model to train the initial sleep state detection model; during the training process of the initial sleep state detection model, controlling the initial sleep state detection model to output predicted sleep state labels for the plurality of sample sleep images, calculating a training loss value based on the predicted sleep state labels and the standard sleep state labels of the plurality of sample sleep images, and adjusting the parameters of the initial sleep state detection model based on the training loss value until the initial sleep state detection model converges, thereby obtaining a trained sleep state detection model.

[0030] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the above method.

[0031] In a fourth aspect, an embodiment of the present application provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is suitable for being loaded by the processor and executing the steps of the above-mentioned method.

[0032] The beneficial effects of the technical solutions provided by some embodiments of the present application include at least:

[0033] The present application provides a sleep state detection method, which continuously acquires sleep images collected for sleep events of a target user; for each sleep image, the sleep image is input into a sleep state detection model, and the sleep state of the target user output by the sleep state detection model based on the sleep image is determined, and the sleep state is either falling asleep or waking up at night; when the sleep state is waking up at night, the waking up auxiliary light is automatically turned on. The embodiment of the present application continuously collects sleep images for sleep events, and calls the sleep state detection model for each sleep image to analyze the sleep state of the target user in the image, thereby effectively distinguishing between bedding wrinkles and human limb movements, and judging whether the user is falling asleep or waking up at night based on the image features of the sleep image; when it is detected that the user's sleep state is waking up at night, the waking up auxiliary light is automatically turned on for the user to protect the user's movement safety when waking up at night, and establish a reliable dynamic mapping relationship between the device operation state and the physiological indicators. In this way, the target user does not need to manually find the switch when waking up at night, and the system can automatically provide corresponding auxiliary functions according to their sleep state, thereby improving the user's sleep experience and life convenience, and forming an intelligent and comprehensive sleeping companion solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 An exemplary system architecture diagram of a sleep state detection method provided in an embodiment of the present application;

[0036] Figure 2 A flowchart of a sleep state detection method provided in an embodiment of the present application;

[0037] Figure 3 A flowchart of a sleep state detection method provided in an embodiment of the present application;

[0038] Figure 4 A flowchart of a model training method for a sleep state detection model provided in an embodiment of the present application;

[0039] Figure 5 A structural block diagram of a sleep state detection device provided in an embodiment of the present application;

[0040] Figure 6 A schematic diagram of the structure of a terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] To make the features and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0042] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims. And in the description of the embodiments of the present application, unless otherwise indicated, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a way to describe the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" refers to two or more than two.

[0043] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0044] A considerable number of users have the habit of waking up in the middle of the night, and in this scenario, the safety issues of existing lighting solutions are very prominent. At night, the human pupil is in a dilated state, and the sudden strong light will cause a temporary visual blind spot. The completely dark environment makes it difficult for users to identify obstacles on the ground (such as scattered clothes, furniture corners, etc.). Especially for children or the elderly with weaker mobility, accidents such as bumps and falls are very likely to occur. The control switches of traditional lighting equipment are usually set in fixed locations such as the wall at the head of the bed, the wall in the corridor, etc. Finding the switch in a dark environment requires users to have a certain degree of spatial cognition and movement coordination, which is very difficult for those who have not yet fully woken up from sleep. In addition, when children get up alone at night, they are prone to fear because they cannot find the light source in time, which further affects their subsequent sleep quality and forms a vicious cycle.

[0045] With the development of smart home devices, there are also solutions in sleep scenarios that use sensors to detect and respond to users' waking up at night. However, relying on traditional sensors (such as pressure sensors and infrared sensors) is difficult to accurately distinguish between normal turning over and the intention to get up at night, and is prone to misidentification. Therefore, there is an urgent need for an intelligent sleep companion function that can sense the user's sleep state in real time and provide corresponding auxiliary functions to solve the problem of insufficient safety when users get up at night in traditional solutions, and meet the actual demand for intelligent and humanized sleep companion functions in modern home environments.

[0046] Therefore, an embodiment of the present application provides a sleep state detection method to solve the above-mentioned technical problem of inconvenience in users' movements when getting up at night.

[0047] See also Figure 1 , Figure 1 This is an exemplary system architecture diagram of a sleep state detection method provided in an embodiment of the present application.

[0048] like Figure 1 As shown, the system architecture may include a terminal 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the terminal 101 and the server 103. The network 102 may include various types of wired communication links or wireless communication links, for example, a wired communication link may include an optical fiber, a twisted pair, or a coaxial cable, and a wireless communication link may include a Bluetooth communication link, a Wireless-Fidelity (Wi-Fi) communication link, or a microwave communication link.

[0049] The terminal 101 can interact with the server 103 through the network 102 to receive a message from the server 103 or send a message to the server 103, or the terminal 101 can interact with the server 103 through the network 102 to receive a message or data sent by other users to the server 103. The terminal 101 can be hardware or software. When the terminal 101 is hardware, it can be various electronic devices, including but not limited to smart watches, smart phones, tablet computers, laptop portable computers and desktop computers. When the terminal 101 is software, it can be installed in the electronic devices listed above, which can be implemented as multiple software or software modules (for example: for providing distributed services), or it can be implemented as a single software or software module, which is not specifically limited here.

[0050] In an embodiment of the present application, the terminal 101 will continuously obtain sleep images collected for the sleep events of the target user; for each sleep image, the terminal 101 will input the sleep image into the sleep state detection model, and determine the sleep state of the target user output based on the sleep image by the sleep state detection model, and the sleep state is either falling asleep or waking up at night; when the sleep state is waking up at night, the terminal 101 automatically turns on the auxiliary light for waking up at night.

[0051] The server 103 may be a business server that provides various services. It should be noted that the server 103 may be hardware or software. When the server 103 is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or it may be implemented as a single server. When the server 103 is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or it may be implemented as a single software or software module, which is not specifically limited herein.

[0052] Alternatively, the system architecture may also not include the server 103. In other words, the server 103 may be an optional device in the embodiments of this specification, that is, the method provided in the embodiments of this specification may be applied to a system structure that only includes the terminal 101, and the embodiments of this application do not limit this.

[0053] It should be understood that Figure 1 The number of terminals, networks, and servers in the figure is only for illustration and any number of terminals, networks, and servers may be used according to implementation requirements.

[0054] See also Figure 2 , Figure 2 This is a flowchart of a sleep state detection method provided in an embodiment of the present application. The execution subject of the embodiment of the present application can be a terminal performing sleep state detection, a processor within the terminal performing the sleep state detection method, or a sleep state detection service within the terminal performing the sleep state detection method. For ease of description, the specific execution process of the sleep state detection method is described below using the processor within the terminal as an example.

[0055] like Figure 2 As shown, the sleep state detection method may at least include:

[0056] S202: Continuously acquire sleep images collected from sleep events of the target user.

[0057] Optionally, to protect the user's safety when waking up at night, embodiments of the present application promptly detect the user's intention to get up at night and provide appropriate lighting support when the user has lighting needs, thereby ensuring the user's safety when getting up at night and effectively improving the user's living comfort. To achieve the detection of the target user's sleep state and the intelligent waking up assistance function, it is first necessary to capture images of the target user's sleep events.

[0058] Specifically, a smart device capable of capturing images, such as a smart terminal device equipped with a camera, can be set up in the target user's sleeping environment. The smart terminal device can be installed on the ceiling of the bedroom, or at a suitable height on the wall, to ensure that the target user's sleeping image in bed can be fully captured. The camera on the smart terminal device also has the ability to capture high-definition images of the target in dark scenes at night. The camera is electrically connected to the system's control module and can continuously acquire sleep images collected based on the target user's sleep events at a preset frequency.

[0059] It should be noted that continuous image acquisition means that during the target user's sleep period (such as 22:00-6:00 the next day), the camera remains in working state to collect sleep images in real time, and the collected images are transmitted to the system's processing module in real time. The preset image acquisition frequency can be user-defined or default. For example, the preset acquisition frequency can be to capture a key image frame every 30 seconds. In addition, the preset acquisition frequency can also be dynamically adjusted while the user is sleeping. For example, the smart terminal device can automatically reduce the image acquisition frequency when the user is in deep sleep by checking the user's sleep quality; or the user can also set the smart terminal device to increase the image acquisition frequency within a specific night-time period (such as 3:00 am-5:00 am) according to the target monitoring user's night-time habits, thereby reducing storage pressure while ensuring data integrity.

[0060] S204 : For each sleep image, input the sleep image into a sleep state detection model, and determine the sleep state of the target user output by the sleep state detection model based on the sleep image, where the sleep state is either falling asleep or waking up at night.

[0061] Optionally, in order to accurately judge the sleep state of the target user, deep learning technology can be used to train a sleep state detection model. The sleep state detection model is used to process various sleep images collected in a dim environment at night and identify the sleep state and intention of the target user to get up at night in the image. Therefore, in the embodiment of the present application, each frame of the collected sleep image is preliminarily preprocessed, such as image noise reduction, normalization and other operations, and then input into the pre-trained sleep state detection model in real time to ensure the timeliness and accuracy of the data. After receiving the sleep image, the sleep state detection model will make a detailed judgment on the sleep state of the target user in the image based on its calculation and analysis capabilities and the characteristics of the image.

[0062] Specifically, the base of the sleep state detection model can be a neural network model built based on a deep learning algorithm, such as a convolutional neural network (CNN). The sleep state detection model will identify the target user's body parts and their movements by analyzing the pixel information of the image, and then determine whether the target user is currently in a state of getting up at night, so that the user's intention to get up at night can be detected in time to provide the user with corresponding lighting assistance. The sleep state of the target user determined based on high-resolution sleep images and deep learning models can accurately reflect the user's current intention to get up at night and provide a reliable basis for subsequent decision-making on lighting assistance. Throughout the process, the sleep state determination method based on the sleep state detection model is conducive to improving the user's sleep experience and life convenience.

[0063] In a feasible implementation, the sleep state detection model can specifically adopt a convolutional neural network architecture, which is composed of an improved CNN-LSTM hybrid model. The input end of the sleep state detection model is provided with a two-level feature extractor, namely a human feature extractor and an environmental feature extractor. Among them, the human feature extractor captures the changes in human body contours through a residual network structure, and the environmental feature extractor filters the interference of light changes through a background difference method, and distinguishes and analyzes the human body and background environment in the sleep image in a more fine-grained manner to obtain a more accurate sleep state determination result, so that a more reliable and accurate decision can be made when providing auxiliary lighting for getting up at night. In addition, the output layer of the model can also be set with a threshold determination mechanism to determine whether the final recognition result is a falling asleep state or a getting up at night state. For example, when the confidence level of the falling asleep state (usually represented by probability) is higher than the confidence level of the getting up at night state, the target user's sleep state is determined to be a falling asleep state, otherwise, the target user's sleep state is determined to be a getting up at night state.

[0064] S206: When the sleep state changes to the night-waking state, the night-waking auxiliary light is automatically turned on.

[0065] Optionally, when the result output by the sleep state detection model is the state of getting up at night, the turning on of the auxiliary light for getting up at night can be triggered. The auxiliary light for getting up at night can be emitted by the smart terminal device itself, or it can be emitted by other devices that have a signal transmission link with the smart terminal device and are set in a suitable location such as the corridor from the bedroom to the bathroom, the bedside of the bedroom, etc. The type of light can be a soft LED light, and its brightness is set so as not to cause strong stimulation to the eyes of the target user while meeting the lighting needs, for example, the brightness is between 5-10 lux.

[0066] Optionally, if the auxiliary light for getting up at night is emitted by the smart terminal device itself, then when the sleep state is the state for getting up at night, the smart terminal device directly generates a light-on signal to the light control module, so that the auxiliary light for getting up at night is turned on. If the auxiliary light for getting up at night is emitted by other devices, then the smart terminal device will transmit the light-on signal to other devices after generating it. When the light control module in the other device receives the light-on signal, it will control the auxiliary light for getting up at night to turn on, providing lighting support for the target user to get up at night. In this way, no matter where the light is located, the target user does not need to manually find the switch when getting up at night. The system can automatically provide corresponding auxiliary functions according to their sleep state, which improves the user's experience and convenience in life.

[0067] In a preferred embodiment, multiple night-time assistance lights can be installed along the user's travelable route. By continuously collecting sleep images, the user's night-time behavior pattern and path of action can be predicted in advance. The lights along the path can be illuminated sequentially according to the path of action, creating a gradual lighting effect to guide the user's walking. The lights can also provide warnings of obstacles in the path to ensure user safety. Furthermore, the auxiliary lights during the night-time waking process can also respond to different levels of waking behavior based on the user's different stages of waking. For example, when the user changes from a lying position to a standing position, the night-time assistance lights gradually brighten, with the light intensity increasing linearly with the distance from the bed. When the user returns to bed, the night-time assistance lights gradually dim and turn off. If the user does not return to bed within a preset time, low-light lighting can be activated throughout the house and a mobile device notification can be sent. If the target user is an elderly person or child, the mobile device receiving the push notification can be other family members in the home, allowing family members to respond promptly to unexpected situations that may occur during the target user's night-time waking process. In this way, the light activation range and brightness curve are dynamically optimized based on the user's night-time waking path and the length of stay, forming a personalized night-time assistance solution.

[0068] In an embodiment of the present application, a sleep state detection method is provided, which continuously acquires sleep images collected for sleep events of a target user; for each sleep image, the sleep image is input into a sleep state detection model, and the sleep state of the target user output by the sleep state detection model based on the sleep image is determined, and the sleep state is whether it is a falling asleep state or a waking up state; when the sleep state is a waking up state, an auxiliary light for waking up is automatically turned on. The embodiment of the present application continuously acquires sleep images for sleep events, and calls a sleep state detection model for each sleep image to analyze the sleep state of the target user in the image, thereby effectively distinguishing between bedding wrinkles and human limb movements, and judging whether the user is falling asleep or waking up at night based on the image features of the sleep image; when it is detected that the user's sleep state is a waking up state, the auxiliary light for waking up is automatically turned on for the user, protecting the user's movement safety when waking up at night, and establishing a reliable dynamic mapping relationship between the device operation state and physiological indicators. In this way, the target user does not need to manually search for a switch when waking up at night, and the system can automatically provide corresponding auxiliary functions according to their sleep state, thereby improving the user's sleep experience and life convenience, and forming an intelligent and comprehensive sleeping companion solution.

[0069] See also Figure 3 , Figure 3 A flowchart of a sleep state detection method provided in an embodiment of the present application.

[0070] like Figure 3 As shown, the sleep state detection method may at least include:

[0071] S302: In response to a user's triggering operation for a night-waking assistance function, the night-waking assistance function is turned on to collect sleep images of a target user's sleep events through a camera device.

[0072] Optionally, the user can manually turn on the night-waking assistance function that provides a night-waking light according to their needs, that is, the smart terminal device responds to the user's trigger operation on the night-waking assistance function and turns on the night-waking assistance function, thereby collecting sleep images of the target user's sleep events through the camera device.

[0073] Alternatively, another feasible implementation method for turning on the night-waking assistance function is included, S304, determining through a clock whether the current time meets the triggering conditions of a preset timed task, where the preset timed task is a timed activation task pre-set by the user for the night-waking assistance function; if the current time meets the triggering conditions, the preset timed task is executed to turn on the night-waking assistance function, so as to capture sleep images of the target user's sleep events through a camera device.

[0074] Optionally, in order to reduce the user's operating costs and facilitate the user's use of the night-waking assistance function, the user can also set the function to automatically start at a scheduled time. For example, if the target user is generally asleep between 10 p.m. and 6 a.m. every night, then the night-waking assistance function can be set to start between 10 p.m. and 6 a.m. every night; or if the target user is generally awake between 3 a.m. and 5 a.m., then the night-waking assistance function can be set to start between 3 a.m. and 5 a.m. The smart terminal device determines the current time through the clock. If the current time meets the conditions for starting the night-waking assistance function set by the user in the preset timed task, the preset timed task can be executed to start the night-waking assistance function, so as to collect sleep images of the target user's sleep events through the camera device.

[0075] S306. In response to a user's viewing instruction for the sleep state detection area, the image acquisition area corresponding to the camera device is displayed in the display interface; in response to a detection area setting instruction triggered by the user for the target display area in the image acquisition area, the target display area is determined as the target detection area in the sleep image.

[0076] Optionally, after turning on the night-time assistance function, the smart terminal device can first calibrate the detection range to facilitate the capture of a complete sleep image. Then, when determining the detection range, the user can be guided to manually define the detection area, that is, in response to the user's viewing instruction for the sleep state detection area, the current image acquisition area corresponding to the camera device is displayed in the display interface. If the user is not satisfied with the currently captured image area, for example, it is not aimed at the bed, the lens is crooked, etc., the target display area that needs to be set as the detection area can be manually adjusted. At this time, the smart terminal device responds to the detection area setting instruction triggered by the user for the target display area in the image acquisition area, and determines the target display area selected / defined by the user as the target detection area in the sleep image.

[0077] Alternatively, another feasible implementation method for determining the target detection area is included, S308, in response to a detection area automatic setting instruction triggered by the user, capturing a sample image of the current scene through a camera device and identifying calibrated furniture in the sample image, where the calibrated furniture is at least one of a bed, a chair, and a sofa; and delineating the target detection area in the sleep image based on the position of the calibrated furniture in the sample image.

[0078] Optionally, if the user prefers not to manually set the detection area, an automatic calibration function is also provided. When the user requests automatic detection area setting, the automatic detection area setting command can be triggered through a specific interactive method. For example, the user can click the "Automatically set detection area" button on the accompanying application of a smart terminal device (such as a mobile app); or issue the voice command "Automatically set sleep detection area" through an intelligent voice device. After receiving this command, the system uses a camera to capture a sample image of the current scene and uses image recognition technology to identify the calibration furniture in the image. The image recognition algorithm can use a deep learning-based sleep state detection model that has been pre-trained on a large amount of image data containing furniture such as beds, chairs, and sofas, or a conventional object recognition algorithm. The input sample image is first preprocessed, including image noise reduction and grayscale conversion, to improve recognition accuracy. The preprocessed image is then analyzed layer by layer through pixel features to identify whether at least one of the following items, a bed, chair, or sofa, is present in the sample image. If so, the target detection area in the sleep image is delineated based on the specific location and outline information of the furniture in the image. For example, if the calibrated furniture identified is a bed, since the bed is the main area where the user sleeps, the system will use the position of the bed in the image as the center and expand a certain pixel distance on all sides of the bed (such as 50-100 pixels outward based on the edge of the bed, and the specific value can be adjusted according to the actual scenario and needs) to ensure that the user's activity range in bed and possible sleeping areas are covered. If other calibrated furniture such as chairs or sofas are also identified and located near the bed, the positions of these furniture can be comprehensively considered to avoid unnecessary overlap or interference between the target detection area and the areas where these furniture are located, ensuring that the target detection area can accurately cover the main space for the user to sleep and related activities, thereby providing accurate range definition for subsequent sleep image analysis and sleep state detection functions based on this area.

[0079] S310: Continuously acquire sleep images collected from sleep events of a target user.

[0080] Regarding step S310 , please refer to the detailed description in step S202 , which will not be repeated here.

[0081] S312 : For each sleep image, input the sleep image into a sleep state detection model, and determine the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0082] Optionally, for each sleep image, the sleep image is input into a sleep state detection model, which then outputs the target user's sleep state based on the target detection area in the sleep image. Specifically, the model identifies whether a target person is present in the target detection area, and if the sleep state detection model identifies a target person, outputs the target user's sleep state as awake.

[0083] In a feasible implementation, the target human body is a human silhouette that reaches a preset ratio with the complete human silhouette; or the target human body is a human body with a preset posture. That is, there can be two verification mechanisms for the recognition and judgment of the target human body: one is to judge whether the user is getting up to go to the bathroom based on the integrity of the human silhouette, calculate the intersection over union (IoU) of the current detected silhouette and the standard human template and the key point integrity score, and when the proportion of the silhouette area exceeds a preset threshold (for example, 75%) and the visibility of the main joints (head and neck, shoulders and hips, and extremities) meets the standard, it is judged to be a complete human silhouette; the second is to judge whether the user is getting up to go to the bathroom based on the recognized human posture, for example, when the action features such as sitting up, bending the knees, and leaning forward are recognized, the target user's status is output as getting up to go to the bathroom.

[0084] S314. When the sleep state changes to the night-waking state, the night-waking auxiliary light is automatically turned on.

[0085] Regarding step S314 , please refer to the detailed description in step S206 , which will not be repeated here.

[0086] S316: Generate notification information corresponding to the sleep event based on the waking state, and send the notification information to the bound target terminal according to a preset notification method.

[0087] Optionally, when the sleep state detection model determines that the target user is in the state of getting up at night, the system can extract the core parameters of the sleep event: including the timestamp of the time of getting up at night, accurate to milliseconds, obtained through system clock synchronization; the posture characteristics of the target body when getting up at night, such as the sitting angle, the degree of separation of the limbs and the bed, etc., are converted into standardized descriptions based on the image recognition results, as well as the serial position of the event of getting up at night in the sleep cycle of the day, for subsequent sleep quality analysis. Based on the above parameters, a notification message can be generated based on a preset template, for example: "Getting up at night was detected at [timestamp], the current posture is sitting and preparing to get out of bed, and the auxiliary light for getting up at night has been turned on." The format of the notification message supports multi-modal output such as text, image thumbnail (capture of the key frame of the sleep image at the moment of getting up at night), and voice broadcast. The specific output form is pre-configured by the user in the setting interface of the target terminal (such as a smartphone APP, the central control screen of the smart terminal device) or reflected in the system default way.

[0088] S318. When the sleep state detection model recognizes the sleeping state based on the sleep image for a consecutive preset number of times, the night-time auxiliary light is automatically turned off.

[0089] Optionally, after turning on the night-time assist light, if the sleep state detection model recognizes that the number of times the user has fallen asleep based on subsequent sleep images reaches a preset number of times, it can be considered that the user has finished getting up at night and has returned to sleep, and the night-time assist light can be automatically turned off. When turning off the light, a gradual dimming technology is specifically used (such as linearly reducing the light brightness from the current value to 0 within 3-5 seconds) to avoid visual stimulation to the user caused by instantaneous light turning off, further improving sleep comfort. In this way, through the dynamic adjustment of the light, a deep linkage between the night-time assist light and the user's sleep state is achieved, which not only ensures immediate lighting support when getting up at night, but also automatically turns off the light after the user falls asleep again to avoid light interference with sleep.

[0090] In an embodiment of the present application, a sleep state detection method is provided, and the user can manually turn on the night-waking assistance function that provides a night-waking light according to his or her needs, that is, the smart terminal device responds to the user's trigger operation for the night-waking assistance function and turns on the night-waking assistance function, thereby collecting sleep images for the target user's sleep events through the camera device. The function can also be set to automatically start at a time. The smart terminal device determines the current time through the clock. If the current time has met the start-up conditions of the night-waking assistance function set by the user in the preset timed task, the preset timed task can be executed to turn on the night-waking assistance function, so as to reduce the user's operating costs and facilitate the user to use the night-waking assistance function. In response to the detection area setting instruction triggered by the user for the target display area in the image acquisition area, the smart terminal device determines the target display area selected / demarcated by the user as the target detection area in the sleep image to support the user to manually demarcate the detection area. If the user does not want to set it manually, the function of automatic calibration of the detection area is also provided to the user. Notification information corresponding to sleep events is generated based on the status of waking up at night, and notification information is sent to the bound target terminal according to the preset notification method. After turning on the auxiliary light for getting up at night, if the sleep state detection model recognizes that the number of times of falling asleep reaches the preset number of consecutive times based on subsequent sleep images, it can be considered that the user has finished getting up at night and re-entered the sleeping state, then the auxiliary light for getting up at night can be automatically turned off. In this way, through dynamic adjustment of the light, the deep linkage between the auxiliary light for getting up at night and the user's sleep state is realized.

[0091] See also Figure 4 , Figure 4 A flowchart of a model training method for a sleep state detection model provided in an embodiment of the present application.

[0092] like Figure 4As shown, the model training method of the sleep state detection model may at least include:

[0093] S402: Construct an initial sleep state detection model for the sleep scene based on the basic model.

[0094] Optionally, the sleep state detection model needs to recognize human movements based on the pixel features of sleep images. In this case, a basic model with a neural network architecture can be obtained, and an initial sleep state detection model for sleep scenes can be constructed based on the basic model. This also allows the initial sleep state detection model to learn specific sample data based on the neural network architecture.

[0095] S404 : Acquire multiple sample sleep images, where the multiple sample sleep images are all sample data with standard sleep state labels.

[0096] Optionally, multiple sample sleep images are obtained for training the initial sleep state detection model. Each of the sample sleep images is sample data with a standard sleep state label. The standard sleep state label represents the correct standard sleep state corresponding to each sample sleep image and can therefore serve as a standard label for the sample sleep images. The sample sleep images contain image data of different sleep states of the user, including both images of falling asleep (i.e., a state in which the user is in a quiet sleep with a relatively stable body posture) and images of waking up at night (e.g., when the user is getting out of bed, sitting up, or getting out of bed).

[0097] S406 : Input a plurality of sample sleep images into an initial sleep state detection model to train the initial sleep state detection model.

[0098] S408. During the training process of the initial sleep state detection model, the initial sleep state detection model is controlled to output predicted sleep state labels for multiple sample sleep images, and a training loss value is calculated based on the predicted sleep state labels and the standard sleep state labels of the multiple sample sleep images. The parameters of the initial sleep state detection model are adjusted according to the training loss value until the initial sleep state detection model converges, thereby obtaining a trained sleep state detection model.

[0099] Optionally, during the model training phase, the initial sleep state detection model is controlled to output predicted sleep state labels for multiple sample sleep images. These predicted sleep state labels are the sleep state prediction results of the initial sleep state detection model for the multiple sample sleep images. The difference between the predicted sleep state labels and the standard sleep state labels represents the difference between the current state of the initial sleep state detection model and the expected performance. Based on this, the model's training loss value can be calculated based on the predicted sleep state labels and the standard sleep state labels of the multiple sample sleep images. The parameters of the initial sleep state detection model are adjusted based on the training loss value until the initial sleep state detection model converges to obtain a trained sleep state detection model. Furthermore, the model's training termination conditions may include, for example, the loss function value meeting the target value condition or the number of iterations reaching a preset threshold. The specific model training termination conditions can be determined based on actual conditions and are not specifically limited here.

[0100] In an embodiment of the present application, a model training method for a sleep state detection model is provided. By labeling and training sample data, the model can accurately extract features from the input sleep images, such as the target user's body posture, movement amplitude, position change, etc., and output the target user's sleep state based on these features, that is, determine whether the target user is in a sleeping state or a night-time state.

[0101] See also Figure 5 , Figure 5 This is a structural block diagram of a sleep state detection device provided in an embodiment of the present application. Figure 5 As shown, the sleep state detection device 500 includes:

[0102] An image acquisition module 510 is configured to continuously acquire sleep images collected from sleep events of a target user;

[0103] A state detection module 520 is configured to input each sleep image into a sleep state detection model and determine the sleep state of the target user output by the sleep state detection model based on the sleep image, where the sleep state is either a falling asleep state or a waking state;

[0104] The night-waking assistance module 530 is used to automatically turn on the night-waking assistance light when the sleeping state changes to the night-waking state.

[0105] Optionally, the sleep state detection device 500 also includes: a detection area manual setting module, which is used to display the image acquisition area corresponding to the camera device in the display interface in response to the user's viewing instruction for the sleep state detection area; in response to the detection area setting instruction triggered by the user for the target display area in the image acquisition area, the target display area is determined as the target detection area in the sleep image; the state detection module 520 is also used to determine the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0106] Optionally, the sleep state detection device 500 also includes: a detection area automatic setting module, which is used to respond to the detection area automatic setting instruction triggered by the user, collect sample images of the current scene through a camera device and identify calibrated furniture in the sample image, where the calibrated furniture is at least one of a bed, a chair, and a sofa; based on the position of the calibrated furniture in the sample image, define the target detection area in the sleep image; the state detection module 520 is also used to determine the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0107] Optionally, the state detection module 520 is also used to control the sleep state detection model to identify whether there is a target human body in the target detection area based on the sleep image, and if the sleep state detection model identifies the target human body, the state of the target user is output as the night-waking state.

[0108] Optionally, the target human body is a human silhouette having a preset ratio to the complete human silhouette; or, the target human body is a human body with a preset posture.

[0109] Optionally, the sleep state detection device 500 also includes: a manual function activation module for activating the waking up assistance function in response to a user's triggering operation on the waking up assistance function, so as to capture sleep images of the target user's sleep events through a camera device.

[0110] Optionally, the sleep state detection device 500 also includes: a function automatic start-up module, which is used to determine through a clock whether the current time meets the triggering conditions of a preset timed task, and the preset timed task is a timed start-up task pre-set by the user for the night-waking assistance function; if the current time meets the triggering conditions, the preset timed task is executed to turn on the night-waking assistance function, so as to collect sleep images of the target user's sleep events through a camera device.

[0111] Optionally, the sleep state detection device 500 further includes: an event notification module, configured to generate notification information corresponding to a sleep event based on the waking state, and send the notification information to a bound target terminal in a preset notification manner.

[0112] Optionally, the sleep state detection device 500 further includes: a light shutoff module, configured to automatically shut off the night-time auxiliary light when the sleep state detection model identifies the sleeping state based on the sleep image for a consecutive preset number of times.

[0113] Optionally, the sleep state detection device 500 also includes: a model training module, which is used to construct an initial sleep state detection model for a sleep scene based on a basic model; obtain multiple sample sleep images, and the multiple sample sleep images are sample data with standard sleep state labels; input the multiple sample sleep images into the initial sleep state detection model to train the initial sleep state detection model; during the training process of the initial sleep state detection model, control the initial sleep state detection model to output predicted sleep state labels for the multiple sample sleep images, and calculate the training loss value based on the predicted sleep state labels and the standard sleep state labels of the multiple sample sleep images, and adjust the parameters of the initial sleep state detection model according to the training loss value until the initial sleep state detection model converges to obtain the trained sleep state detection model.

[0114] In an embodiment of the present application, a sleep state detection device is provided, wherein an image acquisition module is configured to continuously acquire sleep images captured for sleep events of a target user; a state detection module is configured to input the sleep image into a sleep state detection model for each sleep image, and determine the sleep state of the target user output by the sleep state detection model based on the sleep image, whether the sleep state is falling asleep or waking up at night; and a night-waking assistance module is configured to automatically turn on a night-waking assistance light when the sleep state is waking up at night. The embodiment of the present application continuously acquires sleep images for sleep events, and for each sleep image, uses the sleep state detection model to analyze the sleep state of the target user in the image, thereby effectively distinguishing between bedding wrinkles and human body movements, and determining whether the user is falling asleep or waking up at night based on the image features of the sleep image; when the user's sleep state is detected to be waking up at night, the night-waking assistance light is automatically turned on for the user to protect the user's movement safety when waking up at night, and a reliable dynamic mapping relationship between the device operating state and physiological indicators is established. In this way, the target user does not need to manually search for a switch when waking up at night. The system can automatically provide corresponding assistance functions based on their sleep state, improving the user's sleep experience and life convenience, and forming an intelligent and comprehensive sleep companion solution.

[0115] An embodiment of the present application further provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded by a processor and executing the steps of any method in the above embodiments.

[0116] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. Figure 6As shown, the terminal 600 may include: at least one terminal processor 601 , at least one network interface 604 , a user interface 603 , a memory 605 , and at least one communication bus 602 .

[0117] The communication bus 602 is used to implement the connection and communication between these components.

[0118] The user interface 603 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 603 may also include a standard wired interface and a wireless interface.

[0119] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0120] The terminal processor 601 may include one or more processing cores. The terminal processor 601 utilizes various interfaces and circuits to connect various components within the entire terminal 600. It executes instructions, programs, code sets, or instruction sets stored in the memory 605, and accesses data stored in the memory 605 to perform various functions and process data for the terminal 600. Optionally, the terminal processor 601 may be implemented using at least one hardware form factor selected from the group consisting of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The terminal processor 601 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the terminal processor 601 and may be implemented as a separate chip.

[0121] Among them, the memory 605 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 605 includes a non-transitory computer-readable storage medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be optionally at least one storage device located away from the aforementioned terminal processor 601. As Figure 6 As shown, the memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a sleep state detection program.

[0122] exist Figure 6 In the terminal 600 shown, the user interface 603 is mainly used to provide an input interface for the user and obtain data input by the user; and the terminal processor 601 can be used to call the sleep state detection program stored in the memory 605 and specifically perform the following operations:

[0123] Continuously acquiring sleep images collected from sleep events of a target user;

[0124] For each sleep image, the sleep image is input into a sleep state detection model, and the sleep state detection model determines the sleep state of the target user output based on the sleep image, where the sleep state is either falling asleep or waking up at night;

[0125] When the sleep state changes to the night-time state, the night-time auxiliary light will be automatically turned on.

[0126] In some embodiments, the terminal processor 601 also specifically performs the following steps: in response to a user's viewing instruction for the sleep state detection area, the image acquisition area corresponding to the camera device is displayed in the display interface; in response to a detection area setting instruction triggered by the user for the target display area in the image acquisition area, the target display area is determined as the target detection area in the sleep image; when the terminal processor 601 determines the sleep state of the target user output by the sleep state detection model based on the sleep image, the terminal processor 601 specifically performs the following steps: determines the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0127] In some embodiments, the terminal processor 601 further specifically performs the following steps: in response to a detection area automatic setting instruction triggered by a user, the terminal processor 601 collects a sample image of the current scene through a camera device and identifies the calibrated furniture in the sample image, where the calibrated furniture is at least one of a bed, a chair, and a sofa; based on the position of the calibrated furniture in the sample image, the target detection area in the sleep image is delineated; when the terminal processor 601 determines the sleep state of the target user output by the sleep state detection model based on the sleep image, the terminal processor 601 specifically performs the following steps: determines the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image.

[0128] In some embodiments, when the terminal processor 601 determines the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image, it specifically performs the following steps: controlling the sleep state detection model to identify whether there is a target human body in the target detection area based on the sleep image, and if the sleep state detection model identifies the target human body, outputting the target user's status as waking up at night.

[0129] In some embodiments, the target human body is a human silhouette having a preset ratio to a complete human silhouette; or, the target human body is a person with a preset posture.

[0130] In some embodiments, the terminal processor 601 further specifically performs the following steps: in response to the user's triggering operation for the night-waking assistance function, the night-waking assistance function is turned on to collect sleep images of the target user's sleep events through a camera device.

[0131] In some embodiments, the terminal processor 601 also specifically performs the following steps: determining through the clock whether the current time meets the triggering conditions of the preset timed task, the preset timed task is a timed start task pre-set by the user for the night-waking assistance function; if the current time meets the triggering conditions, the preset timed task is executed to turn on the night-waking assistance function, so as to collect sleep images of the target user's sleep events through the camera device.

[0132] In some embodiments, after the terminal processor 601 automatically turns on the auxiliary light for getting up at night when the sleep state changes to the state of getting up at night, it also specifically performs the following steps: generates notification information corresponding to the sleep event based on the state of getting up at night, and sends the notification information in the bound target terminal according to the preset notification method.

[0133] In some embodiments, after turning on the night-waking assistance light when the sleep state is the night-waking state, the terminal processor 601 also specifically performs the following steps: when the sleep state detection model recognizes that the number of times the sleeping state is reached based on the sleep image reaches a preset number of consecutive times, the night-waking assistance light is automatically turned off.

[0134] In some embodiments, the terminal processor 601 further specifically performs the following steps: constructing an initial sleep state detection model for a sleep scene based on a basic model; obtaining multiple sample sleep images, each of which is sample data with a standard sleep state label; inputting the multiple sample sleep images into the initial sleep state detection model, and training the initial sleep state detection model; during the training process of the initial sleep state detection model, controlling the initial sleep state detection model to output predicted sleep state labels for the multiple sample sleep images, and calculating a training loss value based on the predicted sleep state labels and the standard sleep state labels of the multiple sample sleep images, adjusting the parameters of the initial sleep state detection model based on the training loss value until the initial sleep state detection model converges, and obtaining a trained sleep state detection model.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0136] Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of these modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The above-mentioned computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, the above-mentioned process or function according to the embodiment of this specification is generated in whole or in part. The above-mentioned computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The above-mentioned computer instructions can be stored in a computer-readable storage medium or transmitted by the above-mentioned computer-readable storage medium. The above-mentioned computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The above-mentioned computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The above-mentioned available media can be magnetic media (for example, floppy disks, hard disks, tapes), optical media (for example, digital versatile discs (DVDs)), or semiconductor media (for example, solid state disks (SSDs)).

[0138] It should be noted that for the aforementioned method embodiments, for ease of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0139] In addition, it should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0140] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0141] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] The above is a description of a sleep state detection method, device, storage medium, and terminal provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A sleep state detection method, characterized in that: The method comprises: Continuously acquiring sleep images collected from sleep events of a target user; For each sleep image, input the sleep image into a sleep state detection model, and determine the sleep state of the target user output by the sleep state detection model based on the sleep image, where the sleep state is either a falling asleep state or a waking up state; When the sleeping state is a night-waking state, the night-waking auxiliary light is automatically turned on.

2. The method according to claim 1, characterized in that The method further comprises: In response to a user's viewing instruction for the sleep state detection area, displaying the image acquisition area corresponding to the camera device on the display interface; In response to a detection area setting instruction triggered by the user for a target display area in the image acquisition area, determining the target display area as a target detection area in the sleep image; The determining the sleep state of the target user output by the sleep state detection model based on the sleep image includes: The sleep state detection model is used to determine the sleep state of the target user output based on the target detection area in the sleep image.

3. The method according to claim 1, characterized in that The method further comprises: In response to a user-triggered automatic detection area setting instruction, capturing a sample image of the current scene through a camera device and identifying calibration furniture in the sample image, where the calibration furniture is at least one of a bed, a chair, and a sofa; demarcating a target detection area in the sleeping image based on the position of the calibration furniture in the sample image; The determining the sleep state of the target user output by the sleep state detection model based on the sleep image includes: The sleep state detection model is used to determine the sleep state of the target user output based on the target detection area in the sleep image.

4. The method according to claim 2 or 3, characterized in that The determining the sleep state of the target user output by the sleep state detection model based on the target detection area in the sleep image includes: The sleep state detection model is controlled to identify whether there is a target human body in the target detection area based on the sleep image, and if the sleep state detection model identifies the target human body, the state of the target user is output as being awake at night.

5. The method according to claim 1, wherein The method further comprises: In response to a user's triggering operation on a night-time assistance function, the night-time assistance function is turned on to collect sleep images of a target user's sleep events through a camera device.

6. The method according to claim 1, characterized in that The method further comprises: Determine through the clock whether the current time meets the triggering conditions of the preset timer task, which is a timer start task pre-set by the user for the night-time assistance function; If the current time satisfies the trigger condition, the preset timer task is executed to start the night-time assistance function, so as to capture sleep images of the target user's sleep events through a camera device.

7. The method according to claim 1, characterized in that The method further comprises: Build an initial sleep state detection model for sleep scenarios based on the basic model; Acquire a plurality of sample sleep images, wherein the plurality of sample sleep images are all sample data with standard sleep state labels; inputting the plurality of sample sleep images into the initial sleep state detection model to train the initial sleep state detection model; During the training process of the initial sleep state detection model, the initial sleep state detection model is controlled to output predicted sleep state labels for the multiple sample sleep images, and a training loss value is calculated based on the predicted sleep state labels and the standard sleep state labels of the multiple sample sleep images. The parameters of the initial sleep state detection model are adjusted according to the training loss value until the initial sleep state detection model converges, thereby obtaining a trained sleep state detection model.

8. A sleep state detection device, characterized in that: The device comprises: An image acquisition module, configured to continuously acquire sleep images collected from sleep events of a target user; a state detection module, configured to input each sleep image into a sleep state detection model, and determine a sleep state of the target user output by the sleep state detection model based on the sleep image, wherein the sleep state is either a falling asleep state or a waking up state; The night-waking assistance module is used to automatically turn on the night-waking assistance light when the sleeping state is the night-waking state.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the steps of the method according to any one of claims 1 to 7.

10. A terminal, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the program.

Images