A method for sleep state analysis with video

Abstract

The application discloses a method for sleep state analysis by using video, and relates to the technical field of sleep state analysis. When analyzing and monitoring the sleep state, the existing monitoring means monitors the sleep state by wearing similar devices. However, the devices need to be worn to monitor the sleep state, and the wearing process will cause discomfort to the body and will cause local compression to the body, which will affect the sleep of people with poor sleep quality. The application collects sound and images, and the collection process does not need the measured person to wear, so that the measured person is not affected. In the video monitoring, the sleep state is preliminarily analyzed according to the coordinate change and the data analysis of the sound after analyzing the body movement. Furthermore, the sleep monitoring method precision is remarkably improved by further monitoring the eye features.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of sleep state analysis, in particular to a method for sleep state analysis by video. BACKGROUND

[0002] Sleep state refers to the appearance of a person when sleeping. In contrast to the state of wakefulness, a person has four stages of sleep. They are: falling asleep, light sleep, deep sleep, and prolonged deep sleep. Each cycle of sleep process can also be divided into four sleep stages from light to deep.

[0003] For this purpose, Chinese patent number CN113662511A discloses a wearable device sleep monitoring method, device, equipment and storage medium. The method comprises the following steps: when the wearable device is in a stable state and in a sleep mode, the human body core temperature is acquired based on a first preset acquisition frequency within a first preset time; the temperature-related number of times that the human body core temperature is greater than a preset temperature threshold is counted; if the temperature-related number of times is less than a preset temperature-related number of times threshold, it is determined that the wearable device is not in a wearing state; and when the wearable device is not in a wearing state, the sleep mode is exited. The present application determines whether the wearable device is in a wearing state by the human body core temperature. If it is not in a wearing state, the sleep mode is exited, thereby reducing the misjudgment rate of determining whether a user enters a sleep state when sleep monitoring is performed by a wearable device.

[0004] For this purpose, Chinese patent number CN106889975A discloses a method for sleep state analysis by video, comprising the following steps: step S1, acquiring the video of the person being cared for, and converting each frame of color image into a gray scale image; step S2, performing Gaussian blur on each frame of the video, then moving the mean value to obtain the background; subtracting the current frame from the background and the absolute value of the difference to obtain the current change image frame; calculating the number of pixels in the current change image frame whose pixel value is greater than a predetermined threshold, and marking it as BMI; step S3, dividing the BMI into multiple segments according to the time length, and calculating the median value of the BMI of each time length segment; then inputting a finite digital filter; and then obtaining an index A; and performing binaryzation on A to obtain the sleep state of the person being cared for. The present application does not affect the sleep of the person being monitored, and provides a new way for analyzing the sleep state of the person being cared for.

[0005] In the analysis and monitoring of sleep state, the existing monitoring means monitors the sleep state by wearing similar devices, which include wristbands, watches, bracelets and other electronic devices. However, such devices need to be worn to monitor, which may cause discomfort to the body during the wearing process, and may even cause local compression to the body, which may affect the sleep of people with poor sleep quality.

[0006] In view of the above problems, a method for analyzing sleep state by using video is provided. SUMMARY

[0007] The present application aims to provide a method for analyzing sleep state by using video, which solves the problem that sleep monitoring affects sleep in the background art.

[0008] To achieve the above-mentioned purpose, the present application provides the following technical solution: a method for analyzing sleep state by using video, comprising the following steps,

[0009] S1: collecting sound and image in video monitoring;

[0010] S2: collecting limb movement in image collection, establishing limb coordinates for the first time, and establishing X-axis and Y-axis indicators;

[0011] S3: face collection, including face collection in image collection, and establishing whether to enter sleep state by collecting facial expressions;

[0012] S4: eye feature collection, monitoring eye details by image collection device, and monitoring whether the measured unit's eyes are open;

[0013] S5: eye feature analysis, analyzing the movement of eyelids in closed eye state, and observing whether the eyeball is in rotation state.

[0014] Preferably, environment sound and human voice are collected in sound collection, and the collected environment sound and human voice are compared and analyzed to analyze the volume change.

[0015] Preferably, the data is transmitted to the sound statistical unit module for unified processing after human voice collection, and the human voice is recorded as a variable based on the change of environment sound in the processing.

[0016] Preferably, video monitoring includes sound collection and image collection, and image collection is divided into limb movement monitoring and face feature monitoring.

[0017] Preferably, the limb movement collection monitoring device calibrates the region frame, and the region frame is located in the grid coordinate background.

[0018] Preferably, the limb collection region frame is continuously focused on the limb, and the limb changes position, and the limb collection region frame is displaced on the grid coordinate background under continuous focusing.

[0019] Preferably, the coordinate displacement change parameter is collected in the data collection module, and the data collection module collects sound collection data, coordinate collection data, and face feature eye local feature collection data, and outputs the data after collection.

[0020] Preferably, the face collection monitors whether the face has motion, and optical zooming is performed in the state without motion, and eye focusing is performed after zooming in the view, and the eyes are monitored.

[0021] Preferably, the eye feature analysis analyzes whether the eyelid has a jump and a rotation state of the eyeball when the eyes are closed.

[0022] Preferably, the data set analysis processes the sound, the body movement and the eye feature, and outputs a human voice variable, a body coordinate change parameter and an eye feature variable parameter.

[0023] Compared with the prior art, the present application has the following beneficial effects:

[0024] 1. The method for analyzing the sleep state by using the video provided by the present application collects the sound and the image without the need of wearing by the measured person in the collection process, so that the measured person is not affected, analyzes the body movement in the video monitoring, and preliminarily analyzes the sleep state according to the coordinate change and the data analysis of the sound, and further monitors the eye feature, so that the accuracy of the sleep monitoring method is significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0025] Fig. 1 The figure is a method flowchart of the present application.

[0026] Fig. 2 The figure is a sound collection and judgment method schematic diagram of the present application.

[0027] Fig. 3 The figure is a region coordinate schematic diagram of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0029] The present application will be described in detail with reference to the drawings for further understanding the present application.

[0030] In combination with the drawings, Figs. 1-3 the method for analyzing the sleep state by using the video, characterized in that it comprises the following steps,

[0031] S1: sound is collected in video monitoring, and environment sound and human voice are collected respectively in sound collection, the environment sound collected is compared and analyzed with the human voice, the sleep state is analyzed preliminarily through the volume change and regular change, the data is transmitted to the sound statistical unit module for unified processing after the human voice is collected, the environment sound is quantified based on the environment sound change in the processing, the human voice is recorded as a variable, the environment sound in the monitoring record is the noise in the outside world, there is still certain environment sound in a relatively quiet environment, the sound in the environment also affects the sleep state, wherein the sleep is divided into four states, and specifically:

[0032] The first stage is a falling asleep stage, which starts from drowsiness and gradually falls asleep and no longer maintains a wakeful state, at this time, the respiration is slow, the muscle tension is reduced, and the body is slightly relaxed, at this time, the sleeper is easy to be awakened by external sound or touch;

[0033] The second stage is a light sleep stage, or a light sleep stage. In this stage, the sleep is shallow or light to moderate sleep state, and the sleeper is not easy to be awakened, at this time, the muscle is further relaxed, and the electroencephalogram shows a shuttle sleep wave;

[0034] The third stage is a deep sleep stage, at this time, the sleeper enters a deep sleep state, the muscle tension disappears, the muscle is fully relaxed, and the sensory function is further reduced, and it is more difficult to be awakened;

[0035] The fourth stage is an extended deep sleep stage, which is an extension of the third stage, but not every sleeper can reach this stage, and not every sleep cycle can reach this stage, some people cannot reach this stage because their sleep is not deep enough, or only have a short time of third and fourth stage sleep;

[0036] Therefore, in sleep monitoring, the environment sound in sound collection will affect the first and second stages of sleep, when the sleep enters the third and fourth stages, the environment sound will not affect the measured person under normal circumstances, the preliminary judgment of whether the patient enters the sleep state can be made by detecting the human voice, wherein snoring exists during sleep, if snoring occurs, it indicates that the measured person enters the sleep stage, if there is no snoring, the state of the measured person is analyzed through image collection in video monitoring;

[0037] When in the snoring state, if disturbed by the environment sound, the snoring is terminated, and the image collection changes, the sleep stage of the measured person is reduced;

[0038] Image collection in video monitoring, video monitoring includes sound collection and image collection, and the image collection is divided into limb movement monitoring and face feature monitoring;

[0039] S2: In the image acquisition, the limb movement in sleep is collected, the limb movement collection monitoring device calibrates the region frame, the region frame is located in the grid coordinate background, the limb coordinates are established for the first time, the X axis and Y axis are established, the limb collection region frame is continuously focused on the limb, the limb position changes, the continuous focusing limb collection region frame has a plane displacement on the grid coordinate background, the coordinate displacement change parameter is collected in the data collection module, the data collection module collects the sound collection data, the coordinate collection data and the face feature eye local feature collection data, and outputs the data after collection, in the image acquisition of the limb, the coordinates of the limb are recorded as (X1, Y1), when turning over or the limb changes, the coordinate position changes, and the coordinate is recorded as (X2, Y2), the coordinates generated by frequent turning over are recorded, and the sleep state is evaluated, if the turning over frequency is too high, the sleep quality is poor, if the turning over frequency is normal, the sleep state is in good condition;

[0040] S3: Face collection, the image acquisition includes face collection, whether to enter the sleep state is established by collecting facial expressions, whether the face has movement is monitored, optical zoom is performed in the state without movement, the eye is focused after the optical zoom, and the eye is monitored;

[0041] S4: Eye feature collection, the eye details are monitored by the image acquisition device, whether the measured unit eye is open is monitored, the eye is monitored, if the eye is in the open state, it is indicated that sleep is not performed, when the eye is closed, the eyelid is monitored;

[0042] By observing the eye, it is directly shown on the eye that the eyeball can be seen rolling through the eyelid, sometimes very fast, by observing the state of the eye, the sleep is evaluated, when the limb is not moving, it is still in the snoring state, and the eye is rotating, it is indicated that the measured person is in the sleep state;

[0043] S5: Eye feature analysis, the movement of the eyelid in the closed eye state is analyzed, whether the eyeball is in the rotating state is observed, the eyelid is analyzed, whether the eyelid exists jumping and the rotating state of the closed eyeball; whether sleep is monitored by combining the sound monitoring, the limb movement and the face feature, the normal sleep of the measured person is not affected in the monitoring process, the results of the three monitoring are analyzed and uploaded;

[0044] The data set analysis processes the sound, the body movement and the eye feature, outputs the human voice variable, the body coordinate change parameter and the eye feature variable parameter, uploads the information for analysis, and determines that the measured person is not asleep if the body frequently changes and the eyes are open during the analysis. The measured person is determined to be in the sleep state if the body change is within the normal range and the eyes are closed. The measured person is determined to enter the third and fourth stages if the external environment sound does not affect the measured person, and the sleep quality is determined to be reduced if the external environment sound affects the measured person, and the sleep state is determined to be in the second or first state.

[0045] It should be noted that the relational terms herein, such as first and second, and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0046] While the embodiments of the application have been illustrated and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the spirit and scope of the application, which is defined by the appended claims and their equivalents.

Claims

1. A method for sleep state analysis using video, characterized in that: Includes the following steps, S1: Video monitoring involves the acquisition of sound and images. Video monitoring includes the acquisition of sound and images. Image acquisition is divided into body movement monitoring and facial feature monitoring. S2: During image acquisition, limb movements during sleep are captured. The limb coordinates are established for the first time, and the indicators X and Y axes are established. The limb movement acquisition and monitoring device is calibrated with a frame. The frame is placed on a grid coordinate background. The limb acquisition frame is continuously focused on the limb. The limb changes position. Under continuous focusing, the limb acquisition frame undergoes planar displacement on the grid coordinate background. S3: Face capture, image capture includes face capture, determining whether the person has entered a sleep state by capturing facial expressions, monitoring whether there is facial movement, optical zooming is performed when there is no movement, and eye focusing is performed after optical zooming and zooming in on the view to monitor the eyes. S4: Eye feature acquisition, which monitors the details of the eyes through an image acquisition device, and monitors whether the eyes of the subject are open. When the eyes are open, it indicates that the subject is not asleep. When the eyes are closed, the eyelids are monitored. S5: Eye feature analysis, which analyzes and monitors the movement of the eyelids when the eyes are closed, observes whether the eyeballs are in a state of rotation, analyzes whether the eyelids are twitching, and analyzes the rotation state of the eyeballs when the eyes are closed. By combining sound monitoring, limb movements and facial features, it is possible to monitor whether the person is asleep. When the limbs are still, the person is snoring, and the person's eyes are moving, it indicates that the person being tested is asleep.

2. The method for sleep state analysis using video according to claim 1, characterized in that: The sound acquisition process includes the collection of ambient sounds and human voices. The ambient sound data is then compared and analyzed with the human voice data to monitor changes in volume.

3. The method for sleep state analysis using video according to claim 2, characterized in that: After the human voice is collected, the data is transmitted to the sound statistics unit module for unified processing. During the processing, the changes in ambient sound are used as a quantitative measure, while the human voice is recorded as a variable.

4. The method for sleep state analysis using video according to claim 1, characterized in that: The data collection module collects sound acquisition data, coordinate acquisition data, and facial and eye feature acquisition data, and then outputs the collected data.

5. The method for sleep state analysis using video according to claim 1, characterized in that: Data set analysis processes voice, body movements, and eye features, outputting voice variables, body coordinate change parameters, and eye feature variable parameters.

Images