Sleep detection and sleep monitoring device

Abstract

To provide a care recipient monitoring device and system that respects the privacy of the care recipient while continuously monitoring their sleep / bedtime state and any abnormal situations without restraint or contact, and facilitates continuous monitoring and remote monitoring with a small amount of transmitted data. [Solution] The system comprises an imaging device 2 having an event sensor that outputs an event signal only when the brightness change of each pixel exceeds a predetermined threshold in the sleeping image of the person being cared for 1, a body movement processing means 8 that detects the body movements of the person being cared for while they are sleeping using the event signal, and a caregiver's terminal device 4. The body movement processing means 8 generates an event image by grouping the event signals in a predetermined integral time unit shorter than the duration of one frame of the monitor display image, and based on the generated event image, it observes the sleep state of the person being cared for 1 based on changes in their body movements and detects abnormal conditions. If an abnormal situation occurs, an alert signal is generated to the caregiver's terminal device 5.

Description

Technical Field

[0001] The present invention is a monitoring device that detects the sleep state at bedtime and monitors for abnormal situations at bedtime. In particular, it relates to a monitoring device and system that detects the bedtime and sleep state by sensing the movement of the subject when lying in bed and continuously monitors.

Background Art

[0002] In nursing care and medical facilities, it is necessary to grasp the bedtime and sleep state of patients and those receiving care (hereinafter referred to as care recipients) and prevent accidents at bedtime, which has been an issue. To grasp the sleep state by a doctor, the PSG (Poly Somno Graph) test using a polygraph is widely used. However, since several sensors are attached to the body and the test is performed in a contact state, the device is large-scale, the stress on the patient is also large, and it cannot be easily performed.

[0003] In addition, accidents and abnormal situations to be grasped during monitoring in the nursing care and medical fields include abnormal situations due to respiratory disorders such as the care recipient falling or toppling from the bed, apnea continuation, the continuation of the absence state in the bed, violent acts by caregivers or others, and the like. While the number of patients and care recipients who require nursing care, caregiving, and attention due to illness or being bedridden is increasing due to the progress of aging, the number of nursing care and medical staff, caregivers, and those who perform caregiving and monitoring such as relatives, relevant persons, and managers of security companies and condominiums (hereinafter referred to as caregivers) is in a shortage state, and more efficiency and labor saving are required.

[0004] In response to such problems, various techniques for grasping the lying state and sleep depth of care recipients using a pressure-sensitive mat sensor, a monitoring camera, etc. as techniques for monitoring the bedtime state of care recipients have been disclosed. In Patent Document 1, body movement of a person in bed is detected by a pressure sensor installed on the bed, and the sleep state is grasped. Further, in Patent Document 2, a technique for acquiring a moving image by a camera and analyzing the body movement state for each epoch of each frame to determine the sleep depth is disclosed.

[0005] While many of these conventional technologies can monitor the implantation and sleep status of those receiving care, they present challenges such as the difficulty of continuous data acquisition, the enormous amount of data generated when continuous monitoring is performed using video cameras, making them unsuitable for remote centralized management and supervision, and the need to protect the privacy of those receiving care as much as possible.

[0006] Many nursing and medical facilities need to constantly monitor and understand the sleep patterns of individual patients and those receiving care, but continuously monitoring the sleep patterns of those receiving care is difficult. Efficient monitoring would be possible if the sleep patterns of those receiving care could be continuously acquired as data and observed and analyzed from remote locations such as a central management unit or the caregiver's location. However, conventional technologies acquire data such as images, resulting in a massive amount of information, limiting the ability to continuously transmit information on the sleep patterns and monitoring status of those receiving care. Therefore, there is a need to reduce the amount of information as much as possible.

[0007] Although the devices and systems according to the present invention are intended for nursing care and medical settings, they can be applied not only to nursing care and medical use, but also to data communication between the care recipient's home and caregivers such as relatives, apartment managers, and security companies, as well as to data transmission and reception between individuals in remote locations and medical specialists. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2007-252747 [Patent Document 2] Japanese Patent Publication No. 2018-164615 [Overview of the project] [Problems that the invention aims to solve]

[0009] This invention is provided in view of the above circumstances and primarily solves the following problems. (1) Respect the privacy of the person receiving care and constantly monitor their sleep and bedtime without restraint or contact. (2) To identify abnormal situations during sleep and provide means to prevent accidents. (3) Minimize the amount of transmitted data acquired to facilitate continuous monitoring and remote monitoring. [Means for solving the problem]

[0010] To solve these problems and achieve the above objectives, the sleep state monitoring device and system of the present invention is a device and system for a caregiver to monitor the sleep state of a person being cared for, and the device and system comprises: an imaging device having an event sensor that outputs an event signal only when the brightness change of each pixel exceeds a predetermined threshold in the sleep image of the person being cared for; a body movement processing means that analyzes the body movements of the person being cared for while sleeping using the event signal; and a terminal device for the caregiver, wherein the body movement processing means generates an event image by summarizing the event signals in a predetermined integral time unit shorter than the period of one frame of the monitor display image, and based on the generated event image, observes the sleep state by changes in the body movements of the person being cared for and detects abnormal conditions, and if an abnormal situation occurs, an alert signal is generated to the terminal device for the caregiver.

[0011] Furthermore, the sleep state monitoring device and system according to the present invention can also be configured such that the output data of the event signal indicates the coordinates of the pixels, the time, and the change in brightness.

[0012] Furthermore, the sleep state monitoring device and system of the present invention can also be configured such that the body motion processing means is connected to a network means and stored in a cloud server, and the output data of the event signal is sent to the body motion processing means in the cloud server via the network means.

[0013] Furthermore, the sleep state monitoring device and system of the present invention can also be configured so that the body movement processing means detects the time of getting in and out of bed, the time spent sleeping (amount of sleep), the amount of body movement during sleep, the duration of body movement, and the duration of stillness of the person being cared for, based on the event signal, and monitors the sleep state.

[0014] Furthermore, the sleep state monitoring device and system of the present invention can also be configured to generate an alert signal via an alert means when, based on the event signal, the person being cared for becomes stationary beyond a preset range, when the shape of a third party different from the preset shape of the person being cared for is detected, or when the intrusion of an object different from the preset shape of the person being cared for is detected.

[0015] Furthermore, the sleep state monitoring device and system in the present invention may also include a frame image capturing device, and may be configured to activate the frame image capturing device in response to the generation of the alert signal.

[0016] Furthermore, the sleep state monitoring device and system in the present invention can also be configured such that the body movement processing means includes a storage and learning means for output data based on the event signal, and the storage and learning means grasps, stores, and learns the sleep characteristics of the person being cared for. [Effects of the Invention]

[0017] According to the present invention, when using an event sensor camera to grasp the body movements during bedtime, the bedtime image of the care recipient is output and transmitted as an event signal only when the luminance change per pixel exceeds a predetermined threshold value. Therefore, the bedtime state of the care recipient can be grasped with an extremely small amount of data. As a result, it becomes possible to constantly monitor or centrally manage the bedtime state of the care recipient by acquiring it with an event sensor camera at a remote location and transmitting the acquired event signal. Further, by analyzing the body movements based on the acquired event signal, not only can the sleep state be grasped, but also abnormal situations such as falling from the bed and intrusion by a third party can be automatically grasped and an alert can be generated.

Brief Description of the Drawings

[0018] [Figure 1] It is an explanatory diagram showing a configuration example of the bedtime state monitoring device and system of the present invention. [Figure 2] It is an explanatory diagram showing the event signal of the present invention. [Figure 3] It is an example of a hypnogram showing the sleep state during bedtime. [Figure 4] It is an example showing the body movement changes during bedtime according to the present invention in terms of the number of event data for each of the amount of body movement, stationary time, and moving time. [Figure 5] It is an explanatory diagram of falling from the bed during bedtime according to the present invention. [Figure 6] It is a block diagram explaining the combined use of the event image and the frame image according to the present invention.

Embodiments for Carrying Out the Invention

[0019] Hereinafter, embodiments of the bedtime state monitoring device and system according to the present invention will be described in detail with reference to the drawings. Note that any of the explanatory diagrams and drawings described in the following examples are drawn as schematic or schematic diagrams for explaining the present invention, and the actual dimensions and shapes are not particularly limited. Also, the system configuration, block diagram, dimensions, materials, shapes, their relative arrangements, and usage examples used in the examples are not intended to limit the technical scope of the invention only to these unless otherwise specified.

Example

[0020] ) FIG. 1 is an explanatory diagram showing a configuration example of a bedtime state monitoring device and system according to Embodiment 1 of the present invention. An imaging device 2 having an imaging range capable of observing the bedtime state of the cared-for person 1 is installed. The imaging device 2 includes an event sensor that outputs an event signal (or event data) only when the luminance change of each pixel in the bedtime image of the cared-for person 1 exceeds a predetermined threshold value. Although the details will be described later, the output of the imaging device 2 captures the movement and movement of the cared-for person 1 and other objects within the imaging range in the image in conjunction with the passage of time, and outputs it as an event signal of independent pixel information.

[0021] The imaging device 2 is connected to a communication network means 4 via a terminal communication means 3. Through this communication network means 4, it is connected to a monitoring terminal device 5 and a cloud server 6. The monitoring terminal device 5 is assumed to be installed in a nursing center or a centralized management room that watches over the cared-for person 1 at a location away from the residence of the cared-for person 1. However, it may be a monitoring terminal device 5 owned by not only caregivers and medical staff but also relatives, concerned persons, and apartment managers who watch over the cared-for person 1. Also, this network means 4 can be constructed by any form of means, wired or wireless.

[0022] A watching platform 7 is provided in the cloud server 6. This watching platform 7 is composed of a program that constantly analyzes and observes the body movements of the cared-for person 1 at bedtime, grasps and records the bedtime and sleep states, and generates an alert when an abnormal situation occurs.

[0023] The monitoring platform 7 includes a motion processing means 8 that detects the body movements of the person being cared for 1 while they are sleeping, using the event signal output from the imaging device 2. The motion processing means 8 generates an event image by summarizing the event signals in predetermined integration time units. Here, the predetermined integration time unit is formed in units shorter than the duration of one frame of the monitor display image. Based on the generated event image, the sleeping state is recorded based on changes in the body movements of the person being cared for 1, and abnormal conditions are detected. If an abnormal situation occurs, an alert signal is generated to the monitoring terminal device 5 by the alert means 9.

[0024] Furthermore, the monitoring platform 7 is equipped with a memory and learning means 10 that memorizes and learns the sleeping state and sleep characteristics of the person being cared for 1, and feeds this information back to the body movement processing means 8 to more accurately grasp the sleeping state through body movements. The recorded log can also be used for analyzing the sleeping state of the person being cared for 1.

[0025] In typical imaging devices such as video cameras (or frame cameras), full-frame image data is acquired by synchronizing the camera with a vertical synchronization signal, and then inter-frame differences are encoded, real-time compression is performed, or motion changes are extracted from the moving image at multiple frame intervals for motion processing. However, this invention performs image processing that is completely different from such frame image synchronization cameras. The event data processing obtained from the event signal in this invention captures the change in the brightness signal only of pixels where motion has occurred within the imaging range. Therefore, the output event signal is output as pixel coordinates (x, y), time (t), and brightness polarity or brightness change (p), as illustrated in Figure 2. In Figure 2, the coordinates (x1, y1) on the screen move from time t1 to coordinates (x4, y4) at time t4, and the brightness polarity at time t1 changes from (p1) to (p4).

[0026] As an event signal, if there is a change in the limbs of the person being cared for while sleeping, such as slight movement of the limbs, turning over, head movement, or movement of the bedding, the movement trajectory of only the changed part is recorded. Here, the minimum high-speed time resolution can be obtained down to about 1 μsec on the time axis t, but since it is sufficient to detect human body movement, even slight movements due to spasms of the mouth or limbs can be detected at about 100 msec.

[0027] This event signal can be displayed as a movement trajectory image by grouping it into predetermined integration time units. Here, the integration time is set to be shorter than the duration of one frame, which corresponds to the frame rate.

[0028] The event signal outputs the pixel value of the brightness change at the moment movement occurs. Since the background of the room where care recipient 1 is sleeping and the bed are stationary, data is not acquired from them, and only the brightness change due to care recipient 1's body movement is extracted as output data. Because the amount of output data is significantly reduced compared to frame-based images, the transmission of event signals can be made significantly cheaper, and in terms of video signal processing, it is not subject to limitations in bandwidth, memory capacity, noise processing, and image difference processing, and can be constructed with a relatively simple circuit configuration. For example, compared to an image signal compressed by inter-frame difference processing in a frame image, the event data signal can be reduced to (more than 1 / 100 to 1 / 500) the size.

[0029] In the body movement processing means 8, the sleep state of the person being cared for 1 is first detected based on changes in body movement from the acquired event data. Figure 3 shows a sleep progression diagram (hypnogram) used as an international standard (AASM standard) by the American Academy of Sleep Medicine. Sleep states are divided into stages of wakefulness (WK), REM (REM), and non-REM (N1, N2, N3) sleep, depending on the time elapsed since going to sleep. In the wakeful state, the brainwaves show that the person is awake, and in terms of body movement, there is continuous body movement. In the REM sleep state, the muscles of the whole body are relaxed, but the brain is active. Non-REM sleep is further subdivided into N1 (light non-REM), N2 (intermediate non-REM), and N3 (deep non-REM). In terms of body movement, the wakeful state is the most active, and the deep non-REM (N3) state is the least active. Therefore, by detecting the amount of body movement, it is possible to determine to some extent whether the person is in a wakeful state or a non-REM (N3) state.

[0030] In nursing and medical settings, continuously monitoring sleep patterns and observing the health status of care recipients requires digitizing daily sleep patterns. This sleep assessment should ideally include not only sleep quality (REM or non-REM sleep), but also sleep onset, sleep duration, nighttime awakenings, and early morning awakenings.

[0031] In this invention, the event sensor output data continuously acquires the coordinates (x, y), time (t), and brightness polarity (p) on the screen for each pixel where movement occurs, as time progresses. When the person being cared for 1 moves, as time progresses from t1 to t2, the brightness polarity p1 of the pixel at coordinates (x1, y1) changes from 1 (bright) to 0 (dark) due to the change in brightness, and the brightness polarity p2 of the pixel at coordinates (x2, y2) changes from 0 (dark) to 1 (bright). Similarly, the movement of the pixel at coordinates (x, y) can be captured as brightness changes from t3, t4, ... to tn. By continuously acquiring the brightness changes for each pixel, the movement of the person being cared for 1 is detected. Furthermore, by capturing the changes in pixel coordinates, the direction of movement, movement time, and stopping time of the person being cared for 1 can be measured.

[0032] To understand the wakefulness, REM, and non-REM sleep states related to sleep quality, it is necessary to detect body movement. To determine the body movement of care recipient 1 using event sensor output, the total number of pixels is counted for each predetermined period (T). In the AASM criteria, this predetermined period (T) is set at 30 seconds as one epoch period, but if detailed data is not required, an epoch period of 1 to 5 minutes is acceptable.

[0033] This change in body movement generally occurs as the individual moves from an active state to a less active state as they transition from an awakened state to a non-REM state. Since there are individual differences in body movement among people of all ages and genders, the amount of body movement calculated from the event sensor output is stored as data by the memory and learning means 10, and the quality of sleep during sleep is judged by comparing it with the average value of the stored data. In addition, the average of still time and the average of moving time during the same epoch period are obtained from the event data, and the sleep state is observed. The average of still time and the average of moving time are displayed as the average number of times the individual was still and moving, respectively, during the epoch period.

[0034] Figure 4 is a graph showing the changes in wakefulness, REM, and non-REM sleep states during body movement, resting time, and movement time. From this graph, we can understand the trends for wakefulness, REM, and non-REM sleep. As sleep progresses from wakefulness to REM and then to non-REM, body movement decreases, while the average resting time increases. Although there are many individual differences in this trend, generally, a sleep cycle tends to occur where the individual transitions from wakefulness to REM and then to non-REM (N1-N4) states, repeating this cycle two to three times, and then transitioning to a non-REM state once before waking up. In this way, based on event data, the average values ​​of body movement, resting time, and movement time can be obtained, and the changes in these values ​​can be used to determine whether a sleep cycle is occurring, whether a non-REM (deep sleep) state is being maintained, or whether the individual is in a light sleep state.

[0035] To understand a person's sleep state, it's important to assess not only their body movement but also how easily they fall asleep. The quality of their sleep onset is judged by the amount of body movement from implantation to falling asleep. Excessive body movement indicates an awake state; generally, falling asleep within about 15 minutes is considered acceptable. Since this time to fall asleep varies from person to person, accumulating memories and learning about the care recipient's sleep state allows for a more accurate assessment. It's also important to check if the person wakes up several times before entering the non-REM state after falling asleep. Failure to smoothly transition to the non-REM state after falling asleep should also be considered a sign of difficulty falling asleep and require attention.

[0036] Sleep duration is measured not from the time of bed rest to alighting, but from the time of falling asleep through several sleep cycles, based on changes in body movement, and from REM sleep to wakefulness. Since sleep duration naturally varies from person to person, past data for care recipient 1 is referred to. In care monitoring, it is desirable for sleep duration to be consistently stable, and caution is needed if it is extremely short or long. Also, waking up significantly earlier than the normal sleep-wake cycle is a point of concern.

[0037] Changes in body movement can also be used to check for the presence or absence of nighttime awakenings. These nighttime awakenings may include frequent awakenings due to reasons such as frequent urination. While it is not a major problem if the person can return to a non-REM state after waking up, frequent nighttime awakenings require close monitoring. Furthermore, if the awakening time is unstable and constant awakening by a caregiver is required, it is necessary to instruct the person to establish a regular wake-up routine as much as possible. In addition, the event sensor can detect even the movement of the chest and bedding associated with breathing, so body movement during apnea can be detected by the cessation of blanket or futon movement to a certain extent. However, this differs from the apnea interval measured in AHI tests performed in medical settings, so it is used to determine the possibility of sleep apnea syndrome.

[0038] As described above, by using imaging device 2 (event sensor camera) to measure body movement during sleep, it becomes possible to understand sleep duration, bedtime, and wake-up time, as well as changes in body movement (or additionally, the average values ​​of stillness and movement time), whether the person is falling asleep (whether they are experiencing difficulty falling asleep), middle-of-the-night awakenings (duration and frequency), early morning awakenings (delayed awakening), and abnormalities in the sleep cycle (duration and frequency of non-REM and REM states). It is also possible to measure sleep states in more detail by calculating the average time of movement and stillness, as well as acquiring data with different epoch periods. However, this observation of sleep states is not a medical measurement of the state from wakefulness to REM sleep and non-REM sleep in a strict manner, but rather is for caregiving and daily monitoring of patients. By enabling observation with a smaller amount of data, it is possible to continuously understand the sleep state and health status of the person being cared for 1 even from a distance, making health management, caregiving, and monitoring easier.

[0039] Next, we will explain how the imaging device (event sensor camera) 2 detects abnormal situations during sleep. Abnormal situations mainly include the care recipient 1 falling from the bed, tripping in the room, death, and continuous coughing or seizures.

[0040] As shown in Figure 5, when the event sensor camera 2 is installed directly above the person being cared for 1 who is seated on the bed 50, it detects the shape of the bed 50 and the shape of the person being cared for 1. In this case, the shape of the person being cared for 1 may be a frame shape 41 that captures their height and maximum width, or a frame shape 42 that captures only their head or upper body. This is because most falls from beds are due to the head or upper body sliding off the bed. To detect a fall, it is first detected whether the frame shape 51 of the person being cared for 1, or the frame shape 52 of their head or upper body, has moved a predetermined percentage (most of the person being cared for 1's body) away from the set bed shape. Since the person being cared for 1 moving away from the bed 50 does not necessarily mean that they have fallen, if the body or head of the person being cared for 1 remains still for a certain period of time after moving away from the bed, or if slight movement is detected, it is determined that the person has moved away from the bed and there is a possibility of some kind of abnormality.

[0041] Furthermore, to detect falls indoors, prolonged periods of inactivity (such as death or illness), and continuous coughing or seizures, the system will track when continuous movement (continuous activity) and continuous periods of inactivity (still activity) or micro-movements of the entire body and individual body parts of the person being cared for (1) persist for a predetermined period of time.

[0042] To detect falls indoors, the system can either detect changes in the frame shape of the person being cared for (1) and the continuation of a period of time spent crouching after the change, or detect falls by learning the shapes during walking and falling using the memory and learning means 10 and then distinguishing them through pattern recognition.

[0043] Furthermore, to detect periods of inactivity such as death or inactivity due to some malfunction, the system acquires individual event signals for the movement of each body part (head, chest, limbs, etc.) in addition to the overall body frame. If the inactivity continues for a predetermined period of time, an alert is issued by the alert means 9, thereby recognizing prolonged periods of inactivity as abnormal. It is also possible to issue an alert if a specific body part continues to exhibit activity that differs from its normal state, as this may indicate an abnormal situation. The determination of whether the activity in this specific body part differs from the normal state can be made using a learning pattern from the memory / learning means 10, or by pre-recording symptoms such as specific chronic illnesses of the person being cared for 1 as event signals, and issuing an alert signal via the alert means 9 when the pre-recorded event signal is detected.

[0044] Furthermore, an alert signal will be issued if a third party or animal enters the room during sleep, as this is considered an abnormal situation. In particular, if a third party enters the room in addition to the person being cared for while sleeping, it is possible to detect that someone else has entered based on their approximate size and other characteristics, as people are constantly moving. Similarly, the intrusion of animals or small animals such as birds can also be detected because they are constantly moving. However, it is not possible to determine whether a person or animal other than the person being cared for who has entered constitutes an abnormal situation. Therefore, the detection of a third party's intrusion serves as a warning that there is a possibility of an abnormal situation, and if necessary, it is possible to activate a frame camera or other device to determine whether the third party or animal constitutes an abnormal situation.

[0045] This configuration also allows for detection and alerting even if a third party is equivalent to caregiver 11. In recent years, there has been a tendency for violent incidents between care recipient 1 and caregiver 11, as well as disputes and fights between care recipient 1 and other care recipients 1. If the aforementioned third-party intrusion is detected and a violent incident occurs between care recipient 1 and the intruder, the movements of care recipient 1 and the intruder tend to be abnormally violent compared to normal body movements. If the event camera footage indicates that the body movements of care recipient 1 and the intruder are faster than normal, it is possible to issue an alarm as a possible abnormal situation. [Examples]

[0046] In the event sensor camera of the present invention, since only motion is detected, it is difficult to identify the person being cared for 1 or any third party who enters the area. While this is advantageous from the standpoint of protecting privacy, in the various abnormal situations described above, it is also possible to configure the system to use a frame video (color or monochrome) camera in combination to better understand the situation. Figure 6 is an example of a block diagram of the configuration when the event sensor camera and the frame video camera are used together in the imaging device 2. In Figure 6, the same reference numerals are used for components with the same function as in Figure 1.

[0047] The imaging device 2, used to monitor a person being cared for 1 while they are sleeping, has a lens unit 61 attached to a lens mount 62. The image acquired by the lens unit 61 is separated into transmitted light and reflected light by a separation film 64 of a separation prism 63, which is composed of a split prism or a half mirror. The light beam separation by the separation film 64 of the separation prism 63 is usually 50:50, but is not limited to this. The transmitted light that has passed through the separation prism 63 enters the EVS image sensor 66 via an auxiliary prism 65 that is positioned in close contact with the downstream of the separation prism 63. The EVS image sensor 66 extracts only the movement of the subject as an event signal in response to changes in brightness.

[0048] Meanwhile, the reflected light reflected by the reflective film 64 of the light beam separation prism 63 enters the frame image image sensor 68 via an auxiliary prism 67 located on the reflected light exit side. The frame image image sensor 68 is an image sensor that extracts a color frame image by frame synchronization.

[0049] Here, the lens mount 62 is composed of a C-mount, CS-mount, or S-mount for interchangeable lenses, and general-purpose industrial miniature lenses can be used, but it is not limited to these, and a configuration in which the lens is directly attached to a prism or the like without a lens mount is also acceptable. The optical path lengths from the lens section 61 to the event image sensor 66 and the frame image sensor 68 are the same, and each lens maintains a predetermined optical path length. When a lens mount is used, it is set to the flange back length specified for each lens mount (for example, the flange back of a C-mount is 17.526 mm).

[0050] The electrical signal converted photoelectrically by the event image sensor 66 is acquired as an event signal as described above. The event signal, acquired by the imaging device 2 from the location of the person being cared for 1 and converted photoelectrically by the event image sensor 66, is sent to the cloud server 6 via the terminal communication means 3 and the communication network means 4. The cloud server 6 uses the motion processing means 8 within the monitoring platform 7 to detect and determine the sleep state and the presence or absence of abnormal situations of the person being cared for 1 while they are sleeping, records this in the memory and learning means 10, and grasps the trends and daily sleep patterns of the person being cared for 1. The event signal is used to generate an event image display signal by the image processing means 69 within the cloud server 6.

[0051] Meanwhile, the frame video signal, which has been photoelectrically converted by the frame image sensor 68, is converted into a digital signal by the A / D conversion circuit 70 and sent to the cloud server 6 via the communication network means 4. The frame video signal is then used by the frame image processing means 71 in the cloud server 6 to generate a frame image display signal.

[0052] On a daily basis, the caregiver 11 obtains event image display signals from the cloud server 6 via the monitoring terminal device 5 and communication network means 4 as needed, and displays them as event images on a handheld display device (such as a monitor television) 73. This event signal image processing means 69 may be located within the cloud server 6 or within the monitoring terminal device 5. If the event image processing means 69 is located within the monitoring terminal device 5, it is configured to send only event signals from the cloud server 6 to the monitoring terminal device 5.

[0053] Normally, the system monitors sleep states and abnormal situations based on event signals acquired by the event image sensor 66 of the imaging device 2. However, if the possibility of an abnormal situation arises as described above, the alert means 9 issues an alert signal. When the body motion processing means 8 detects any abnormal situation, the alert means 9 sends an alert (warning) signal to the monitoring terminal device 5. This alert signal is communicated to the monitoring terminal device 5 by sound or flashing light. At the same time, a display switching means 72 is configured to switch from an event image to a frame image. The display switching means 72 normally outputs the image to be displayed on the display device (monitor) 73 as an event image, but is configured to also send a frame image when an alert signal indicating a possible abnormal situation is issued. In the monitoring terminal device 5, which is monitored by caregiver 11, when an alert is issued, the monitor 73 checks the location of caregiver 1, who has been alerted, using both the event image and the frame image.

[0054] According to the present invention, an event sensor camera is used to output an event signal only when the brightness change of each pixel exceeds a predetermined threshold, allowing the sleep status of the person being cared for to be understood with an extremely small amount of data. Furthermore, the system is configured to acquire frame images only when there is a possibility of an abnormal situation occurring with the person being cared for. Therefore, transmission costs can be significantly reduced compared to transmitting frame images, and image processing can be made simpler and the transmission speed faster compared to frame image processing. Thus, it becomes possible to constantly monitor and centrally manage the sleep status of the person being cared for from a remote location. In addition, by analyzing body movements using the acquired event signals, it is possible not only to understand the sleep state but also to automatically detect abnormal situations such as falling from the bed or intrusion by a third party and generate an alert. As a result, caregivers do not need to constantly check on or monitor the person being cared for at night, enabling efficient monitoring. [Industrial applicability]

[0055] Such sleep monitoring devices and systems can be widely applied not only to medical and care workers such as caregivers and caregivers in the medical and nursing fields, but also to relatives, related parties, security companies, and building managers in apartment buildings who monitor patients at night while they sleep. This opens up a wide range of potential uses in various fields such as nursing care, medical care, welfare, and the security industry. [Explanation of Symbols]

[0056] 1. Person receiving care 2. Imaging device 3. Terminal communication methods 4. Communication Network Means 5. Monitoring terminal device 6. Cloud (Server) 7. Monitoring Platform 8. Body movement processing means 9. Alert methods 10. Memory and Learning Methods 11 Caregivers 61 Lens section 62 Lens Mount 63 Separation prism 64 Separation membrane 65, 67 Auxiliary prism 66 Event Image Sensor 68-frame image sensor 69 Event Image Processing Means 70 A / D conversion circuit 71 Frame image processing means 72 Display switching means 73 Display device (monitor)

Claims

1. In a device and system for a caregiver to monitor the sleeping state of a person being cared for, The apparatus and system include an imaging device having an event sensor that outputs an event signal only when the brightness change of each pixel in the image of the person being cared for while sleeping exceeds a predetermined threshold, A body movement processing means that analyzes the body movements of the person being cared for while they are sleeping using the event signal, The caregiver's terminal device is provided, The motion processing means generates an event image by grouping the event signals in a predetermined integral time unit shorter than the duration of one frame of the monitor display image. A sleep state monitoring device and system characterized by observing the sleep state of the person being cared for based on the generated event image, detecting abnormal conditions, and generating an alert signal to the caregiver's terminal device if an abnormal situation occurs.

2. The sleep status monitoring device and system according to claim 1, characterized in that the output data of the event signal indicates the coordinates of the pixels, the time, and the change in brightness.

3. The aforementioned motion processing means is stored in a cloud server connected to a network means. The sleep state monitoring device and system according to claim 2, characterized in that the output data of the event signal is sent via the network means to the body motion processing means in the cloud server.

4. The sleep state monitoring device and system according to claim 3, characterized in that the body movement processing means detects the time of getting into and out of bed, the time spent sleeping (amount of sleep), the amount of body movement during sleep, the duration of body movement, and the duration of stillness of the person being cared for, based on the event signal, and monitors the sleep state.

5. The sleep monitoring device and system according to claim 3, characterized in that, based on the event signal, an alert signal is generated by the alert means when the person being cared for becomes stationary beyond a predetermined range, when the shape of a third party different from the predetermined shape of the person being cared for is detected, or when the intrusion of an object different from the predetermined shape of the person being cared for is detected.

6. The sleep state monitoring device and system according to claim 5, further comprising a frame image capturing device, wherein the frame image capturing device is activated in response to the generation of the alert signal.

7. The sleep state monitoring device and system according to any one of claims 3 to 6, wherein the body movement processing means includes a means for storing and learning output data based on the event signal, and the storage and learning means grasps, stores, and learns the sleep characteristics of the person being cared for.

Images