Non-face-to-face non-contact bed falling monitoring system, non-contact sleep monitoring system and method

Abstract

The invention provides a non-face-to-face non-contact bed falling monitoring system, a non-contact sleep monitoring system and a non-contact sleep monitoring method, which are used for monitoring bed falling of a user by using a radar signal and a thermal imaging signal so as to improve the reliability of bed falling monitoring and analyze and guide the sleep state of the user.

Description

Technical Field

[0001] The present invention relates to a non-face-to-face and non-contact bed-fall monitoring system for improving the reliability of bed-fall monitoring by monitoring a user's bed-fall using radar signals and thermal imaging signals, and a contact sleep monitoring system and method for analyzing and guiding a user's sleeping state. Background Art

[0002] In recent years, the number of people in domestic society who need care or concern has continued to increase.

[0003] People who need care or concern can be defined as those who need help in an emergency or do not have a cohabiting partner, such as the elderly, the disabled, and single-person households. The latest survey shows that the elderly population in South Korea has exceeded 9 million, the number of single-person households and the number of registered disabled people have exceeded 6.6 million and 2.6 million respectively. In addition, the number of people who died without a cohabiting partner and were later found to have died alone exceeded 10,000.

[0004] Therefore, falling out of bed is an increasing accident in which a person falls regardless of his / her intentions and is a dangerous accident that may cause various complications or even death.

[0005] Therefore, bed fall monitoring technology that can accurately monitor bed fall accidents and respond quickly is very important.

[0006] Bed fall monitoring devices that use cameras monitor the movements of the elderly through video and send an alarm to the guardian when a fall is detected, but there are issues of human rights violations and privacy.

[0007] In addition, the existing bed fall monitoring device using radar signals has the following problem: even if a bed fall actually occurs, it is judged that it is not a bed fall, or it is judged that the bed fall occurred due to radio wave interference caused by wind or the like.

[0008] In addition, the existing wearable bed-fall monitoring device has the disadvantage of being unable to correctly analyze the user's fall from bed because the user often forgets to wear the wearable device or loses the wearable device itself.

[0009] In addition, due to the irregular lifestyle of single-person households, sleep quality may become unstable, which may harm health. In order to test sleep quality, it is necessary to make an appointment with a special hospital, visit the doctor on time, and go through the cumbersome process of using special equipment directly while sleeping, so there is a problem that it is difficult to test sleep quality.

[0010] Summary of the invention

[0011] Technical problem to be solved by the invention

[0012] In order to solve the problems of the prior art as described above, an object of the present invention is to provide a non-face-to-face and non-contact bed fall monitoring system that improves the reliability of bed fall monitoring while protecting the privacy of the user.

[0013] Furthermore, another object of the present invention is to provide a non-face-to-face and non-contact bed-fall monitoring system that can monitor a user's bed-fall without wearing a monitoring device.

[0014] Furthermore, another object of the present invention is to provide a non-face-to-face and non-contact bed-fall monitoring system that determines the degree of danger when a bed-fall occurs and provides a danger notification to a guardian or medical staff.

[0015] Furthermore, another object of the present invention is to provide a non-contact sleep monitoring system and method that uses a radar sensor unit to obtain biological activity information of a user in a sleeping state and uses a thermal imaging sensor unit to obtain turning over movements as auxiliary signals, thereby improving the reliability of user status analysis.

[0016] Furthermore, another object of the present invention is to provide a non-contact sleep monitoring system and method that can protect the privacy of a user without using thermal imaging data as it is.

[0017] Technical solutions to solve problems

[0018] The non-face-to-face and non-contact bed-fall monitoring system of the present invention includes: a radar sensor unit, which is used to transmit radar to a user and then receive the radar reflected from the user, and use a Doppler radar algorithm to generate a time series Doppler radar signal; a thermal imaging sensor unit, which is used to photograph the user above the user to generate thermal imaging data; a radar signal pattern analysis unit, which is used to receive the time series Doppler radar signal from the radar sensor unit, and analyze the user's status based on the received time series Doppler radar signal; a thermal imaging data processing unit, which is used to receive the thermal imaging data generated by the thermal imaging sensor unit, process the received thermal imaging data and analyze the user's status; and a bed-fall judgment unit, which judges whether a bed-fall occurs based on the user's status data received from the radar signal pattern analysis unit and the thermal imaging data processing unit.

[0019] The radar signal pattern analysis unit may be configured to separate a movement change signal and a biological signal from the time series Doppler radar signal received by the radar sensor unit, and use the separated movement change signal as a time series movement Doppler radar signal to analyze the state of the user.

[0020] The radar signal pattern analysis unit can be configured to generate first state data indicating a fall from bed when the time series moving Doppler radar signal undergoes a yin-yang change with an amplitude greater than or equal to a predetermined size within a first time and the amplitude becomes 0; and generate second state data indicating that the user is slowly falling and suspected of falling from bed when the time series moving Doppler radar signal undergoes a yin-yang change with an amplitude greater than or equal to a predetermined size after the first time.

[0021] The thermal imaging data processing unit can process the thermal imaging data by executing the following steps: determining an analysis range in the received thermal imaging data; determining an edge of the user in the determined analysis range; and processing the thermal imaging data using a preset threshold value so that it has a closed curve shape from the determined edge to obtain a closed curve video.

[0022] The thermal imaging data processing unit may further perform the following steps to form a marked video: performing video marking on the closed curve video.

[0023] The thermal imaging data processing unit may further perform the following steps to form a spectrum diagram: performing spectrum conversion on the marked video.

[0024] The thermal imaging data processing unit can judge the user's state as one of the fourth state to the sixth state based on at least one of the closed curve video, the marked video and the spectrum diagram, and form fourth state data to sixth state data respectively.

[0025] The fourth state data may indicate a state in which the user is located on an object at a predetermined height from the ground, the fifth state data may indicate a state in which the user is falling from an object at a predetermined height from the ground, and the sixth state data may indicate a state in which the user has fallen from an object at a predetermined height from the ground.

[0026] The bed-fall judgment unit can be configured as follows: when the first state data is received from the radar signal pattern analysis unit, the fourth state data, the fifth state data and the sixth state data are received in sequence from the thermal imaging data processing unit, and the fourth state data is generated before the generation time of the first state data, it is judged that the user has fallen out of bed, and a first signal indicating that the user has fallen out of bed is generated; when the first state data judged as the first state is received from the radar signal pattern analysis unit, the sixth state data, the fifth state data and the fourth state data are received in sequence from the thermal imaging data processing unit, and the sixth state data is generated before the generation time of the first state data, it is judged that the user has not fallen out of bed, and a second signal indicating that the bed-fall has not occurred is generated; when the second state data indicating a suspected bed-fall is received from the radar signal pattern analysis unit, the fourth state data, the fifth state data and the sixth state data are received in sequence from the thermal imaging data processing unit, and the first state data is generated before the generation time of the sixth state data When the fourth state data is generated before the generation time of , it is judged that the user has fallen out of bed, and a third signal is generated indicating that the user has slowly fallen and has fallen out of bed; when the third state data is received from the radar signal pattern analysis unit, and only the fourth state data is received from the thermal imaging data processing unit after the fourth time starting from the time when the third state data is received, a fourth signal is generated indicating that the user has a high risk of falling out of bed; when the first state data or the second state data is received from the radar signal pattern analysis unit, when the fourth state data, the fifth state data and the sixth state data are received in sequence from the thermal imaging data processing unit, and the first state data or the second state data is received again from the radar signal pattern analysis unit after the fourth time, and the sixth state data, the fifth state data and the fourth state data are received in sequence from the thermal imaging data processing unit, it is judged that the user who fell out of bed has returned to the original position, and a fifth signal is generated indicating that the user has returned to the original position.

[0027] The radar signal pattern analysis unit may analyze the separated biological signal and generate the sixth signal notifying a dangerous situation when a difference between the biological signal and a biological signal pattern of a predetermined period is greater than or equal to a predetermined range.

[0028] The non-contact falling-out-of-bed and sleep monitoring system further comprises a falling-out-of-bed risk determination unit.

[0029] The bed-falling danger level judgment unit may be configured as follows: when the sixth signal is continuously received from the radar signal pattern analysis unit within a fifth time, the first signal is received from the bed-falling judgment unit, and the fourth signal is not continuously received within the fifth time, it is judged that the user is continuously in a state of falling out of bed and the biological signal is poor, and a seventh signal is generated to notify the user that the danger level due to falling out of bed is very high; when the sixth signal is not received from the radar signal pattern analysis unit within the fifth time, and the fourth signal is not received from the bed-falling judgment unit within the fifth time, an eighth signal is generated indicating that the user is in a state of falling out of bed but the biological signal of the user is stable; and when the third signal is received from the bed-falling judgment unit, a ninth signal is generated to notify the user that the possibility of falling out of bed is high.

[0030] The non-face-to-face and non-contact bed fall monitoring system also includes an alarm unit, which can be configured to: when the seventh signal is received from the bed fall risk judgment unit, generate a first alarm signal indicating that the user is in a very dangerous state and send it to the preset guardian and medical staff; when the eighth signal is received from the bed fall risk judgment unit, generate a second alarm signal to notify the user of the fall bed status and send it to the preset guardian and medical staff; when the ninth signal is received from the bed fall risk judgment unit, generate a third alarm signal to notify the user that he is about to fall from the bed and send it to the preset guardian and medical staff.

[0031] The non-contact sleep monitoring system and method of the present invention include: a Doppler signal acquisition unit, which uses a radar to acquire a Doppler signal including biological activity information; an auxiliary signal processing unit, which uses a thermal imaging sensor to acquire a turning over action as an auxiliary signal; a Doppler signal analysis unit, which analyzes the Doppler signal and acquires spectrum energy with a preset period, determines whether the spectrum energy is acquired periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and a sleep interval definition unit, which uses the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the spectrum energy and the combination of the non-periodic spectrum energy to define the sleep state of each interval in the entire sleep interval.

[0032] The Doppler signal includes a breathing Doppler signal for acquiring breathing information of the user and a heartbeat Doppler signal for acquiring heartbeat information of the user.

[0033] The Doppler signal analysis unit may perform a fast Fourier transform on the Doppler signal to obtain the spectrum energy.

[0034] The sleep interval definition unit can define an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the entire sleep interval is greater than or equal to 5:5 and the non-periodic spectrum energy does not exist as a deep sleep interval, and define an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy is less than 5:5 and the non-periodic spectrum energy does not exist as an apnea interval.

[0035] The sleep interval definition unit may define an interval in which the non-periodic spectrum energy exists and is greater than or equal to a preset value as a turning over interval, and define an interval in the apnea interval in which the non-periodic spectrum energy is less than or equal to a preset value as a snoring interval.

[0036] According to another embodiment of the present invention, a non-contact sleep monitoring system and method include: a Doppler signal acquisition unit, which uses a radar to acquire a Doppler signal including biological activity information; an auxiliary signal processing unit, which uses a thermal imaging sensor to acquire a turning over action as an auxiliary signal; a Doppler signal analysis unit, which analyzes the Doppler signal and acquires spectrum energy with a preset period, determines whether the spectrum energy is acquired periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and a sleep interval definition unit, which uses a preset ratio range and the non-periodic spectrum energy to define the sleep state of the remaining sleep interval based on the average of the spectrum energy of the sleep entry interval that meets the preset benchmark in the entire sleep interval.

[0037] The non-contact sleep monitoring system and method of the present invention include: a Doppler signal acquisition step, in which a Doppler signal acquisition unit uses a radar to acquire a Doppler signal including biological activity information; an auxiliary signal processing step, in which the auxiliary signal processing unit uses a thermal imaging sensor to acquire a turning over action as an auxiliary signal; a Doppler signal analysis step, in which a Doppler signal analysis unit analyzes the Doppler signal and acquires spectrum energy with a preset period, determines whether the spectrum energy is acquired periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and a sleep interval definition step, in which a sleep interval definition unit uses the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the spectrum energy and a combination of the non-periodic spectrum energy to define the sleep state of each interval in the entire sleep interval.

[0038] The Doppler signal includes a breathing Doppler signal for acquiring breathing information of the user and a heartbeat Doppler signal for acquiring heartbeat information of the user.

[0039] In the Doppler signal analysis step, a fast Fourier transform may be performed on the Doppler signal to obtain the spectrum energy.

[0040] In the sleep interval definition step, the interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the entire sleep interval is greater than or equal to 5:5 and the non-periodic spectrum energy does not exist can be defined as a deep sleep interval, and the interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy is less than 5:5 and the non-periodic spectrum energy does not exist can be defined as an apnea interval.

[0041] In the sleep interval definition step, the interval in which the non-periodic spectrum energy exists and the non-periodic spectrum energy is greater than or equal to a preset size can be defined as a turning over interval, and the interval in the apnea interval in which the non-periodic spectrum energy is less than or equal to a preset size can be defined as a snoring interval.

[0042] The non-contact sleep monitoring system and method of the present invention include: a Doppler signal acquisition step, in which a Doppler signal acquisition unit uses a radar to acquire a Doppler signal including biological activity information; an auxiliary signal processing step, in which the auxiliary signal processing unit uses a thermal imaging sensor to acquire a turning over action as an auxiliary signal; a Doppler signal analysis step, in which a Doppler signal analysis unit analyzes the Doppler signal and acquires spectrum energy with a preset period, determines whether the spectrum energy is acquired periodically, and uses the auxiliary signal to distinguish the spectrum energy; and

[0043] In the sleep interval definition step, the sleep interval definition unit defines the sleep state of the remaining sleep intervals using a preset ratio range and non-periodic spectrum energy based on the average of the spectrum energy of the sleep entry intervals that meet the preset reference in the entire sleep interval.

[0044] Effects of the Invention

[0045] The present invention has the following advantages: the reliability of falling-out-of-bed monitoring is improved while protecting the privacy of the user.

[0046] Furthermore, the present invention has the following advantages: it adopts a non-face-to-face and non-contact method instead of a wearable method, and can determine in real time whether the user has fallen out of bed even if the user does not wear a separate device.

[0047] Furthermore, the present invention has the following advantages: when it is determined that the person has fallen out of bed, the degree of danger is determined and notified to the guardian or medical staff, thereby preventing or coping with the danger caused by the fall out of bed in advance.

[0048] Furthermore, the present invention has the following advantages: when monitoring a fall from bed, the thermal imaging video can be processed into a closed curve, a marked image and a spectrum, thereby protecting the privacy of the user.

[0049] Furthermore, the present invention has the following advantages: the user's risk level is analyzed based on the user's biological signal and the thermal imaging image after video processing, so that appropriate measures can be taken.

[0050] Furthermore, the present invention has the following advantages: using radar to obtain biological activity information of a user while sleeping does not affect the user's body and does not disturb the user's sleep.

[0051] Furthermore, the present invention has the following advantages: the user's breathing and heartbeat information can be used to accurately determine the user's sleep state in each sleep interval.

[0052] Furthermore, the present invention has the following advantages: the sleep monitoring method using the Doppler signal and the thermal imaging profile improves reliability compared with the traditional sleep monitoring method based on the Doppler signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A non-face-to-face and non-contact bed-fall monitoring system according to an embodiment of the present invention is shown.

[0054] Figure 2 FIG. 1 shows a movement pattern analyzed by a radar signal pattern analysis unit according to an embodiment of the present invention.

[0055] Figure 3 The process of determining the fourth state by analyzing the thermal imaging data is shown.

[0056] Figure 4 The process of determining the fifth state by analyzing the thermal imaging data is shown.

[0057] Figure 5 The process of determining the sixth state by analyzing the thermal imaging data is shown.

[0058] Figure 6 is a block diagram of a non-contact sleep monitoring system according to an embodiment of the present invention.

[0059] Figure 7 is a flow chart of a non-contact sleep monitoring method according to an embodiment of the present invention.

[0060] Figure 8 FIG. 1 is a diagram showing actual simulation experiment results using an embodiment of the present invention.

[0061] Fig. 9 The process of processing thermal imaging data by the auxiliary signal processing unit of the present invention is shown.

[0062] Best Mode for Carrying Out the Invention

[0063] The present invention is characterized in that it includes: a radar sensor unit, which is used to transmit radar to a user and then receive the radar reflected from the user, and use a Doppler radar algorithm to generate a time series Doppler radar signal; a thermal imaging sensor unit, which is used to photograph the user above the user to generate thermal imaging data; a radar signal pattern analysis unit, which is used to receive the time series Doppler radar signal from the radar sensor unit, and analyze the user's state based on the received time series Doppler radar signal; a thermal imaging data processing unit, which is used to receive the thermal imaging data generated by the thermal imaging sensor unit, process the received thermal imaging data and analyze the user's state; and a fall from bed judgment unit, which judges whether a fall from bed occurs based on the user's state data received from the radar signal pattern analysis unit and the thermal imaging data processing unit. DETAILED DESCRIPTION

[0064] According to one embodiment of the present invention, a non-face-to-face and non-contact bed fall monitoring system includes a radar sensor unit 100, a thermal imaging sensor unit 200, a radar signal pattern analysis unit 300, a thermal imaging data processing unit 400, a bed fall judgment unit 500, a bed fall risk judgment unit 600 and an alarm unit 700.

[0065] The non-face-to-face and non-contact bed-fall monitoring system according to an embodiment of the present invention can be installed in a user's sleeping space or in a living space.

[0066] 1. Radar sensor unit 100

[0067] The radar sensor unit 100 may transmit radar to a user and then receive radar reflected from the user, and use a Doppler radar algorithm to form a radar signal such as Figure 2 The time series Doppler radar signal shown in FIG. The time series Doppler radar signal means that the X axis represents time and the Y axis represents the magnitude of the Doppler radar signal. In the time series Doppler radar signal, the positive data and negative data of the Y axis are in opposite directions, and the magnitude of the amplitude, i.e., the absolute value of the Y axis, represents the speed of the user's movement or the movement of a large area.

[0068] 2. Thermal imaging sensor unit 200

[0069] The thermal imaging sensor unit 200 may be an infrared thermal imaging. The infrared thermal imaging camera may photograph the user above the user. The thermal imaging data of the user is sent to the judgment unit. Preferably, the infrared thermal imaging camera may be installed on the ceiling above the bed in the bedroom.

[0070] 3. Radar signal pattern analysis unit 300

[0071] The radar signal pattern analysis unit 300 may separate a movement change signal and a biological signal from the time series Doppler radar signal received by the radar sensor unit 100. Hereinafter, the time series Doppler radar signal separated into the movement change signal is referred to as a time series movement Doppler radar signal.

[0072] When analyzing the time series moving Doppler radar signal, the positive phase and negative phase of the Y axis represent opposite directions. Therefore, when the phase change of the Y axis changes from a positive phase to a negative phase, it means that it has changed to a second movement opposite to the first movement direction, such as from front to back, from back to front, from top to bottom, or from bottom to top. Figure 2 ,The Doppler time series data corresponding to falling out of bed showed a phase change of positive->negative->positive->negative->positive, which means that the movement of the user's entire body or part of the body changed in the opposite direction four times.

[0073] Furthermore, the magnitude of the amplitude of the time series moving Doppler radar signal (absolute value of the Y axis) means at least one of speed and movement amplitude. That is, a large amplitude means fast movement or a large movement area, such as movement of arms or legs, rather than a biological signal.

[0074] Therefore, when a large area of ​​the body such as an arm or leg changes direction quickly, the large amplitude phase of the Y axis can change from positive to negative or from negative to positive. In the present invention, the change of the Y axis phase from positive to negative or from negative to positive is called a yin-yang change.

[0075] The radar signal pattern analysis unit 300 is configured to generate first state data indicating a fall from bed when the amplitude of the time series moving Doppler radar signal changes from positive to negative after the amplitude of the data is greater than or equal to a predetermined size within a first time and then becomes 0.

[0076] In one embodiment, the radar signal pattern analysis unit 300 is configured to generate first state data indicating a fall from bed when the amplitude is greater than or equal to a predetermined range and the amplitude of the time series moving Doppler radar signal becomes 0 after a positive-negative change occurs within 200 ms.

[0077] Furthermore, the radar signal pattern analysis unit 300 can generate second state data indicating that the user is slowly falling and suspected of falling out of bed when a positive-negative change occurs in the time series moving Doppler radar signal with an amplitude greater than or equal to a predetermined size after the first time.

[0078] Furthermore, the radar signal pattern analysis unit 300 is based on the analysis of the time series moving Doppler radar signal. When the amplitude of the time series moving Doppler radar signal is greater than or equal to a predetermined range, a series moving Doppler radar signal occurs at a first time and a second time series moving Doppler radar signal occurs after a second time, a third state data indicating that the user is moving continuously although a fall from the bed is not generated can be generated.

[0079] When the time series mobile Doppler radar signal is analyzed to generate the second state data, there is a disadvantage that it is impossible to accurately determine whether the user has fallen out of bed.

[0080] Furthermore, when analyzing the time series mobile Doppler radar signal, contrary to falling out of bed, when the user jumps from under the bed to the bed, the direction of the limbs will change rapidly, resulting in a large amplitude of the mobile Doppler radar signal and a yin-yang change within a predetermined time, which may be mistakenly judged as falling out of bed.

[0081] Furthermore, when analyzing the time series mobile Doppler radar signal, when the user is lying on the edge of the bed and the possibility of falling out of bed is high, it is not determined as a fall out of bed, but is determined as a fall out of bed only after the fall actually occurs, which has the disadvantage of not being able to prevent the fall out of bed.

[0082] To compensate for this, the present invention simultaneously performs analysis of time series mobile Doppler radar signals and thermal imaging data analysis.

[0083] 4. Thermal imaging data processing unit 400

[0084] The thermal imaging data processing unit 400 may process the thermal imaging data received from the thermal imaging sensor unit 200 and analyze the state of the user.

[0085] The thermal imaging data processing unit 400 determines an analysis range in the received thermal imaging data. Then, the thermal imaging data processing unit 400 determines an edge of the user in the determined analysis range. Then, the thermal imaging data processing unit 400 processes the thermal imaging data using a preset threshold value so that the determined edge has a closed curve shape to obtain a closed curve video.

[0086] Afterwards, the thermal imaging data processing unit 400 may further perform video marking on the closed curve converted video to form a marked video. Afterwards, the thermal imaging data processing unit 400 may further perform spectrum conversion on the marked video to form a spectrum graph.

[0087] Figure 3The process of determining the fourth state by analyzing the thermal imaging data received from the thermal imaging sensor unit 200 is shown. The fourth state indicates that the user is located on an object at a predetermined height from the ground (for example, lying on a bed). Figure 3 a is the thermal imaging data received from the thermal imaging sensor unit 200, Figure 3 b is the thermal imaging data that determines the analysis range, Figure 3 c is the edge of the user, Figure 3 d is the user's closed curve video, Figure 3 e is the user's video after being marked, Figure 3 f is the result of spectrographically converting the marked user's video.

[0088] Figure 4 The fifth state is determined by analyzing the thermal imaging data received from the thermal imaging sensor unit 200. The fifth state indicates that the user is falling from an object (eg, a bed) at a predetermined height from the ground. Figure 4 a is the thermal imaging data received from the thermal imaging sensor unit 200, Figure 4 b is the thermal imaging data that determines the analysis range, Figure 4 c is the edge of the user, Figure 4 d is the user's closed curve video, Figure 4 e is the user's video after being marked, Figure 4 f is the result of spectrographically converting the marked user's video.

[0089] Figure 5 The sixth state is determined by analyzing the thermal imaging data received from the thermal imaging data processing unit 400. The sixth state indicates that the user has fallen from an object (eg, a bed) at a predetermined height above the ground. Figure 5 a is the thermal imaging data received from the thermal imaging sensor unit 200, Figure 5 b is the thermal imaging data that determines the analysis range, Figure 5 c is the edge of the user, Figure 5 d is the user's closed curve video, Figure 5 e is the user's video after being marked, Figure 5 f is the result of spectrographically converting the marked user's video.

[0090] The thermal imaging data processing unit 400 can be based on the closed curve videos 3d, 4d, 5d, the marked videos 3e, 4e, 5e and the spectrum Figure 3At least one of f, 4f, and 5f is divided into three states for judgment, namely, the state of lying on an object at a predetermined height (for example, a bed), the state of falling from an object at a predetermined height (for example, a bed), and the state of having fallen from an object at a predetermined height (for example, a bed), and the sixth state, and fourth state data indicating being in the fourth state, fifth state data indicating being in the fifth state, and sixth state data indicating being in the sixth state are generated, and the fourth state data to the sixth state data generated in this way and the generation time of the fourth state data to the sixth state data are sent to the bed fall judgment unit 500.

[0091] Preferably, the thermal imaging data processing unit 400 can judge the fourth state to the sixth state by analyzing the shape of the spectrum diagram, generate fourth state data indicating being in the fourth state, fifth state data indicating being in the fifth state, and sixth state data indicating being in the sixth state, and send the fourth state data to the sixth state data generated in this way and the generation time of the fourth state data to the sixth state data to the bed fall judgment unit 500.

[0092] 5. Falling from bed judgment unit 500

[0093] The bed-fall judgment unit 500 can match the first state data to the third state data received from the radar signal pattern analysis unit 300 and the fourth state data to the sixth state data received from the thermal imaging data processing unit 400 at the same time, and then match the first state data to the third state data and at least one of the fourth state data to the sixth state data to analyze whether a fall from bed occurs and the degree of danger of the fall from bed.

[0094] (1) Determining a fall from bed (generating a first signal)

[0095] The bed-fall judgment unit 500 can be configured as follows: when first state data judged as a first state is received from the radar signal pattern analysis unit 300, fourth state data, fifth state data and sixth state data are received in sequence from the thermal imaging data processing unit 400, and when the fourth state data is generated before the generation time of the first state data, it is judged that the user has fallen out of bed, a first signal indicating that the user has fallen out of bed is generated and sent to the bed-fall risk judgment unit 600.

[0096] (2) Determining that no fall from bed occurred (generating a second signal)

[0097] On the contrary, it can be configured as follows: when first state data judged as a first state is received from the radar signal pattern analysis unit 300, sixth state data, fifth state data and fourth state data are received in sequence from the thermal imaging data processing unit 400, and when the sixth state data is generated before the generation time of the first state data, it is judged that the user has jumped onto an object of a predetermined height instead of falling out of bed, and a second signal indicating that no fall from bed has occurred is generated and sent to the fall from bed risk judgment unit 600.

[0098] (3) Determined as a slow fall from bed (generating a third signal)

[0099] The bed-fall judgment unit 500 can be configured as follows: when second state data indicating a suspected fall from bed is received from the radar signal pattern analysis unit 300, fourth state data, fifth state data and sixth state data are received in sequence from the thermal imaging data processing unit 400, and when the fourth state data is generated before the generation time of the first state data, it is judged that the user has fallen from bed, and a third signal indicating that the user has slowly fallen and the user has fallen from bed is generated and sent to the bed-fall risk judgment unit 600.

[0100] (4) Determining that the risk of falling out of bed is high (generating a fourth signal)

[0101] The bed-falling judgment unit 500 may be configured as follows: when third state data judged as the third state is received from the radar signal pattern analysis unit 300, and only fourth state data is received from the thermal imaging data processing unit 400 after a third time from the moment the third state data is received, it is judged that the user has a high risk of falling out of bed, a fourth signal indicating a high risk of falling out of bed is generated and sent to the bed-falling hazard judgment unit 600.

[0102] (5) Determined to be back to the original position (generating the fifth signal)

[0103] The bed-fall judgment unit 500 may be configured as follows: when first state data judged as a first state is received from the radar signal pattern analysis unit 300, fourth state data, fifth state data and sixth state data are received in sequence from the thermal imaging data processing unit 400, and after a fourth time, the first state data judged as the first state is received again from the radar signal pattern analysis unit 300, and the sixth state data, the fifth state data and the fourth state data are received in sequence from the thermal imaging data processing unit 400, it is judged that the user who has fallen out of bed has returned to the original position, a fifth signal indicating that the user has returned to the original position is generated and sent to the bed-fall risk judgment unit 600.

[0104] (5) Biosignal difference (generating the sixth signal)

[0105] The radar signal pattern analysis unit 300 analyzes the separated biological signal, and when the difference between the biological signal and the biological signal pattern during the preset period is greater than or equal to a predetermined range, generates a sixth signal notifying a dangerous situation and sends it to the bed-falling risk determination unit 600. For example, the radar signal pattern analysis unit 300 can be configured to: when the heart rate or respiratory rate as a biological signal decreases by more than 50% compared with the average heart rate or average respiratory rate for one month or one week, it is determined that the biological signal of the user is poor, and the sixth signal indicating that the user's state is dangerous is generated.

[0106] 6. Falling risk assessment unit 600

[0107] The bed-falling danger level judgment unit 600 may receive the sixth signal from the radar signal pattern analysis unit 300 and the first to fifth signals from the bed-falling judgment unit 500, and classify and analyze the danger level caused by the user's bed-falling to generate signals representing various dangers.

[0108] (1) The risk of falling out of bed is very high (generating the seventh signal)

[0109] The bed-falling danger level judgment unit 600 may be configured as follows: when the sixth signal is continuously received from the radar signal pattern analysis unit 300 within a fifth time, the first signal is received from the bed-falling judgment unit 500, and the fourth signal is not continuously received within the fifth time, it is judged that the user is continuously in a bed-falling state (i.e., the user fails to climb out of bed after falling out of bed), and the biological signal continues to be in a poor state within a predetermined time, a seventh signal is generated to notify the user that the danger level caused by falling out of bed is very high and is sent to the alarm unit 700.

[0110] (2) The risk of falling out of bed is very high (generating the seventh signal)

[0111] The fall-out-of-bed danger level judgment unit 600 may be configured as follows: when the sixth signal is continuously received from the radar signal pattern analysis unit 300 within a fifth time, the third signal is received from the fall-out-of-bed judgment unit 500, and the fourth signal is not continuously received within the third time, it is judged that the user is continuously in a fall-out-of-bed state (i.e., fails to climb out of bed after falling out of bed), and the biological signal continues to be in a poor state within a predetermined time, a seventh signal is generated to notify the user that the danger level caused by falling out of bed is very high and is sent to the alarm unit 700.

[0112] (3) The patient continues to fall out of bed but the biological signal is stable (generating the eighth signal)

[0113] The fall-out-of-bed danger level judgment unit 600 may be configured as follows: when the sixth signal is not received continuously from the radar signal pattern analysis unit 300 within a fifth time, the third signal is received from the fall-out-of-bed judgment unit 500, and the fourth signal is not received continuously within the third time, it is judged that although the user continues to be in a fall-out-of-bed state (i.e., fails to climb out of bed after falling out of bed), the biological signal is not bad, an eighth signal is generated to notify the user that the danger level caused by falling out of bed is not high and the signal is sent to the alarm unit 700.

[0114] (4) High probability of falling out of bed (generating the ninth signal)

[0115] The bed-fall risk determination unit 600 may be configured to generate a ninth signal notifying the user that there is a high possibility of falling out of bed and send the ninth signal to the alarm unit 700 when receiving the fourth signal from the bed-fall determination unit 500 .

[0116] (4) Not falling out of bed (no signal generated)

[0117] The bed-fall risk level determination unit 600 may be configured to: when receiving the second signal from the bed-fall determination unit 500, analyze that the risk level of the user's bed-fall state is not high, and not generate a separate signal.

[0118] (5) The falling state ends and the biological signal is stable (no signal is generated)

[0119] The falling-out-of-bed danger judgment unit 600 may be configured as follows: when the sixth signal is not received from the radar signal pattern analysis unit 300 within the fifth time, and the fourth signal is received within the fifth time, it is judged that the biological signal of the user is stable, and the falling-out-of-bed state of the user is ended (i.e., the user has climbed onto the bed by himself), and there is no risk to the user's safety, and no separate signal is generated.

[0120] 7. Alarm Department 700

[0121] When receiving the seventh signal from the fall-out-risk determination unit 600 , the alarm unit 700 may generate a first alarm signal to notify the user that the user is in a fall-out-of-bed state and has a poor biological signal, and send the first alarm signal to a preset guardian and medical staff.

[0122] When the alarm unit 700 receives the eighth signal from the fall-out-risk determination unit 600 , it can notify the user of the second alarm signal that the biological signal is stable even though the user is in a fall-out-of-bed state, and send the second alarm signal to a preset guardian and medical staff.

[0123] When the alarm unit 700 receives the ninth signal from the fall-out-risk determination unit 600, it may generate a third alarm signal notifying the user that the user is about to fall out of bed and send the third alarm signal to a preset guardian or medical staff. The guardian or medical staff may take appropriate measures to prevent the user from falling out of bed upon receiving the third alarm signal.

[0124] Figure 6 1 is a block diagram of a non-contact sleep monitoring system according to an embodiment of the present invention. The non-contact sleep monitoring system 1 according to an embodiment of the present invention is installed in a space where a user lives and sleeps, such as a bedroom, and when the user falls asleep, the sleep can be monitored and Doppler signals of biological activities can be acquired, and the acquired Doppler signals can be analyzed and the user's sleep state can be defined. To this end, the present invention can use a biological activity measuring device using a radar pre-installed in the above space. Figure 6 As shown, the non-contact sleep monitoring system 1 according to an embodiment of the present invention includes a Doppler signal acquiring unit 11 , an auxiliary signal processing unit 13 , a Doppler signal analyzing unit 15 and a sleep interval defining unit 17 .

[0125] The Doppler signal acquisition unit 11 acquires a Doppler signal including biological activity information from a radar installed in the user's sleeping space. The Doppler signal includes the user's breathing information (respiratory Doppler signal) and the user's heartbeat information (heartbeat Doppler signal). The Doppler signal can be defined as a signal including a Doppler frequency generated by the user's movement when the radar transmits radio waves to the user and the transmitted radio waves are returned. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal can be Doppler signals of the movement changes of body organs that are movable during breathing and body organs that are movable during heartbeat.

[0126] The auxiliary signal processing unit 13 acquires auxiliary signals using an auxiliary biological activity information acquisition device also installed in the user's sleeping space, and analyzes the acquired auxiliary signals to acquire information such as movement of other organs (turning over, etc.) other than the user's breathing or heartbeat movement. To this end, the auxiliary signal processing unit 13 may acquire measurement results from a thermal imaging sensor as an auxiliary biological activity information acquisition device. The thermal imaging sensor may be used to measure the user's body movement.

[0127] The auxiliary signal processing unit 13 may use the thermal imaging sensor to continuously obtain the contour information of the user's body. When the thermal imaging sensor is used to continuously obtain the contour information of the user's body, the contour information generated by the user's actions such as turning over can be obtained.

[0128] Preferably, the auxiliary signal processing unit 13 can analyze the thermal imaging data obtained from the thermal imaging sensor to continuously obtain the user's movement information. In the present invention, in order to protect the user's privacy, the thermal imaging data is not used as it is, but the user's movement information is obtained through profile information or spectrum diagram.

[0129] The auxiliary signal processing unit 13 can process the thermal imaging data received from the thermal imaging sensor to analyze the movement of the user.

[0130] The auxiliary signal processing unit 13 determines an analysis range in the received thermal imaging data. Then, the auxiliary signal processing unit 13 determines an edge of the user in the determined analysis range. Then, the auxiliary signal processing unit 13 can process the thermal imaging data using a preset threshold value so that it has a closed curve shape from the determined edge to obtain a closed curve video.

[0131] Afterwards, the auxiliary signal processing unit 13 may further perform video marking on the closed curve converted video to form a marked video. Afterwards, the auxiliary signal processing unit 13 may further perform spectrum conversion on the marked video to form a spectrum graph.

[0132] Fig. 9 The structure of the auxiliary signal processing unit 13 acquiring the movement information of the user is shown. Figure 4 a is the thermal imaging data acquired by the thermal imaging sensor, Figure 4 b is the thermal imaging data that determines the analysis range, Figure 3 c is the edge of the user, Figure 4 d is the user's closed curve video, Figure 3 e is the user's video after being marked, Figure 4 f is the result of spectrographically converting the marked user's video.

[0133] The body contour information acquired in the auxiliary signal processing unit 13 can be used in the Doppler signal analysis unit 15 described later. For example, when the Doppler signal analysis unit 15 acquires non-periodic spectral energy using acquisition time information of turning over or snoring action represented as non-periodic spectral energy, it can be used as auxiliary information for determining what kind of biological activity information the energy is.

[0134] The Doppler signal analysis unit 15 analyzes the Doppler signal acquired by the Doppler signal acquisition unit 11, and acquires spectrum energy at a preset period. The Doppler signal analysis unit 15 applies a preset function to the Doppler signal to acquire spectrum energy, and analyzes the acquired spectrum energy according to a preset reference, so as to acquire spectrum energy of the respiratory Doppler signal and the heartbeat Doppler signal, that is, respiratory spectrum energy and heartbeat spectrum energy.

[0135] In one embodiment of the present invention, the Doppler signal analysis unit 15 can use fast Fourier transform (FFT) as a preset function to obtain spectrum energy from the Doppler signal at a preset period. Fourier transform is a well-known function, which refers to a transformation that decomposes a signal sampled in time or space into time frequency or space frequency components. Using this function, the Doppler signal analysis unit 15 can transform the Doppler signal obtained at a preset period into the spectrum energy of a specific frequency component.

[0136] After acquiring the spectral energy of a specific frequency component, the Doppler signal analysis unit 15 according to an embodiment of the present invention confirms whether the acquired spectral energy of the specific frequency component is a periodically generated spectral energy. The Doppler signal analysis unit 15 can continuously acquire the spectral energy of the acquired Doppler signal of a preset period, and determine whether the obtainable continuous spectral energy has periodicity. When it has periodicity, the corresponding spectral energy can be determined as the spectral energy of breathing or heartbeat, and when it does not have periodicity, the corresponding spectral energy can be determined as not the spectral energy of breathing or heartbeat.

[0137] The Doppler signal analysis unit 15 can determine whether the non-periodic spectrum energy has an energy greater than a preset value, and if it has an energy greater than the preset value, the corresponding spectrum energy is determined to be the spectrum energy of the Doppler frequency generated by the user's body movement. Alternatively, if it has an energy less than or equal to the preset value, the corresponding spectrum energy is determined to be the spectrum energy of the Doppler frequency generated by the user's snoring.

[0138] Alternatively, the Doppler signal analysis unit 15 according to an embodiment of the present invention may use a preset judgment algorithm to analyze the periodic spectrum energy, and as a result, the respiratory spectrum energy and the heartbeat spectrum energy may be obtained by distinction.

[0139] Furthermore, the Doppler signal analysis unit 15 according to an embodiment of the present invention can understand the type of non-periodic spectrum energy using the auxiliary signal obtained by the auxiliary signal processing unit 13. As an example, the Doppler signal analysis unit 15 can use thermal imaging information to obtain the non-periodic spectrum energy of the user's turning over (body movement), and judge the obtained non-periodic spectrum energy as a noise signal, and remove it from the sleep interval analysis data described later.

[0140] Or, as another example, the Doppler signal 15 may define a preset energy size capable of distinguishing body movement from snoring using an auxiliary signal obtained by a thermal imaging sensor without defining a preset energy size for distinguishing the two movements using non-periodic spectrum energy.

[0141] According to an embodiment of the present invention, the Doppler signal analysis unit 15 continuously obtains the spectrum energy of a specific frequency component. If the determination of whether periodicity occurs is completed, the sleep interval definition unit 17 uses the analysis result to define the sleep state of each interval in the entire sleep interval.

[0142] The entire sleep interval includes at least one of a sleep entry interval, a deep sleep interval, a tossing interval, an apnea interval, and a snoring interval. The sleep entry interval refers to the interval in which the user enters sleep, and is an interval that can be used to determine whether sleep has entered or not in almost all users according to statistics. Alternatively, the deep sleep interval refers to a sleep interval in which the user's body activities are stably maintained, and the tossing interval refers to a sleep interval in which there is movement in the user's body activities. In addition, the apnea interval refers to a sleep interval in which no breathing activity occurs in the user's body activities, and the snoring interval refers to a sleep interval in which snoring occurs in the user's body activities.

[0143] The higher the ratio of deep sleep intervals in the sleep intervals and the lower the ratio of the remaining intervals, the higher the sleep quality. Therefore, by defining each sleep interval in the present invention, the sleep quality information of the corresponding user can be obtained. Moreover, the information can be used to provide a prescription corresponding to the sleep quality for each user later.

[0144] The sleep interval definition unit 17 can define the sleep state of each interval in the entire sleep interval using the analysis result of the Doppler signal analysis unit 15. As described above, each interval in the entire sleep interval includes at least one of a sleep entry interval, a deep sleep interval, a tossing interval, an apnea interval, and a snoring interval. The sleep interval definition unit 17 obtains the analysis result from the Doppler signal analysis unit 15, and uses a preset state judgment standard to confirm which sleep interval the corresponding analysis result belongs to.

[0145] The preset state judgment standard can be a preset value using the ratio of the respiratory spectrum energy to the heartbeat spectrum energy obtained by the Doppler signal analysis unit 15. As described above, since the magnitude of the body activity occurring under normal breathing movements is greater than the magnitude of the body activity occurring under heartbeat movements, the magnitude of the respiratory spectrum energy in a stable sleep interval appears to be greater than the magnitude of the heartbeat spectrum energy. Alternatively, high-frequency vibrations may be captured in the snoring interval, and additional spectra other than the respiratory spectrum energy and the heartbeat spectrum energy may appear. Furthermore, no respiratory movement or weak respiratory movement occurs in the no-breathing interval, so the magnitude of the respiratory spectrum energy may be reduced to the magnitude of the heartbeat spectrum energy.

[0146] The sleep interval definition unit 17 of an embodiment of the present invention may define an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy, ie, the spectrum ratio is greater than or equal to 5:5, as a deep sleep interval.

[0147] Alternatively, the sleep interval definition unit 17 may define an interval in which the non-periodic spectrum energy is less than or equal to a preset size in an interval in which the spectrum ratio is less than 5:5 as a snoring interval. If the non-periodic spectrum energy is not obtained within a preset time, the corresponding interval may be defined as an apnea interval.

[0148] In another embodiment of the present invention, the sleep interval definition unit 17 may define the sleep interval using the sleep entry interval in addition to the spectrum ratio. In the sleep entry interval, similar to the deep sleep interval, generally, there is no turning over, no breathing, and no snoring. Therefore, the sleep interval definition unit 17 of the present invention may first obtain the sleep entry interval, and determine the deep sleep interval based on the obtained sleep entry interval.

[0149] on the other hand, Figure 7 FIG. 1 is a flow chart showing a non-contact sleep monitoring method according to an embodiment of the present invention. Figure 6 The non-contact sleep monitoring method of the present invention is described by taking a system as an example, but the present invention is not limited to this.

[0150] The non-contact sleep monitoring method 10 according to an embodiment of the present invention is installed in a space where the user lives and sleeps, such as a bedroom, and when the user falls asleep, the sleep can be monitored and the Doppler signal of biological activity can be obtained, and the obtained Doppler signal can be analyzed and the user's sleep state can be defined. To this end, the present invention can use a biological activity measuring device using radar pre-installed in the above space. Figure 7 As shown, the non-contact sleep monitoring method 10 according to an embodiment of the present invention includes a Doppler signal acquisition step S11, an auxiliary signal processing step S13, a Doppler signal analysis step S15 and a sleep interval definition step S17.

[0151] In the Doppler signal acquisition step S11, the Doppler signal acquisition unit acquires a Doppler signal including biological activity information from a radar installed in the user's sleeping space. The Doppler signal includes the user's breathing information (respiratory Doppler signal) and the user's heartbeat information (heartbeat Doppler signal). The Doppler signal can be defined as a signal including a Doppler frequency generated by the user's movement when the radar transmits radio waves to the user and the transmitted radio waves are returned. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal can be Doppler signals of the movement changes of body organs that are movable during breathing and body organs that are movable during heartbeat.

[0152] In the auxiliary signal processing step S13, an auxiliary biological activity information acquisition device also installed in the user's sleeping space is used to acquire an auxiliary signal, and the acquired auxiliary signal is analyzed to obtain information such as movement of other organs (turning over, etc.) or noise other than the user's breathing or heartbeat movement. To this end, in the auxiliary signal processing step S13, a measurement result can be acquired from a thermal imaging sensor as an auxiliary biological activity information acquisition device. The thermal imaging sensor can be used to measure the user's body movement.

[0153] In the auxiliary signal processing step S13, the thermal imaging sensor may be used to continuously obtain the contour information of the user's body. When the thermal imaging sensor is used to continuously obtain the contour information of the user's body, the contour information generated by the user's actions such as turning over can be obtained.

[0154] The body contour information and noise information acquired in the auxiliary signal processing step S13 can be used in the Doppler signal analysis step S15 described later. For example, when the acquisition time information of the turning over action or snoring action represented as non-periodic spectral energy is used to acquire the non-periodic spectral energy in the Doppler signal analysis step S15, it can be used as auxiliary information for determining what kind of biological activity information the energy is.

[0155] In the Doppler signal analysis step S15, the Doppler signal acquired in the Doppler signal acquisition step S11 is analyzed to acquire spectrum energy at a preset period. The Doppler signal analysis step S15 applies a preset function to the Doppler signal to acquire spectrum energy, and analyzes the acquired spectrum energy according to a preset reference, so that the spectrum energy of the respiratory Doppler signal and the heartbeat Doppler signal, i.e., the respiratory spectrum energy and the heartbeat spectrum energy, can be acquired.

[0156] In one embodiment of the present invention, in the Doppler signal analysis step S15, a fast Fourier transform (FFT) can be used as a preset function to obtain spectral energy from the Doppler signal at a preset period. Fourier transform is a well-known function, which refers to a transformation that decomposes a signal sampled in time or space into time frequency or space frequency components. Using this function, the Doppler signal obtained at a preset period can be transformed into spectral energy of a specific frequency component in the Doppler signal analysis step S15.

[0157] After acquiring the spectral energy of a specific frequency component, the Doppler signal analysis step S15 according to an embodiment of the present invention confirms whether the acquired spectral energy of the specific frequency component is a periodically generated spectral energy. The Doppler signal analysis step S15 can continuously acquire the spectral energy of the acquired Doppler signal of a preset period, and determine whether the obtainable continuous spectral energy has periodicity. When it has periodicity, the corresponding spectral energy can be determined as the spectral energy of breathing or heartbeat, and when it does not have periodicity, the corresponding spectral energy can be determined as not the spectral energy of breathing or heartbeat.

[0158] The Doppler signal analysis step S15 can determine whether the non-periodic spectrum energy has an energy greater than a preset value, and if it has an energy greater than the preset value, the corresponding spectrum energy is determined to be the spectrum energy of the Doppler frequency generated by the user's body movement. Alternatively, if it has an energy less than or equal to the preset value, the corresponding spectrum energy is determined to be the spectrum energy of the Doppler frequency generated by the user's snoring.

[0159] Alternatively, according to an embodiment of the present invention, the Doppler signal analysis step S15 may use a preset judgment algorithm to analyze the periodic spectrum energy, and as a result, the respiratory spectrum energy and the heartbeat spectrum energy may be distinguished and acquired.

[0160] Furthermore, in the Doppler signal analysis step S15 according to an embodiment of the present invention, the auxiliary signal obtained in the Doppler signal analysis step S15 can be used to understand the type of non-periodic spectrum energy. As an example, in the Doppler signal analysis step S15, thermal imaging information can be used to obtain the non-periodic spectrum energy of the user's turning over (body movement), and the obtained non-periodic spectrum energy is determined as a noise signal and removed from the sleep interval analysis data described later.

[0161] Alternatively, as another example, in the Doppler signal analysis step S15, without defining a preset size of energy for distinguishing between body movement and snoring using non-periodic spectrum energy, an auxiliary signal obtained by a thermal imaging sensor can be used to define a preset size of energy that can distinguish between the two movements.

[0162] According to an embodiment of the present invention, in the Doppler signal analysis step S15, the spectrum energy of a specific frequency component is continuously obtained. If the judgment on whether the periodicity is generated is completed, the analysis result is used in the sleep interval definition step S17 to define the sleep state of each interval in the entire sleep interval.

[0163] The entire sleep interval includes at least one of a sleep entry interval, a deep sleep interval, a tossing interval, an apnea interval, and a snoring interval. The sleep entry interval refers to the interval in which the user enters sleep, and is an interval that can be used to determine whether sleep has entered or not in almost all users according to statistics. Alternatively, the deep sleep interval refers to a sleep interval in which the user's body activities are stably maintained, and the tossing interval refers to a sleep interval in which there is movement in the user's body activities. In addition, the apnea interval refers to a sleep interval in which no breathing activity occurs in the user's body activities, and the snoring interval refers to a sleep interval in which snoring occurs in the user's body activities.

[0164] The higher the ratio of deep sleep intervals in the sleep intervals and the lower the ratio of the remaining intervals, the higher the sleep quality. Therefore, by defining each sleep interval in the present invention, the sleep quality information of the corresponding user can be obtained. Moreover, the information can be used to provide a prescription corresponding to the sleep quality for each user later.

[0165] In the sleep interval definition step S17, the analysis result of the Doppler signal analysis step S15 can be used to define the sleep state of each interval in the entire sleep interval. As described above, each interval in the entire sleep interval includes at least one of a sleep entry interval, a deep sleep interval, a turning over interval, an apnea interval, and a snoring interval. In the sleep interval definition step S17, the analysis result is obtained from the Doppler signal analysis step S15, and the preset state judgment standard is used to confirm which sleep interval the corresponding analysis result belongs to.

[0166] The preset state judgment standard can be a preset value using the ratio of the respiratory spectrum energy to the heartbeat spectrum energy obtained in the Doppler signal analysis step S15. As described above, since the magnitude of the body activity occurring under normal breathing movements is greater than the magnitude of the body activity occurring under heartbeat movements, the magnitude of the respiratory spectrum energy in a stable sleep interval appears to be greater than the magnitude of the heartbeat spectrum energy. Alternatively, high-frequency vibrations may be captured in the snoring interval, and additional spectra other than the respiratory spectrum energy and the heartbeat spectrum energy may appear. Furthermore, no respiratory movement or weak respiratory movement occurs in the no-breathing interval, so the magnitude of the respiratory spectrum energy may be reduced to the magnitude of the heartbeat spectrum energy.

[0167] In the sleep interval definition step S17 of an embodiment of the present invention, an interval in which the ratio of respiratory spectrum energy to heartbeat spectrum energy, ie, a spectrum ratio greater than or equal to 5:5, may be defined as a deep sleep interval.

[0168] Alternatively, in the sleep interval definition step S17, the interval in which the non-periodic spectrum energy is less than or equal to a preset size in the interval in which the spectrum ratio is less than 5:5 can be defined as a snoring interval. If the non-periodic spectrum energy is not obtained within a preset time, the corresponding interval can be defined as an apnea interval.

[0169] In another embodiment of the present invention, in the sleep interval definition step S17, in addition to the spectrum ratio, the sleep entry interval can also be used to define the sleep interval. In the sleep entry interval, similar to the deep sleep interval, there is usually no turning over, no breathing, and no snoring. Therefore, in the sleep interval definition step S17 of the present invention, the sleep entry interval can be first obtained, and the deep sleep interval can be determined based on the obtained sleep entry interval.

[0170] on the other hand, Figure 8 FIG. 2 is a diagram showing the results of an actual simulation experiment using an embodiment of the present invention. Figure 8 , the entire sleep interval can be represented as A. The B interval is the sleep entry interval as described above. According to statistics, most users enter sleep in the interval between 10 minutes and 40 minutes, and the average is 20 minutes to 30 minutes when the user actually starts sleeping. The C interval is a stable sleep interval, that is, a deep sleep interval, D is a snoring interval, E is a tossing interval, and F is a no breathing interval.

[0171] Interval C is a stable deep sleep interval, at which time the size of the respiratory spectrum energy a1 is sufficiently larger than the size of the heartbeat spectrum energy b1. Moreover, from the periodicity of the spectrum energy, it can be seen that the spectrum energy does not contain irregular peaks and repeats in a regular period. Therefore, in one embodiment of the present invention, this sleep interval can be defined as a deep sleep interval. At this time, as described above, in the present invention, the situation where the ratio of the respiratory spectrum energy a1 to the heartbeat spectrum energy b1 is greater than or equal to 5:5 is defined as a deep sleep interval. In another embodiment, an interval with a periodicity greater than or equal to a preset similarity and an energy less than or equal to a preset size based on interval B (sleep entry interval) can be defined as a deep sleep interval.

[0172] The D interval is a snoring interval. Although the magnitude of the respiratory spectrum energy a2 is greater than the magnitude of the heartbeat spectrum b2, a snoring spectrum energy c2 that does not appear in other intervals is further detected. It can be seen that high-frequency spectrum energy, such as C2, with a frequency higher than the respiratory / heartbeat frequency is detected in the D interval. Therefore, the sleep interval definition unit and the sleep interval definition step of the present invention can define the D interval as a snoring interval in which the user is in a snoring state.

[0173] The F interval is a anesthetic interval, in which the magnitude of the respiratory spectrum energy a3 is similar to the magnitude of the heartbeat spectrum energy b3. More specifically, the ratio of the respiratory spectrum energy a to the heartbeat spectrum energy b3 is less than 5:5, and unlike the D interval, it is an interval in which the snoring spectrum energy c2 is not detected. This is because, in the process of using radar to obtain Doppler signals, the user's body respiratory activity is weak or non-existent, resulting in a significant reduction in the magnitude of the acquired respiratory spectrum energy a3. Therefore, Figure 8 As shown in the spectrum analysis results of the F interval, when comparing the size of the respiratory spectrum energy a3 and the size of the heartbeat spectrum energy b3, if the ratio is less than 5:5, the F interval can be defined as a apnea interval in the sleep interval definition unit and the sleep interval definition step according to one embodiment of the present invention.

[0174] The E interval is an interval where turning over occurs, and at this time, a spectrum energy of a higher magnitude is obtained compared to the spectrum energy detected in the B, C, D, F intervals, etc. This is because a biological activity with much higher energy than the respiratory spectrum energy or the heartbeat spectrum energy occurs, and in the present invention, this biological activity is analyzed as turning over.

[0175] In summary, Figure 6 and Figure 7 The system and method use radar to obtain Doppler signals of the user's biological activities, wherein the breathing spectrum energy and the heartbeat spectrum energy are used to define the sleep state according to each sleep interval. The experimental results are as follows Figure 8 As shown, Figures 6 to 8 The present invention described in the disclosure has the effect that it is possible to confirm the sleep state of a user and use the confirmed sleep state to respond to an emergency or provide content for improving the sleep quality of the user.

[0176] Although one embodiment of the present invention is described above, the spirit of the present invention is not limited to the embodiment given in this specification, and those skilled in the art who understand the spirit of the present invention can easily derive other embodiments by adding, changing, deleting, and adding components within the same spirit, but this will also be considered to be within the spirit of the present invention.

[0177] Industrial Applicability

[0178] The non-face-to-face and non-contact bed fall monitoring system, non-contact sleep monitoring system and method of the present invention are characterized in that the reliability of bed fall monitoring is improved while protecting the privacy of the user. A non-face-to-face and non-contact method is adopted instead of a wearable method, so that even if the user does not wear a separate device, the user's bed fall can be judged in real time. When it is judged as a fall, the degree of danger is judged and the guardian or medical staff is notified, so as to prevent or respond to the danger caused by the fall in advance. When monitoring the fall, the thermal imaging video can be processed into a closed curve, marked as an image and spectrally, so as to protect the privacy of the user. The user's degree of danger is analyzed based on the user's biological signal and the thermal imaging image processed by the video, so that appropriate response can be made. The user's biological activity information during sleep is obtained by using radar, which does not affect the user's body and does not interfere with the user's sleep. The user's breathing and heartbeat information is used to accurately judge the user's sleep state in each sleep interval.

Claims

1. A non-face-to-face and non-contact bed fall monitoring system, characterized in that: include: A radar sensor unit, configured to transmit radar to a user and then receive radar reflected from the user, and generate a time series Doppler radar signal using a Doppler radar algorithm; A thermal imaging sensor unit, configured to photograph the user at an upper part of the user to generate thermal imaging data; a radar signal pattern analysis unit, configured to receive the time series Doppler radar signal from the radar sensor unit, and analyze the user's state based on the received time series Doppler radar signal; a thermal imaging data processing unit, configured to receive the thermal imaging data generated by the thermal imaging sensor unit, process the received thermal imaging data and analyze the state of the user; and The bed-falling judgment unit judges whether a bed-falling occurs based on the user's status data received from the radar signal pattern analysis unit and the thermal imaging data processing unit.

2. The non-face-to-face and non-contact bed-fall monitoring system according to claim 1, characterized in that: The radar signal pattern analysis unit is configured to: A movement change signal and a biological signal are separated from the time series Doppler radar signal received by the radar sensor unit, and the separated movement change signal is used as a time series movement Doppler radar signal to analyze the state of the user.

3. The non-face-to-face and non-contact bed-fall monitoring system according to claim 2, characterized in that: The radar signal pattern analysis unit is configured to: When the time series moving Doppler radar signal undergoes a positive-negative change with an amplitude greater than or equal to a predetermined size within a first time, and the amplitude becomes 0, the first state data indicating that a fall from the bed has occurred is generated. When the time series moving Doppler radar signal undergoes a positive-negative change with an amplitude greater than or equal to a predetermined size after the first time, second state data indicating that the user is slowly falling and suspected of falling out of bed is generated.

4. The non-face-to-face and non-contact bed-fall monitoring system according to claim 3, characterized in that: The thermal imaging data processing unit processes the thermal imaging data by executing the following steps: determining an analysis scope based on the received thermal imaging data; determining the edge of the user within the determined analysis range; The thermal imaging data is processed using a preset threshold value so that the thermal imaging data has a closed curve shape from the determined edge to obtain a closed curve video.

5. The non-face-to-face and non-contact bed-fall monitoring system according to claim 4, characterized in that: The thermal imaging data processing unit further performs the following steps to form a marked video: The video after the closed curve conversion is performed with video marking.

6. The non-face-to-face and non-contact bed-fall monitoring system according to claim 5, characterized in that: The thermal imaging data processing unit further performs the following steps to form a spectrum diagram: The labeled video is spectrally converted.

7. The non-face-to-face and non-contact bed-fall monitoring system according to any one of claims 3 to 5, characterized in that: The thermal imaging data processing unit determines the state of the user as one of the fourth state to the sixth state based on at least one of the closed curve video, the marked video and the spectrum diagram, and generates fourth state data to sixth state data respectively. The fourth state data indicates that the user is located on an object at a predetermined height from the ground. The fifth state data indicates that the user is falling from an object at a predetermined height from the ground. The sixth state data indicates a state in which the user has fallen from an object at a predetermined height from the ground.

8. The non-face-to-face and non-contact bed-fall monitoring system according to claim 7, characterized in that: The bed-fall determination unit is configured as follows: When the first state data is received from the radar signal pattern analysis unit, the fourth state data, the fifth state data and the sixth state data are received in sequence from the thermal imaging data processing unit, and when the fourth state data is generated before the generation time of the first state data, it is determined that the user has fallen out of bed, and a first signal indicating that the user has fallen out of bed is generated, When first state data determined as a first state is received from the radar signal pattern analysis unit, sixth state data, fifth state data, and fourth state data are received in sequence from the thermal imaging data processing unit, and the sixth state data is generated before the generation time of the first state data, it is determined that the user has not fallen out of bed, and a second signal indicating that the user has not fallen out of bed is generated, When second state data indicating suspected falling out of bed is received from the radar signal pattern analysis unit, fourth state data, fifth state data and sixth state data are received in sequence from the thermal imaging data processing unit, and when the fourth state data is generated before the generation time of the first state data, it is determined that the user has fallen out of bed, and a third signal indicating that the user has slowly fallen and the user has fallen out of bed is generated, When the third state data is received from the radar signal pattern analysis unit, and only the fourth state data is received from the thermal imaging data processing unit after a fourth time from the moment when the third state data is received, a fourth signal is generated indicating that the user has a high risk of falling out of bed, When the first state data or the second state data is received from the radar signal pattern analysis unit, when the fourth state data, the fifth state data and the sixth state data are received in sequence from the thermal imaging data processing unit, and after the fourth time, the first state data or the second state data is received again from the radar signal pattern analysis unit, and the sixth state data, the fifth state data and the fourth state data are received in sequence from the thermal imaging data processing unit, it is determined that the user who fell out of bed has returned to the original position, and a fifth signal indicating that the user has returned to the original position is generated.

9. The non-face-to-face and non-contact bed-fall monitoring system according to claim 8, characterized in that: The radar signal pattern analysis unit analyzes the separated biological signal and generates the sixth signal notifying a dangerous situation when a difference between the biological signal and a biological signal pattern of a predetermined period is greater than or equal to a predetermined range.

10. The non-face-to-face and non-contact bed-fall monitoring system according to claim 9, characterized in that: The non-contact bed-fall and sleep monitoring system further includes a bed-fall risk determination unit, The bed-fall risk determination unit is configured as follows: When the sixth signal is continuously received from the radar signal pattern analysis unit within the fifth time, the first signal is received from the bed-fall judgment unit, and the fourth signal is not continuously received within the fifth time, it is determined that the user is continuously in a state of falling out of bed and the biological signal is poor, and a seventh signal is generated to notify the user that the danger level caused by falling out of bed is very high, When the sixth signal is not received from the radar signal pattern analysis unit within the fifth time, and the fourth signal is not received from the bed-fall judgment unit within the fifth time, an eighth signal is generated indicating that the user is in a bed-fall state but the user's biological signal is stable. When the third signal is received from the bed-fall determination unit, a ninth signal is generated to notify that the possibility of the user falling out of bed is high.

11. The non-face-to-face and non-contact bed-fall monitoring system according to claim 10, characterized in that: The non-face-to-face and non-contact bed fall monitoring system also includes an alarm unit. The alarm unit is configured to: When the seventh signal is received from the falling-out-of-bed risk determination unit, a first alarm signal indicating that the user is in a very dangerous state is generated and sent to a preset guardian and medical staff. When the eighth signal is received from the bed-falling risk determination unit, a second alarm signal notifying the user of the bed-falling state is generated and sent to a preset guardian and medical staff, When the ninth signal is received from the fall-out-risk determination unit, a third alarm signal is generated to notify the user that the user is about to fall out of bed and is sent to a preset guardian and medical staff.

12. A non-contact sleep monitoring system, characterized in that: include: A Doppler signal acquisition unit, which uses radar to acquire Doppler signals including biological activity information; An auxiliary signal processing unit uses a thermal imaging sensor to obtain the turning action as an auxiliary signal; A Doppler signal analysis unit analyzes the Doppler signal and obtains spectrum energy at a preset period, determines whether the spectrum energy is obtained periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and The sleep interval definition unit defines the sleep state of each interval in the entire sleep interval using a ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the spectrum energy and a combination of the non-periodic spectrum energy.

13. The non-face-to-face and non-contact bed-fall monitoring system according to claim 12, characterized in that: The Doppler signal includes a breathing Doppler signal for acquiring breathing information of the user and a heartbeat Doppler signal for acquiring heartbeat information of the user.

14. The non-face-to-face and non-contact bed-fall monitoring system according to claim 13, characterized in that: The Doppler signal analysis unit performs fast Fourier transform on the Doppler signal to obtain the spectrum energy.

15. The non-face-to-face and non-contact bed-fall monitoring system according to claim 14, characterized in that: The sleep interval definition unit defines an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the entire sleep interval is greater than or equal to 5:5 and the non-periodic spectrum energy does not exist as a deep sleep interval, and defines an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy is less than 5:5 and the non-periodic spectrum energy does not exist as an apnea interval.

16. The non-face-to-face and non-contact bed-fall monitoring system according to claim 15, characterized in that: The sleep interval definition unit defines an interval in which the non-periodic spectrum energy exists and is greater than or equal to a preset value as a turning over interval, and defines an interval in which the non-periodic spectrum energy in the apnea interval is less than or equal to a preset value as a snoring interval.

17. A non-contact sleep monitoring system, characterized in that: include: A Doppler signal acquisition unit, which uses radar to acquire Doppler signals including biological activity information; An auxiliary signal processing unit uses a thermal imaging sensor to obtain the turning action as an auxiliary signal; A Doppler signal analysis unit analyzes the Doppler signal and obtains spectrum energy at a preset period, determines whether the spectrum energy is obtained periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and The sleep interval definition unit defines the sleep state of the remaining sleep intervals by using a preset ratio range and the non-periodic spectrum energy based on an average of the spectrum energies of the sleep entry intervals that meet a preset reference in the entire sleep interval.

18. A non-contact sleep monitoring method, characterized in that: include: A Doppler signal acquisition step, in which a Doppler signal acquisition unit acquires a Doppler signal including biological activity information using a radar; An auxiliary signal processing step, wherein the auxiliary signal processing unit uses a thermal imaging sensor to obtain a turning-over action as an auxiliary signal; a Doppler signal analysis step, wherein the Doppler signal analysis unit analyzes the Doppler signal and obtains spectrum energy at a preset period, determines whether the spectrum energy is obtained periodically, and when the spectrum energy is determined to be non-periodic spectrum energy, uses the auxiliary signal to distinguish the non-periodic spectrum energy; and In the sleep interval definition step, the sleep interval definition unit defines the sleep state of each interval in the entire sleep interval using a ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the spectrum energy and a combination of the non-periodic spectrum energy.

19. The non-contact sleep monitoring method according to claim 18, characterized in that: The Doppler signal includes a breathing Doppler signal for acquiring breathing information of the user and a heartbeat Doppler signal for acquiring heartbeat information of the user.

20. The non-contact sleep monitoring method according to claim 19, characterized in that: In the Doppler signal analysis step, a fast Fourier transform is performed on the Doppler signal to obtain the spectrum energy.

21. The non-contact sleep monitoring method according to claim 20, characterized in that: In the sleep interval definition step, an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy in the entire sleep interval is greater than or equal to 5:5 and the non-periodic spectrum energy does not exist is defined as a deep sleep interval, and an interval in which the ratio of the respiratory spectrum energy to the heartbeat spectrum energy is less than 5:5 and the non-periodic spectrum energy does not exist is defined as an apnea interval.

22. The non-contact sleep monitoring method according to claim 21, characterized in that: In the sleep interval definition step, the interval in which the non-periodic spectrum energy exists and the non-periodic spectrum energy is greater than or equal to a preset value is defined as a turning over interval, and the interval in the apnea interval in which the non-periodic spectrum energy is less than or equal to a preset value is defined as a snoring interval.

23. A non-contact sleep monitoring method, characterized in that: include: A Doppler signal acquisition step, in which a Doppler signal acquisition unit acquires a Doppler signal including biological activity information using a radar; An auxiliary signal processing step, wherein the auxiliary signal processing unit uses a thermal imaging sensor to obtain a turning-over action as an auxiliary signal; a Doppler signal analysis step, in which a Doppler signal analysis unit analyzes the Doppler signal and obtains spectrum energy at a preset period, determines whether the spectrum energy is obtained periodically, and uses the auxiliary signal to distinguish the spectrum energy; and In the sleep interval definition step, the sleep interval definition unit defines the sleep state of the remaining sleep intervals using a preset ratio range and non-periodic spectrum energy based on the average of the spectrum energy of the sleep entry intervals that meet the preset reference in the entire sleep interval.

Images