Non-face-to-face, non-contact fall monitoring system, non-contact sleep monitoring system and method

Abstract

The present invention is characterized by being embodied in a non-face-to-face, non-contact fall monitoring system and a non-contact sleep monitoring system and method that improves the reliability of fall detection by detecting a user's fall using radar signals and thermal image signals, and also analyzes and provides guidance on the user's sleep state.

Description

[Technical Field]

[0001] The present invention relates to a non-face-to-face, non-contact fall monitoring system that improves the reliability of fall detection by detecting a user's fall using radar signals and thermal image signals, and a non-contact sleep monitoring system and method that also analyzes and provides guidance on the user's sleep state. [Background technology]

[0002] In recent years, the population requiring care or attention has been increasing in Korean society. This can be defined as a population that requires assistance in the event of an emergency or has no one living with them, and includes, for example, the elderly, the disabled, and people living alone. According to a recent survey, the elderly population in Korea has exceeded 9 million, while the number of people living alone and those with disabilities has exceeded 6.6 million and 2.6 million, respectively. Furthermore, the number of people who have died alone, meaning they are discovered after their death without a cohabitant, has also exceeded 10,000.

[0003] As a result, the number of accidents involving falls is increasing. Falls are accidents that occur regardless of the individual's will, and they are dangerous accidents that can cause various complications and even lead to death.

[0004] Therefore, a fall detection technology that can accurately detect a fall accident and quickly respond to it is important.

[0005] The fall detection device using a camera detects the movements of the elderly through images and provides an alert to the guardian when a fall is detected, but there are problems such as human rights violations and privacy issues.

[0006] Furthermore, conventional fall detection devices that use radar signals have the problem of determining that a fall did not occur even when one actually did, or determining that a fall occurred due to radio wave interference caused by wind or the like even when no fall actually occurred.

[0007] In addition, conventional wearable fall detection devices have the disadvantage that users often forget to wear the wearable device or lose the wearable device itself, making it impossible to properly analyze the user's fall.

[0008] In addition, such furniture for single-person living can lead to irregular sleep quality if the individual leads an irregular lifestyle, which can cause health problems. In order to measure the quality of sleep, a person must make an appointment with a specialized hospital, visit the hospital at a suitable time, and go through the cumbersome process of using specialized equipment while sleeping, which makes it difficult to measure the quality of sleep regularly. Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention has been made to solve the above problems, and an object of the present invention is to provide a non-face-to-face, non-contact fall monitoring system that can protect the user's privacy and increase the reliability of fall detection.

[0010] Another object of the present invention is to provide a non-face-to-face, non-contact fall monitoring system that can monitor a user's fall without the user having to wear a sensing device.

[0011] Another object of the present invention is to provide a non-face-to-face, non-contact fall monitoring system that determines the degree of risk when a fall occurs and notifies the guardian or medical staff of the risk.

[0012] Another object of the present invention is to provide a non-contact sleep monitoring system and method that increases the reliability of analysis of a user's state during sleep by using a radar sensor unit to acquire biological activity information during the user's sleep state and using a thermal image sensor unit to acquire turning over movements as an auxiliary signal.

[0013] Another object of the present invention is to provide a non-contact sleep monitoring system and method that can protect the privacy of a user by not using thermal imaging data as it is. [Means for solving the problem]

[0014] The non-face-to-face, non-contact fall monitoring system of the present invention may include a radar sensor unit that transmits radar toward a user, receives radar reflected from the user, and generates a time-series Doppler radar signal using a Doppler radar algorithm; a thermal image sensor unit that images the user from above the user and generates thermal image data; a radar signal pattern analysis unit that receives the time-series Doppler radar signal from the radar sensor unit and analyzes the user's condition based on the transmitted time-series Doppler radar signal; a thermal image data processing unit that receives the thermal image data generated by the thermal image sensor unit and processes the transmitted thermal image data to analyze the user's condition; and a fall determination unit that determines whether or not a fall has occurred based on the user's condition data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit.

[0015] The radar signal pattern analysis unit may separate a motion change signal and a biological signal from the time-series Doppler radar signal transmitted from the radar sensor unit, and analyze the user's condition using the separated motion change signal as a time-series motion Doppler radar signal.

[0016] The radar signal pattern analysis unit may generate first status data indicating that a fall has occurred if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal within a first hour and then the amplitude becomes zero, and may generate second status data indicating that the user is slowly falling and that a fall is suspected if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal after the first hour.

[0017] The thermal image data processing unit can process the thermal image data by including the steps of: determining an analysis range from the transmitted thermal image data; determining a user edge within the determined analysis range; and processing the thermal image data to have a closed curve shape from the determined edge using a preset threshold to obtain a closed curve image.

[0018] The thermal image data processing unit can form a labeled image by further including a step of performing image labeling on the closed curve image.

[0019] The thermal image data processing unit may further include the step of spectrographically converting the labeled image into a spectrogram to form a spectrogram.

[0020] The thermal image data processing unit can determine the user's state as one of the fourth state to the sixth state based on at least one of the closed curve image, the labeled image, and the spectrogram, and form the fourth state data to the sixth state data, respectively.

[0021] The fourth state data may indicate a state in which the user is positioned on an object at a predetermined height from the floor, the fifth state data may indicate a state in which the user is falling from an object at a predetermined height from the floor, and the sixth state data may indicate a state in which the user has fallen from an object at a predetermined height from the floor.

[0022] the fall determination unit receives first status data from the radar signal pattern analysis unit, receives the fourth status data, the fifth status data, and the sixth status data sequentially from the thermal image data processing unit, determines that the user has fallen if the fourth status data is generated before the first status data generation time, generates a first signal indicating that the user has fallen, receives the first status data determining that the user has fallen from the radar signal pattern analysis unit, receives sixth status data, the fifth status data, and the fourth status data sequentially from the thermal image data processing unit, determines that the user has not fallen if the sixth status data is generated before the first status data generation time, generates a second signal indicating that a fall has not occurred, receives the second status data indicating that a fall is suspected from the radar signal pattern analysis unit, receives the fourth status data, the fifth status data, and the sixth status data sequentially from the thermal image data processing unit, and generates a second signal indicating that a fall has not occurred at the time of generation of the first status data. and a fourth signal indicating that the user is at a high risk of falling. In the case where the fourth status data is generated before the fourth hour, the radar signal pattern analysis unit determines that the user has fallen and generates a third signal indicating that the user has fallen, and generates a fourth signal indicating that the user has fallen. In the case where the third status data is transmitted from the radar signal pattern analysis unit and only the fourth status data is transmitted from the thermal image data processing unit until the fourth hour or later from the time when the third status data was transmitted, the radar signal pattern analysis unit determines that the user has fallen and generates a fifth signal indicating that the user has returned to the original position. In the case where the first status data or the second status data is transmitted from the radar signal pattern analysis unit and the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit after the fourth hour, the radar signal pattern analysis unit determines that the user has fallen and generates a fifth signal indicating that the user has returned to the original position.

[0023] The radar signal pattern analysis unit can analyze the separated biological signal and generate the sixth signal notifying of a dangerous situation when the biological signal differs from a biological signal pattern for a predetermined period by a predetermined range or more.

[0024] The non-contact fall and sleep monitoring system may further include a fall risk determination unit.

[0025] If the fall risk determination unit receives the sixth signal continuously for five hours from the radar signal pattern analysis unit and the first signal from the fall determination unit, and does not receive the fourth signal at all within the five hours, the fall risk determination unit determines that the user's fall state continues and that a biological signal is poor, and generates a seventh signal notifying that the user is at high risk due to a fall; if the sixth signal is not received from the radar signal pattern analysis unit for the five hours and the fourth signal is not received from the fall determination unit for the five hours, the fall risk determination unit generates an eighth signal indicating that the user's fall state continues but that the biological signal of the user is stable; and if the fall risk determination unit receives the third signal from the fall determination unit, the fall risk determination unit generates a ninth signal notifying that the user is likely to fall.

[0026] The non-face-to-face, non-contact fall monitoring system further includes an alarm unit, which, when the seventh signal is transmitted from the fall risk determination unit, generates a first alarm signal notifying that the user is in a very critical state and transmits the first alarm signal to a predetermined guardian and medical staff; when the eighth signal is transmitted from the fall risk determination unit, generates a second alarm signal notifying that the user has fallen and transmits the second alarm signal to a predetermined guardian or medical staff; and when the ninth signal is transmitted from the fall risk determination unit, generates a third alarm signal notifying that the user is on the verge of falling and transmits the third alarm signal to a predetermined guardian or medical staff.

[0027] The non-contact sleep monitoring system and method of the present invention may include a Doppler signal acquisition unit that acquires a Doppler signal containing biological activity information using a radar; an auxiliary signal processing unit that acquires turning over movements as an auxiliary signal using a thermal image sensor; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a predetermined period, determines whether the spectral energy is periodically acquired, and, if the spectral energy is determined to be aperiodic spectral energy, classifies the aperiodic spectral energy using the auxiliary signal; and a sleep segment definition unit that defines a sleep state for each segment in an entire sleep segment using a ratio of respiratory spectral energy and heart rate spectral energy among the spectral energy and a combination of the aperiodic spectral energy.

[0028] The Doppler signals may include a respiratory Doppler signal that acquires information about the user's breathing, and a heart rate Doppler signal that acquires information about the user's heartbeat.

[0029] The Doppler signal analysis unit can obtain the spectral energy by performing a fast Fourier transform on the Doppler signal.

[0030] The sleep section definition unit can define a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy is 5:5 or more in the entire sleep section as a deep sleep section, and can define a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy is less than 5:5 as an apnea section.

[0031] The sleep section definition unit may define a section in which the non-periodic spectral energy exists and appears at a magnitude equal to or greater than a predetermined magnitude as a tossing section, and may define a section in the apnea section in which the non-periodic spectral energy appears at a magnitude equal to or less than a predetermined magnitude as a snoring section.

[0032] According to another embodiment of the present invention, a contactless sleep monitoring system and method may include a Doppler signal acquisition unit that acquires a Doppler signal including biological activity information using a radar; an auxiliary signal processing unit that acquires a turning over movement as an auxiliary signal using a thermal image sensor; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a predetermined period, determines whether the spectral energy is periodically acquired, and classifies the spectral energy using the auxiliary signal; and a sleep segment definition unit that defines sleep states for the remaining sleep segments using a predetermined percentage range and non-periodic spectral energy based on an average of the spectral energy of a sleep onset segment that satisfies a predetermined criterion among all sleep segments.

[0033] The non-contact sleep monitoring system and method of the present invention may include the steps of: acquiring a Doppler signal including biological activity information in a Doppler signal acquisition unit using a radar; processing an auxiliary signal acquired using a thermal image sensor in an auxiliary signal processing unit, which acquires a turning over movement as an auxiliary signal; analyzing the Doppler signal in a Doppler signal analysis unit to acquire spectral energy at a predetermined period, determining whether the spectral energy is periodically acquired, and analyzing the Doppler signal by classifying the spectral energy using the auxiliary signal; and defining sleep periods in a sleep period definition unit, which define a sleep state for each period in an entire sleep period using a ratio of respiratory spectral energy and heart rate spectral energy and a combination of non-periodic spectral energy among the spectral energy.

[0034] The Doppler signals may include a respiratory Doppler signal that acquires information about the user's breathing, and a cardiac Doppler signal that acquires information about the user's heartbeat.

[0035] The step of analyzing the Doppler signal may involve performing a fast Fourier transform on the Doppler signal to obtain the spectral energy.

[0036] The step of defining the sleep intervals may define an interval in which the non-periodic spectral energy does not exist among intervals in which the ratio of the respiratory spectral energy to the heart rate spectral energy is 5:5 or more in the entire sleep interval as a deep sleep interval, and may define an interval in which the non-periodic spectral energy does not exist among intervals in which the ratio of the respiratory spectral energy to the heart rate spectral energy is less than 5:5 as an apnea interval.

[0037] The step of defining the sleep section may define a section in which the non-periodic spectral energy exists and appears at a level equal to or greater than a preset level as a tossing section, and may define a section in the apnea section in which the non-periodic spectral energy appears at a level equal to or less than a preset level as a snoring section.

[0038] The non-contact sleep monitoring system and method of the present invention may include the steps of: acquiring a Doppler signal including biological activity information using a radar in a Doppler signal acquisition unit; processing an auxiliary signal, which acquires turning over as an auxiliary signal using a thermal image sensor in an auxiliary signal processing unit; analyzing the Doppler signal in a Doppler signal analysis unit to acquire spectral energy at a predetermined period, determining whether the spectral energy is periodically acquired, and classifying the spectral energy using the auxiliary signal; and defining a sleep state for the remaining sleep periods using a predetermined percentage range and non-periodic spectral energy based on an average of the spectral energy of a sleep onset period that satisfies a predetermined criterion among all sleep periods in a sleep period definition unit. [Effects of the Invention]

[0039] The present invention has the advantage of protecting the user's privacy while increasing the reliability of fall detection.

[0040] In addition, the present invention has an advantage in that it is a non-face-to-face, non-contact type that is not a wearable type, and can detect a user's fall in real time without the user having to wear a separate device.

[0041] Furthermore, the present invention has the advantage that if a fall is detected, the degree of danger is determined and notified to guardians or medical staff, thereby making it possible to prevent or deal with the risk of a fall in advance.

[0042] Furthermore, the present invention has the advantage that, when monitoring a fall, the thermal image is processed to create a closed curve, a labeling image, and a spectrogram, thereby protecting the user's privacy.

[0043] Furthermore, the present invention has the advantage that it is possible to analyze the user's risk level based on the user's biosignal and the processed thermal image, and take appropriate measures.

[0044] Furthermore, the present invention has the advantage that by using radar to acquire biological activity information of the user while sleeping, it does not affect the user's body and does not disturb the user's sleep.

[0045] Furthermore, the present invention has an advantage in that it can accurately determine the sleep state of the user for each sleep period using the user's respiration and heart rate information.

[0046] Furthermore, the present invention has the advantage of improving reliability compared to conventional Doppler signal-based sleep monitoring by using the contour of a thermal image together with a Doppler signal for sleep monitoring. [Brief explanation of the drawings]

[0047] [Figure 1] FIG. 1 is a diagram illustrating a non-face-to-face, non-contact fall monitoring system according to one embodiment of the present invention. [Figure 2] FIG. 2 is a diagram illustrating a movement pattern analyzed by a radar signal pattern analyzer according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing the process of analyzing the thermal imaging data and determining the fourth state. [Figure 4] FIG. 4 is a diagram showing the process of analyzing the thermal imaging data and determining the fifth state. [Figure 5] FIG. 5 is a diagram showing the process of analyzing the thermal imaging data and determining the sixth state. [Figure 6] FIG. 6 is a block diagram illustrating a non-contact sleep monitoring system according to one embodiment of the present invention. [Figure 7] FIG. 7 is a flowchart illustrating a method for non-contact sleep monitoring according to one embodiment of the present invention. [Figure 8] FIG. 8 is a graph showing the results of an actual conspiracy experiment using one embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing a process of processing thermal imaging data in the auxiliary signal processing unit of the present invention. BEST MODE FOR CARRYING OUT THE INVENTION

[0048] The present invention is characterized by including a radar sensor unit that transmits radar toward a user, receives radar reflected from the user, and generates a time-series Doppler radar signal using a Doppler radar algorithm; a thermal image sensor unit that images the user from above the user and generates thermal image data; a radar signal pattern analysis unit that receives the time-series Doppler radar signal from the radar sensor unit and analyzes the user's condition based on the transmitted time-series Doppler radar signal; a thermal image data processing unit that receives the thermal image data generated by the thermal image sensor unit and processes the transmitted thermal image data to analyze the user's condition; and a fall determination unit that determines whether or not a fall has occurred based on the user's condition data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit. DETAILED DESCRIPTION OF THE INVENTION

[0049] A non-face-to-face, non-contact fall monitoring system according to one embodiment of the present invention may include a radar sensor unit 100, a thermal image sensor unit 200, a radar signal pattern analysis unit 300, a thermal image data processing unit 400, a fall determination unit 500, a fall risk determination unit 600, and an alarm unit 700.

[0050] A non-face-to-face, non-contact fall monitoring system according to an embodiment of the present invention can be installed not only in the space where a user sleeps but also in the space where the user lives.

[0051] 1. Radar sensor part (100) The radar sensor unit 100 transmits radar toward a user, receives radar reflected from the user, and uses a Doppler radar algorithm to generate a time-series Doppler radar signal as shown in Fig. 2. Here, in the time-series Doppler radar signal, the X-axis represents time and the Y-axis represents the magnitude of the Doppler radar signal. In the time-series Doppler radar signal, positive and negative data on the Y-axis represent opposite directions, and the magnitude of the amplitude, which is the absolute value of the Y-axis, represents the speed of the user's movement or the movement of a large area.

[0052] 2. Thermal image sensor unit (200) The thermal image sensor unit 200 may be an infrared thermal imaging camera. The infrared thermal imaging camera may capture an image of the user from above the user. The thermal image data of the user is transmitted to the determination unit. Preferably, the infrared thermal imaging camera may be installed on the ceiling above the bed in the bedroom.

[0053] 3. Radar signal pattern analysis unit (300) The radar signal pattern analysis unit 300 can separate a motion change signal and a biological signal from the time series Doppler radar signal transmitted from the radar sensor unit 100. The time series Doppler radar signal separated from the motion change signal is hereinafter referred to as a time series motion Doppler radar signal.

[0054] When analyzing the time-series movement Doppler radar signal, the positive and negative phases of the Y-axis indicate opposite directions. Therefore, when the Y-axis phase changes from a positive phase to a negative phase, it means that the movement has changed to a second movement that is opposite to the direction of the first movement, such as from front to back, from back to front, from up to down, or from down to up. Referring to Figure 2, the Doppler time-series data corresponding to a fall indicates that the movement of the user's entire body or part of their body has changed in the opposite direction over a total of four phase changes: positive → negative → positive → negative → positive.

[0055] The amplitude of the time-series motion Doppler radar signal (absolute value on the Y axis) indicates either the speed or the magnitude of the movement, or both. In other words, a large amplitude indicates fast movement or a large area of ​​movement, such as arm or leg movement, which is not a biological signal.

[0056] Therefore, when a body part with a large surface area, such as an arm or a leg, is rapidly turned, the phase of the Y-axis with a large amplitude can change from positive to negative or from negative to positive. In the present invention, a change in the Y-axis phase from positive to negative or from negative to positive is referred to as a positive / negative change.

[0057] The radar signal pattern analysis unit 300 generates first status data indicating that a fall has occurred when a positive or negative change occurs in data having a predetermined or greater amplitude in the time-series movement Doppler radar signal within the first hour, and then the amplitude becomes 0.

[0058] In one embodiment, the radar signal pattern analysis unit 300 generates first status data indicating that a fall has occurred when the amplitude of the time-series movement Doppler radar signal remains constant at 0 after the amplitude is greater than or equal to a predetermined range and a positive / negative change occurs within 200 ms.

[0059] In addition, if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal after the first time, the radar signal pattern analysis unit 300 may generate second status data indicating that the user is slowly falling and is suspected of having fallen.

[0060] In addition, based on the analysis of the time-series motion Doppler radar signal, if the magnitude of the amplitude of the time-series motion Doppler radar signal is equal to or greater than a predetermined range and the amplitude of the second time-series motion Doppler radar signal occurs at or after a second time after the generation of the first time-series motion Doppler radar signal, the radar signal pattern analysis unit 300 can generate third status data, which means that the user is not falling but is continuing to move.

[0061] When the second status data is generated during the analysis of the time-series movement Doppler radar signal, it is difficult to accurately determine whether the user has fallen.

[0062] Furthermore, in the case where a user jumps from under the bed onto the bed, which is the opposite of a fall, the time-series motion Doppler radar signal analysis may erroneously determine that the user has fallen because the direction of the hands and feet changes quickly, and the amplitude of the motion Doppler radar signal is large, resulting in a positive / negative change occurring within a predetermined time period.

[0063] Furthermore, the time-series motion Doppler radar signal analysis cannot determine whether a fall has occurred when a person is lying on the edge of the bed and there is a high possibility of a fall occurring, even though the person has not actually fallen. It only determines whether a fall has occurred after the fall has actually occurred, which has the disadvantage that it cannot prevent falls.

[0064] The present invention utilizes thermal imaging data analysis together with time series motion Doppler radar signal analysis to complement this.

[0065] 4. Thermal image data processing unit (400) The thermal image data processing unit 400 processes the thermal image data transmitted from the thermal image sensor unit 200 to analyze the state of the user.

[0066] The thermal image data processing unit 400 determines an analysis range from the received thermal image data. Then, the thermal image data processing unit 400 determines a user edge within the determined analysis range. Then, the thermal image data processing unit 400 processes the thermal image data to have a closed curve shape from the determined edge using a preset threshold, thereby obtaining a closed curve image.

[0067] Then, the thermal image data processing unit 400 may further perform image labeling on the closed curve image to form a labeled image, and then the thermal image data processing unit 400 may spectrogramize the labeled image to form a spectrogram.

[0068] 3 is a diagram illustrating a process of analyzing the thermal image data transmitted from the thermal image sensor unit 200 and determining the fourth state. Here, the fourth state refers to a state in which the user is positioned on an object at a predetermined height from the floor (e.g., lying on a bed). FIG. 3a is the thermal image data transmitted from the thermal image sensor unit 200, FIG. 3b is the thermal image data with the analysis range determined, FIG. 3c is the edge of the user, FIG. 3d is the closed curve image of the user, FIG. 3e is the labeled image of the user, and FIG. 3f is the spectrogram of the labeled image of the user.

[0069] 4 is a diagram illustrating a process of analyzing the thermal image data transmitted from the thermal image sensor unit 200 and determining a fifth state. Here, the fifth state refers to a state in which the user has fallen from an object (e.g., a bed) located at a predetermined height from the floor. FIG. 4a shows the thermal image data transmitted from the thermal image sensor unit 200, FIG. 4b shows the thermal image data with the analysis range determined, FIG. 4c shows the edge of the user, FIG. 4d shows the closed curve image of the user, FIG. 4e shows the labeled image of the user, and FIG. 4f shows the spectrogram of the labeled image of the user.

[0070] 5 is a diagram illustrating a process of analyzing the thermal image data transmitted from the thermal image data processing unit 400 and determining the sixth state. Here, the sixth state refers to a state in which the user has fallen from an object (e.g., a bed) located at a predetermined height from the floor. FIG. 5a shows the thermal image data transmitted from the thermal image sensor unit 200, FIG. 5b shows the thermal image data with the analysis range determined, FIG. 5c shows the edge of the user, FIG. 5d shows the closed curve image of the user, FIG. 5e shows the labeled image of the user, and FIG. 5f shows the spectrogram of the labeled image of the user.

[0071] The thermal image data processing unit 400 judges the state into a fourth state in which the subject is lying on an object of a predetermined height (e.g., a bed), a fifth state in which the subject has fallen from an object of a predetermined height (e.g., a bed), and a sixth state in which the subject has fallen from an object of a predetermined height (e.g., a bed) based on at least one of the closed curve images 3d, 4d, and 5d, the labeled images 3e, 4e, and 5e, and the spectrograms 3f, 4f, and 5f, and generates fourth state data indicating the fourth state, fifth state data indicating the fifth state, and sixth state data indicating the sixth state. The fourth state data to sixth state data thus generated can be transmitted to the fall determination unit 500 together with the time at which the fourth state data to sixth state data were generated.

[0072] Preferably, the thermal image data processing unit 400 determines the fourth to sixth states through a morphological analysis of the spectrogram, generates fourth state data indicating the fourth state, fifth state data indicating the fifth state, and sixth state data indicating the sixth state, and transmits the fourth to sixth state data thus generated to the fall determination unit 500 together with the generation times of the fourth to sixth state data.

[0073] 5. Fall detection unit (500) The fall determination unit 500 can match the first to third status data transmitted from the radar signal pattern analysis unit 300 with the fourth to sixth status data transmitted from the thermal imaging data processing unit 400 at the same time, and then match at least one of the first to third status data with the fourth to sixth status data to determine whether or not a fall has occurred and analyze the risk of a fall.

[0074] (1) Determined as a fall (generates the first signal) The fall determination unit 500 receives first status data determined to be a first status from the radar signal pattern analysis unit 300, and sequentially receives fourth status data, fifth status data, and sixth status data from the thermal image data processing unit 400. If the fourth status data is generated before the first status data generation time, the fall determination unit 500 determines that the user has fallen, generates a first signal indicating that the user has fallen, and transmits the first signal to the fall risk determination unit 600.

[0075] (2) Determined that it was not a fall (generates second signal) Conversely, when the first status data determined to be a first status is transmitted from the radar signal pattern analysis unit 300, and the sixth status data, fifth status data, and fourth status data are sequentially transmitted from the thermal image data processing unit 400, and the sixth status data is generated before the time when the first status data is generated, the fall risk determination unit 600 can determine that the user jumped onto an object at a predetermined height that was not a fall, and generate a second signal indicating that a fall did not occur and transmit it to the fall risk determination unit 600.

[0076] (3) It is determined that the fall occurred at a slow speed (generating the third signal). The fall determination unit 500 receives second status data indicating that a fall is suspected from the radar signal pattern analysis unit 300, and sequentially receives fourth, fifth, and sixth status data from the thermal image data processing unit 400. If the fourth status data is generated before the first status data is generated, the fall determination unit 500 determines that the user has fallen, and generates a third signal indicating that the user has fallen as the user slowly falls over, and transmits the third signal to the fall risk determination unit 600.

[0077] (4) The risk of falling is judged to be high (generates the fourth signal). The fall determination unit 500 receives third status data determined to be a third status from the radar signal pattern analysis unit 300, and when only fourth status data is received from the thermal image data processing unit 400 from the time the third status data is received until the third hour or later, the fall determination unit 500 determines that there is a high risk of the user falling, and generates a fourth signal indicating a high risk of a fall and transmits it to the fall risk determination unit 600.

[0078] (5) Determine that the device has returned to its original position (generates the fifth signal) When the fall determination unit 500 receives the first status data determined to be a first status from the radar signal pattern analysis unit 300, receives the fourth status data, the fifth status data, and the sixth status data sequentially from the thermal image data processing unit 400, and receives the first status data determined to be a first status from the radar signal pattern analysis unit 300 again after a fourth time and receives the sixth status data, the fifth status data, and the fourth status data sequentially from the thermal image data processing unit 400, the fall determination unit 500 determines that the user who fell has returned to their original position, and generates a fifth signal indicating that the user has returned to their original position and transmits it to the fall risk determination unit 600.

[0079] (5) Poor biological signals (generates the sixth signal) The radar signal pattern analysis unit 300 can analyze the separated biological signal, and when the biological signal differs by a predetermined range or more from a biological signal pattern for a predetermined period, generate a sixth signal notifying a dangerous situation and transmit the sixth signal to the fall risk determination unit 600. For example, when the heart rate or respiratory rate, which is a biological signal, decreases or increases by 50% or more from the average heart rate or average respiratory rate for one month or one week, the radar signal pattern analysis unit 300 can generate a sixth signal notifying that the user's biological signal is bad and the user's condition is dangerous.

[0080] 6. Fall risk determination unit (600) The fall risk determination unit 600 receives the sixth signal from the radar signal pattern analysis unit 300 and the first to fifth signals from the fall determination unit 500, analyzes the risk of the user falling, and generates a signal indicating each risk.

[0081] (1) Extreme risk of falling (generates signal 7) If the fall risk determination unit 600 receives the sixth signal continuously from the radar signal pattern analysis unit 300 for a fifth time period, receives the first signal from the fall determination unit 500, and does not receive the fourth signal within the fifth time period, the fall risk determination unit 600 determines that the user is in a persistent fall state (i.e., has fallen and is unable to get up into bed) and that the biological signals have been poor for a certain period of time, and generates a seventh signal notifying that the user is in a very high risk of falling and transmits the seventh signal to the alarm unit 700.

[0082] (2) There is a high risk of falling (signal 7 is generated) If the fall risk determination unit 600 receives the sixth signal continuously from the radar signal pattern analysis unit 300 for a fifth time period, receives the third signal from the fall determination unit 500, and does not receive the fourth signal within the third time period, the fall risk determination unit 600 determines that the user continues to fall (i.e., has fallen and is unable to get up into bed) and that the biological signals have been poor for a certain period of time, and generates a seventh signal notifying that the user is at a very high risk of falling and transmits the seventh signal to the alarm unit 700.

[0083] 3) The fall continues, but the vital signs are stable (generating the 8th signal) If the sixth signal is not received from the radar signal pattern analysis unit 300 during the fifth time period, and if the third signal is received from the fall determination unit 500 but the fourth signal is not received within the third time period, the fall risk determination unit 600 can generate an eighth signal to notify the alarm unit 700 that the user continues to fall (i.e., has fallen and is unable to get up into bed), but the biological signal is not bad and the user is not at high risk of falling.

[0084] (4) There is a high possibility of a fall (No. 9 signal is generated). When the fall risk determination unit 600 receives the fourth signal from the fall determination unit 500, it can generate a ninth signal notifying that the user is likely to fall and transmit it to the alarm unit 700.

[0085] (4) No fall occurred (no signal generated) When the fall risk determination unit 600 receives the second signal from the fall determination unit 500, it analyzes that the user has a low risk of falling and does not generate a separate signal.

[0086] (5) The fall state has ended and the biological signals are stable (signals are not yet generated). If the fall risk determination unit 600 does not receive the sixth signal from the radar signal pattern analysis unit 300 within the five hours and receives the fourth signal within the five hours, it determines that the user's biological signals are stable and the user's fall has ended (i.e., the user has climbed into bed on their own), so there is no danger to the user's safety and does not generate a separate signal.

[0087] 7. Alarm section (700) When the seventh signal is transmitted from the fall risk determination unit 600, the alarm unit 700 generates a first alarm signal notifying that the user has fallen and that the vital signs are poor and in a very critical state, and can transmit the first alarm signal to a predetermined guardian and medical staff.

[0088] When the eighth signal is transmitted from the fall risk determination unit 600, the alarm unit 700 can generate a second alarm signal notifying that the user is in a fall state but that the biological signals are stable, and transmit the second alarm signal to a predetermined guardian or medical staff.

[0089] The alarm unit 700 may generate a third alarm signal notifying that the user is on the verge of falling and transmit the third alarm signal to a predetermined guardian or medical staff when the alarm unit 700 receives the ninth signal from the fall risk determination unit 600. When the third alarm signal is transmitted, the guardian or medical staff may take appropriate measures to prevent the user from falling.

[0090] FIG. 6 is a block diagram showing a contactless sleep monitoring system according to an embodiment of the present invention. The contactless sleep monitoring system 1 according to an embodiment of the present invention is installed in a space where a user lives and sleeps, such as a room, and can be configured to detect sleep when the user falls asleep, acquire Doppler signals related to biological activity, and analyze the acquired Doppler signals to define the user's sleep state. To this end, the present invention can utilize a radar-based biological activity measuring device already installed in the space. As shown in FIG. 6, the contactless sleep monitoring system 1 according to an embodiment of the present invention can be configured to include a Doppler signal acquisition unit 11, an auxiliary signal processing unit 13, a Doppler signal analysis unit 15, and a sleep segment definition unit 17.

[0091] The Doppler signal acquisition unit 11 is configured to acquire Doppler signals containing biological activity information from a radar installed in the user's sleeping space. The Doppler signals are configured to include a respiratory Doppler signal that is information on the user's breathing and a heartbeat Doppler signal that is information on the user's heartbeat. A Doppler signal can be defined as a signal that includes a Doppler frequency generated by the user's movement when radio waves are transmitted from the radar to the user and the transmitted radio waves return. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal may be Doppler signals corresponding to changes in the movement of bodily organs that move during breathing and heartbeat, respectively.

[0092] The auxiliary signal processor 13 is configured to acquire an auxiliary signal using an auxiliary biological activity information acquisition device additionally provided in the user's sleep space, and analyze the acquired auxiliary signal to acquire information on the user's body movements other than breathing or heartbeat movements (turning over in sleep, etc.). To this end, the auxiliary signal processor 13 can acquire measurement results from a thermal image sensor, which is an auxiliary biological activity information acquisition device. The thermal image sensor can be provided to measure the user's body movements.

[0093] The auxiliary signal processing unit 13 can continuously acquire contour information of the user's body using a thermal image sensor. When continuously acquiring contour information of the user's body using a thermal image sensor, it can acquire changes in the contour information that occur due to movements such as the user turning over in bed.

[0094] Preferably, the auxiliary signal processor 13 can continuously acquire information on the user's movements by analyzing the thermal imaging data acquired by the thermal imaging sensor. In the present invention, for the sake of user privacy, the information on the user's movements can be acquired through contour information or spectrograms, rather than using the thermal imaging data as is.

[0095] The auxiliary signal processor 13 processes the thermal image data transmitted from the thermal image sensor to analyze the user's movements.

[0096] The auxiliary signal processor 13 determines an analysis range from the transmitted thermal imaging data. Then, the auxiliary signal processor 13 determines a user edge within the determined analysis range. Then, the auxiliary signal processor 13 processes the thermal imaging data to have a closed curve shape from the determined edge using a preset threshold, thereby obtaining a closed curve image.

[0097] Thereafter, the auxiliary signal processing unit 13 may further perform image labeling on the closed curve image to form a labeled image, and then may form a spectrogram by converting the labeled image into a spectrogram.

[0098] 9 is a diagram showing the process of acquiring information on user movement by the auxiliary signal processor 13. 4a is thermal image data acquired by the thermal image sensor, 4b is thermal image data with the analysis range determined, 4c is the edge of the user, 4d is a closed curve image of the user, 4e is a labeled image of the user, and 4f is a spectrogram of the labeled image of the user.

[0099] The body contour information acquired by the auxiliary signal processing unit 13 can be used in the Doppler signal analysis unit 15 described later. For example, when the Doppler signal analysis unit 15 acquires non-periodic spectral energy using acquisition time information for the turning over motion represented by non-periodic spectral energy, the body contour information can be used as auxiliary information for determining what biological activity information the energy represents.

[0100] The Doppler signal analysis unit 15 is configured to acquire spectral energy at a predetermined period by analyzing the Doppler signal acquired by the Doppler signal acquisition unit 11. The Doppler signal analysis unit 15 applies a predetermined function to the Doppler signal to acquire spectral energy, and analyzes the acquired spectral energy according to a predetermined criterion to acquire respiratory spectral energy and heartbeat spectral energy, which are spectral energies for the respiratory Doppler signal and the heartbeat Doppler signal.

[0101] In one embodiment of the present invention, the Doppler signal analyzer 15 may use a fast Fourier transform (FFT) as a preset function used to acquire spectral energy from the Doppler signal at a preset period. The Fourier transform is a well-known function that decomposes a signal sampled in time or space into time frequency or spatial frequency components. By using this function, the Doppler signal analyzer 15 can convert the Doppler signal acquired at a preset period into spectral energy of a specific frequency component.

[0102] When the spectral energy of a specific frequency component is acquired, the Doppler signal analyzer 15 according to an embodiment of the present invention is configured to check whether the acquired spectral energy of the specific frequency component is spectral energy that occurs periodically. The Doppler signal analyzer 15 may be configured to continuously acquire spectral energy for a Doppler signal acquired at a preset period and determine whether the acquired continuous spectral energy has periodicity. Here, if the spectral energy has periodicity, it may be determined that the spectral energy is spectral energy corresponding to respiration or heart rate. If the spectral energy does not have periodicity, it may be determined that the spectral energy is not spectral energy corresponding to respiration or heart rate.

[0103] The Doppler signal analysis unit 15 checks whether the aperiodic spectral energy has energy greater than a preset value, and if the energy is greater than the preset value, determines that the spectral energy is the spectral energy corresponding to the Doppler frequency generated by the user's body movement.Also, if the energy is equal to or less than the preset value, determines that the spectral energy is the spectral energy corresponding to the Doppler frequency generated by the user's snoring.

[0104] In addition, the Doppler signal analysis unit 15 according to an embodiment of the present invention may analyze the periodic spectral energy using a pre-set judgment algorithm, and as a result, may separately acquire the respiration spectral energy and the heart rate spectral energy.

[0105] Furthermore, the Doppler signal analysis unit 15 according to an embodiment of the present invention can identify the type of aperiodic spectral energy using the auxiliary signal acquired by the aforementioned auxiliary signal processing unit 13. Here, the Doppler signal analysis unit 15 can be configured to acquire aperiodic spectral energy corresponding to the user's turning over (body movement) using thermal image information, for example, and determine the acquired aperiodic spectral energy as a noise signal to remove it from data for analyzing a sleep period, which will be described later.

[0106] In addition, as another embodiment, if a pre-set magnitude of energy that can distinguish between body movement and snoring is not defined using non-periodic spectral energy, the Doppler signal unit 15 can define a pre-set magnitude of energy that can distinguish between both movements using an auxiliary signal acquired using a thermal image sensor.

[0107] According to an embodiment of the present invention, the Doppler signal analysis unit 15 continuously acquires the spectral energy of a specific frequency component and, once it has determined whether or not it is a periodic occurrence, the sleep segment definition unit 17 is configured to define a sleep state for each segment in the entire sleep segment using the analysis results.

[0108] The total sleep period may include at least one of a sleep onset period, a deep sleep period, a tossing and turning period, an apnea period, and a snoring period. The sleep onset period refers to a period in which a user enters sleep and is a period that occurs statistically in almost all users, allowing determination of whether or not a user has entered sleep. The deep sleep period refers to a sleep period in which the user's physical activity is stably maintained, and the tossing and turning period refers to a sleep period in which the user's physical activity involves movement. The apnea period refers to a sleep period in which no respiratory activity occurs during the user's physical activity, and the snoring period refers to a sleep period in which the user's physical activity involves snoring.

[0109] The higher the ratio of deep sleep segments to the lower the ratio of remaining segments in a sleep segment, the higher the quality of sleep. Therefore, by defining each sleep segment according to the present invention, it is possible to obtain information on the sleep quality of the user and later provide a prescription for improving the sleep quality for each user.

[0110] The sleep segment definition unit 17 is configured to define a sleep state for each segment in the entire sleep segment using the analysis results of the Doppler signal analysis unit 15. As described above, each segment in the entire sleep segment includes at least one of a sleep onset segment, a deep sleep segment, a tossing and turning segment, an apnea segment, and a snoring segment. The sleep segment definition unit 17 obtains the analysis results from the Doppler signal analysis unit 15 and determines which sleep segment the analysis results represent using pre-set state determination criteria.

[0111] The preset state determination criterion may be a preset value using the ratio of the respiration spectrum energy and the heart rate spectrum energy acquired by the Doppler signal analysis unit 15. As described above, since the magnitude of physical activity caused by respiration is generally greater than the magnitude of physical activity caused by heartbeat, the magnitude of the respiration spectrum energy appears greater than the magnitude of the heart rate spectrum energy in a stable sleep period. Also, since high-frequency vibrations are captured in a snoring period, additional spectra other than the respiration spectrum energy and the heart rate spectrum energy may appear. Also, since no respiration occurs or only a weak respiration occurs in an apnea period, the magnitude of the respiration spectrum energy may decrease by the magnitude of the heart rate spectrum energy.

[0112] The sleep segment definition unit 17 of one embodiment of the present invention can define a segment in which the spectral ratio, which is the ratio of respiratory spectral energy to heart rate spectral energy, is 5:5 or more as a deep sleep segment.

[0113] Furthermore, the sleep section definition unit 17 can define a section in which the spectral ratio is less than 5:5 and in which the non-periodic spectral energy is acquired at a level equal to or less than a preset value as a snoring section, and can define the section in which the non-periodic spectral energy is not acquired for a preset time as an apnea section.

[0114] In another embodiment of the present invention, the sleep segment definition unit 17 may define a sleep segment using a sleep onset segment in addition to the spectrum ratio. Since tossing and turning, apnea, and snoring generally do not occur in the sleep onset segment, similar to the deep sleep segment, the sleep segment definition unit 17 of the present invention may be configured to first obtain the sleep onset segment and determine the deep sleep segment based on the obtained sleep onset segment.

[0115] Meanwhile, a flowchart of a non-contact sleep monitoring method according to an embodiment of the present invention is shown in Fig. 7. For convenience of explanation, the non-contact sleep monitoring method of the present invention will be described below as being performed using the system of Fig. 6, but the present invention is not necessarily limited thereto.

[0116] A non-contact sleep monitoring method 10 according to an embodiment of the present invention may be installed in a space where a user lives and sleeps, such as a room, and may be configured to detect sleep when the user falls asleep, acquire a Doppler signal for biological activity, and analyze the acquired Doppler signal to define the user's sleep state. To this end, the present invention may utilize a biological activity measuring device using a radar that is already installed in the space. As shown in FIG. 7, the non-contact sleep monitoring method 10 according to an embodiment of the present invention may include step S11 of acquiring a Doppler signal, step S13 of processing an auxiliary signal, step S15 of analyzing the Doppler signal, and step S17 of defining a sleep period.

[0117] The step S11 of acquiring a Doppler signal is performed by the Doppler signal acquisition unit 11 acquiring a Doppler signal including biological activity information from a radar installed in the user's sleep space. The Doppler signal is formed to include a respiratory Doppler signal that is information about the user's breathing and a heartbeat Doppler signal that is information about the user's heartbeat. The Doppler signal may be defined as a signal including a Doppler frequency generated by the user's movement when radio waves are transmitted from the radar to the user and the transmitted radio waves return. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal may be Doppler signals corresponding to changes in the movement of bodily organs that move during breathing and heartbeat, respectively.

[0118] The step S13 of processing the auxiliary signal is configured to acquire the auxiliary signal using an auxiliary biological activity information acquisition device additionally provided in the user's sleep space, and analyze the acquired auxiliary signal to acquire information on the user's body movements other than breathing or heartbeat movements (turning over in sleep, etc.). To this end, the step S13 of processing the auxiliary signal may acquire measurement results from a thermal image sensor, which is the auxiliary biological activity information acquisition device. The thermal image sensor may be provided to measure the user's body movements.

[0119] In step S13 of processing the auxiliary signal, contour information of the user's body can be continuously acquired using a thermal image sensor. When contour information of the user's body is continuously acquired using a thermal image sensor, changes in the contour information caused by movements such as the user turning over in bed can be acquired.

[0120] The body contour information obtained in step S13 of processing the auxiliary signal can be used in step S15 of analyzing the Doppler signal, which will be described later. For example, when non-periodic spectral energy is obtained in step S15 of analyzing the Doppler signal using acquisition time information for turning over or snoring, which is represented by non-periodic spectral energy, the body contour information can be used as auxiliary information to determine what biological activity information the energy represents.

[0121] Step S15 of analyzing the Doppler signal is configured to analyze the Doppler signal acquired in step S11 of acquiring the Doppler signal using a Doppler signal analyzer to acquire spectral energy at a predetermined period. Step S15 of analyzing the Doppler signal may acquire spectral energy by applying a predetermined function to the Doppler signal, and analyze the acquired spectral energy according to a predetermined criterion to acquire respiratory spectral energy and cardiac spectral energy, which are spectral energies for the respiratory Doppler signal and cardiac Doppler signal.

[0122] In one embodiment of the present invention, step S15 of analyzing the Doppler signal can utilize a fast Fourier transform (FFT) as a preset function used to obtain spectral energy from the Doppler signal at a preset period. The Fourier transform is a well-known function that decomposes a signal sampled in time or space into time frequency or spatial frequency components. By utilizing this function, step S15 of analyzing the Doppler signal can convert the Doppler signal obtained at a preset period into spectral energy of a specific frequency component.

[0123] After acquiring the spectral energy of a specific frequency component, step S15 of analyzing the Doppler signal according to an embodiment of the present invention is configured to check whether the acquired spectral energy of the specific frequency component is spectral energy that occurs periodically. Step S15 of analyzing the Doppler signal may be configured to continuously acquire spectral energy for the Doppler signal acquired at a pre-set period and determine whether the acquired continuous spectral energy has periodicity. Here, if the spectral energy has periodicity, it may be determined that the spectral energy is spectral energy corresponding to respiration or heart rate, and if the spectral energy does not have periodicity, it may be determined that the spectral energy is not spectral energy corresponding to respiration or heart rate.

[0124] Step S15 of analyzing the Doppler signal can be configured to check whether the aperiodic spectral energy has energy greater than a preset magnitude, and if the spectral energy has energy greater than the preset magnitude, determine that the spectral energy is the spectral energy corresponding to the Doppler frequency generated by the user's body movement, or if the spectral energy has energy equal to or less than the preset magnitude, determine that the spectral energy is the spectral energy corresponding to the Doppler frequency generated by the user's snoring.

[0125] In addition, in step S15 of analyzing the Doppler signal according to an embodiment of the present invention, a pre-set judgment algorithm is used to analyze the periodic spectral energy, and as a result, breathing spectral energy and heart rate spectral energy can be distinguished and obtained.

[0126] Furthermore, in step S15 of analyzing the Doppler signal according to an embodiment of the present invention, the type of aperiodic spectral energy can be identified using the auxiliary signal acquired in the aforementioned auxiliary signal processing step S13. Here, step S15 of analyzing the Doppler signal can be configured to acquire aperiodic spectral energy corresponding to the user's turning over (body movement) using thermal image information, for example, and determine the acquired aperiodic spectral energy as a noise signal to remove it from data for sleep period analysis, which will be described later.

[0127] In addition, in another embodiment, step S15 of analyzing the Doppler signal may be configured to define a pre-set magnitude of energy that can distinguish between body movement and snoring using an auxiliary signal acquired using a thermal image sensor if a pre-set magnitude of energy that can distinguish between both movements is not defined using non-periodic spectral energy.

[0128] In step S15 of analyzing the Doppler signal according to an embodiment of the present invention, once the spectral energy of a specific frequency component is continuously obtained and a determination is made as to whether or not it is a periodic occurrence, step S17 of defining a sleep period is performed to define a sleep state for each period in the entire sleep period using the analysis results.

[0129] The total sleep period may include at least one of a sleep onset period, a deep sleep period, a tossing and turning period, an apnea period, and a snoring period. The sleep onset period refers to a period in which a user enters sleep and is a period that occurs statistically in almost all users, allowing determination of whether or not a user has entered sleep. The deep sleep period refers to a sleep period in which the user's physical activity is stably maintained, and the tossing and turning period refers to a sleep period in which the user's physical activity involves movement. The apnea period refers to a sleep period in which no respiratory activity occurs during the user's physical activity, and the snoring period refers to a sleep period in which the user's physical activity involves snoring.

[0130] The higher the ratio of deep sleep segments to the rest of the sleep segments, the higher the quality of sleep. Therefore, if the definition of each sleep segment can be performed according to the present invention, it is possible to obtain information on the sleep quality of the user and later provide a prescription for improving the sleep quality for each user.

[0131] The step S17 of defining sleep segments is configured to define a sleep state for each segment in the entire sleep segment in the sleep segment definition unit using the analysis result of the step S15 of analyzing the Doppler signal. As described above, each segment in the entire sleep segment includes at least one of a sleep onset segment, a deep sleep segment, a tossing and turning segment, an apnea segment, and a snoring segment. The step S17 of defining sleep segments acquires the analysis result from the step S15 of analyzing the Doppler signal and determines which sleep segment the analysis result represents using a previously set state determination criterion.

[0132] The pre-set state determination criterion may be a pre-set value using the ratio of the respiration spectrum energy and the heart rate spectrum energy acquired in step S15 of analyzing the Doppler signal. As described above, since the magnitude of physical activity caused by respiration is generally greater than the magnitude of physical activity caused by heartbeat, the magnitude of the respiration spectrum energy appears greater than the magnitude of the heart rate spectrum energy in a stable sleep period. Also, in a snoring period, high-frequency vibrations are captured, and additional spectra other than the respiration spectrum energy and the heart rate spectrum energy may appear. Also, in an apnea period, respiration does not occur or occurs only weakly, so the magnitude of the respiration spectrum energy may decrease by the magnitude of the heart rate spectrum energy.

[0133] In step S17 of defining a sleep section according to an embodiment of the present invention, a section in which the spectral ratio between respiration spectral energy and heart rate spectral energy is 5:5 or more can be defined as a deep sleep section.

[0134] In addition, in step S17 of defining the sleep section, a section in which the spectral ratio is less than 5:5 and in which the non-periodic spectral energy is acquired at a level equal to or less than a preset value can be defined as a snoring section, and if the non-periodic spectral energy is not acquired for a preset time, the section can be defined as an apnea section.

[0135] In another embodiment of the present invention, step S17 of defining a sleep zone may define the sleep zone using a sleep onset zone other than the spectrum ratio. Since tossing and turning, apnea, and snoring generally do not occur in the sleep onset zone, similar to the deep sleep zone, step S17 of defining a sleep zone of the present invention may be configured to first obtain the sleep onset zone and then determine the deep sleep zone based on the obtained sleep onset zone.

[0136] Meanwhile, Figure 8 is a graph showing the results of an actual conspiracy experiment using an embodiment of the present invention. Referring to Figure 8, the total sleep period can be represented by A. Period B is the sleep onset period, which, as described above, is the period when statistically most users enter sleep, and can be the period when users actually begin to sleep, appearing as an average of 20 to 30 minutes out of a 10 to 40 minute period. Period C is the deep sleep period, which is a stable sleep period, period D is the snoring period, period E is the tossing and turning period, and period F is the apnea period.

[0137] Section C is a stable deep sleep section, in which the magnitude of the respiratory spectral energy a1 is significantly greater than the magnitude of the heart rate spectral energy b1. Furthermore, the periodicity of the spectral energy indicates that the spectral energy does not contain irregular peaks and is repeated at regular intervals. Therefore, in one embodiment of the present invention, such a sleep section can be defined as a deep sleep section. In this case, as described above, the deep sleep section may be defined as a section in which the ratio of the respiratory spectral energy a1 to the heart rate spectral energy b1 is 5:5 or greater. In other embodiments, the deep sleep section may be defined as a section that has a periodicity greater than a predetermined similarity based on section B, the sleep onset section, and has energy less than a predetermined magnitude.

[0138] Section D is a snoring section, where the magnitude of the respiration spectrum energy a2 is greater than the magnitude of the heartbeat spectrum b2, but where snoring spectrum energy c2, which does not appear in other sections, is also measured. It can be seen that section D, like c2, is a high-frequency spectrum energy measured, which is a frequency higher than the respiration / heartbeat frequency. Therefore, in the sleep section definition unit and the step of defining a sleep section according to the present invention, section D can be defined as a snoring section, where the user is snoring.

[0139] Section F is an apnea section, where the magnitude of the breathing spectrum energy a3 appears similar to the magnitude of the heart rate spectrum energy b3. More specifically, the ratio of the breathing spectrum energy a3 to the heart rate spectrum energy b3 is less than 5:5, and unlike section D, this is a section where the snoring spectrum energy c2 is not measured. This is because the magnitude of the acquired breathing spectrum energy a3 is significantly reduced because breathing activity is weak or nonexistent in the user's body during the process of acquiring a Doppler signal using a radar. Therefore, when comparing the magnitude of the breathing spectrum energy a3 to the magnitude of the heart rate spectrum energy b3, as in the spectrum analysis result for section F of Figure 8, if the ratio appears to be less than 5:5, section F can be defined as an apnea section in the sleep section definition unit and sleep section defining step according to an embodiment of the present invention.

[0140] Section E is a section where turning over occurs, and is a section where spectral energy having a higher magnitude than the spectral energy detected in sections B, C, D, F, etc. This is a section where biological activity having much higher energy than the respiratory spectral energy or the heart rate spectral energy occurs, and the present invention is configured to analyze such biological activity as turning over.

[0141] That is, the device configuration and method of Figures 6 and 7 are configured to acquire and analyze Doppler signals of a user's biological activity using a radar, and define a sleep state for each sleep section using the respiratory spectrum energy and heart rate spectrum energy among them, and the experimental results thereof are shown in Figure 8. By using the present invention described in Figures 6 to 8, it is possible to confirm a user's sleep state and to respond to an emergency situation or provide content for improving the user's sleep quality using the confirmed sleep state.

[0142] Although one embodiment of the present invention has been described above, the concept of the present invention is not limited to the embodiment presented in this specification, and a person skilled in the art who understands the concept of the present invention can easily propose other embodiments by adding, changing, deleting, or adding components within the scope of the same concept, which can also be said to fall within the scope of the concept of the present invention. [Industrial Applicability]

[0143] The non-face-to-face, non-contact fall monitoring system and non-contact sleep monitoring system and method of the present invention protect the user's privacy while increasing the reliability of fall detection, and are non-wearable and non-face-to-face, non-contact, and can detect a user's fall in real time without the user having to wear a separate device. If a fall is detected, the system assesses the level of danger and notifies guardians or medical staff to prevent the risk of a fall or provide guidance on how to respond. During fall and sleep monitoring, the system processes thermal images into closed curves, labeling images, and spectrograms to protect the user's privacy. The system analyzes the user's level of danger based on the user's biosignals and the processed thermal images and provides guidance on how to respond appropriately. The system uses radar to acquire bioactivity information of the user while sleeping, so it does not affect the user's body and can accurately determine the user's sleep state for each sleep period using the user's breathing and heart rate information without disturbing the user's sleep.

Claims

1. a radar sensor unit that transmits radar toward a user, receives radar reflected from the user, and generates a time-series Doppler radar signal using a Doppler radar algorithm; a thermal image sensor unit that is an infrared thermal image camera that captures an image of the user from above the user and generates thermal image data; a radar signal pattern analysis unit that receives the time series Doppler radar signal from the radar sensor unit and analyzes a user's state based on the received time series Doppler radar signal; a thermal image data processing unit that receives the thermal image data generated by the thermal image sensor unit, determines an analysis range using the thermal image data to determine a user's edge, processes the thermal image data to have a closed curve shape from the determined edge using a preset threshold to obtain a closed curve image, performs image labeling using the closed curve image to form a labeled image, and spectrograms the labeled image to form a spectrogram, thereby analyzing the user's condition; A fall determination unit that determines whether or not a fall has occurred based on the user's status data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit; A non-face-to-face, non-contact fall detection system comprising:

2. The radar signal pattern analysis unit The non-face-to-face, non-contact fall detection system of claim 1, characterized in that a movement change signal and a biological signal are separated from the time-series Doppler radar signal transmitted from the radar sensor unit, and the separated movement change signal is used as a time-series movement Doppler radar signal to analyze the user's condition.

3. The radar signal pattern analysis unit generating first status data indicating that a fall has occurred when, within the first hour, a positive / negative change occurs in data having a magnitude of amplitude equal to or greater than a predetermined value in the time-series movement Doppler radar signal and then the magnitude of the amplitude becomes zero; The non-face-to-face, non-contact fall detection system of claim 2, characterized in that if a positive or negative change in data having an amplitude greater than a predetermined value occurs in the time-series movement Doppler radar signal after the first time, second status data is generated, indicating that the user is slowly falling and a fall is suspected.

4. The thermal image data processing unit includes: determining the user's state as one of a fourth state to a sixth state based on at least one of the closed curve image, the labeled image, and the spectrogram, and forming fourth state data to sixth state data, respectively; the fourth state data indicates a state in which the user is positioned on an object at a predetermined height from a floor, The non-face-to-face, non-contact fall detection system of claim 3, wherein the fifth status data indicates a state in which the user has fallen from an object located at a predetermined height from the floor, and the sixth status data indicates a state in which the user has fallen from an object located at a predetermined height from the floor.

5. The fall determination unit When the first status data is transmitted from the radar signal pattern analysis unit, and the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit, and the fourth status data is generated before the first status data generation time, it is determined that the user has fallen, and a first signal indicating that the user has fallen is generated. When the first status data is transmitted from the radar signal pattern analysis unit, and the sixth status data, the fifth status data, and the fourth status data are sequentially transmitted from the thermal image data processing unit, and the sixth status data is generated before the first status data generation time, it is determined that the user has not fallen, and a second signal indicating that a fall has not occurred is generated. When the radar signal pattern analysis unit transmits second status data indicating that a fall is suspected, and the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit, and the fourth status data is generated before the first status data generation time, it is determined that the user has fallen, and a third signal indicating that the user has fallen is generated. When third status data is transmitted from the radar signal pattern analysis unit and only fourth status data is transmitted from the thermal image data processing unit until a fourth time after the time when the third status data is transmitted, a fourth signal is generated indicating that there is a high risk of the user falling; 5. The non-face-to-face, non-contact fall detection system of claim 4, wherein when the first status data or the second status data is transmitted from the radar signal pattern analysis unit, the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit, and after a fourth time, the first status data or the second status data is further transmitted from the radar signal pattern analysis unit, and the sixth status data, the fifth status data, and the fourth status data are sequentially transmitted from the thermal image data processing unit, it is determined that the user who fell has returned to their original position, and a fifth signal indicating that the user has returned to their original position is generated.

6. The non-face-to-face, non-contact fall detection system of claim 5, wherein the radar signal pattern analysis unit analyzes the separated vital signs and generates a sixth signal notifying of a dangerous situation if the vital signs differ from the vital signs pattern for a predetermined period by more than a predetermined range.

7. The non-face-to-face, non-contact fall detection system further includes a fall risk determination unit, The fall risk determination unit if the sixth signal is continuously received from the radar signal pattern analysis unit for a fifth time period and the first signal is received from the fall determination unit, and if the fourth signal is not received within the fifth time period, it is determined that the user has continued to fall and that the biological signal is in a poor state, and a seventh signal is generated to notify that the user is in a very high risk of falling; If the sixth signal is not received from the radar signal pattern analysis unit for the fifth time period and the fourth signal is not received from the fall determination unit for the fifth time period, an eighth signal is generated indicating that the user continues to have fallen but that the user's biological signal is stable; The non-face-to-face, non-contact fall detection system described in claim 6, characterized in that when a third signal is transmitted from the fall determination unit, a ninth signal is generated to notify that there is a high possibility that the user will fall.

8. The non-face-to-face, non-contact fall detection system further includes an alarm unit, The alarm unit When the seventh signal is transmitted from the fall risk determination unit, a first alarm signal is generated to notify that the user is in a very critical state, and the first alarm signal is transmitted to a predetermined guardian and medical staff; When the eighth signal is transmitted from the fall risk determination unit, a second alarm signal is generated to notify a fall state of the user, and the second alarm signal is transmitted to a predetermined guardian or medical staff; The non-face-to-face, non-contact fall detection system of claim 7, characterized in that when the ninth signal is transmitted from the fall risk determination unit, a third alarm signal is generated to notify that the user is on the verge of falling, and the third alarm signal is transmitted to a predetermined guardian or medical staff.

Images