Non-face-to-face, non-contact fall monitoring system, non-contact sleep monitoring system and method

Abstract

The present invention is characterized by being embodied in a non-face-to-face, non-contact fall monitoring system and a non-contact sleep monitoring system and method that improves the reliability of fall detection by detecting a user's fall using radar signals and thermal image signals, and also analyzes and provides guidance on the user's sleep state.

Description

[Technical field]

[0001] The present invention relates to a non-face-to-face, non-contact type fall monitoring system that improves the reliability of fall detection by detecting a user's fall using radar signals and thermal image signals, and a non-contact type sleep monitoring system and method that also analyzes and provides guidance on a user's sleep state. [Background technology]

[0002] In recent years, the population that needs care or attention has been increasing in Korean society. The population that needs care or attention can be defined as the population that needs help in case of emergency or has no one living with them, and can include, for example, the elderly, the disabled, and those living alone. According to a recent survey, the elderly population in Korea has exceeded 9 million, and the population of those living alone and those with disabilities has exceeded 6.6 million and 2.6 million, respectively. In addition, the population of people who die alone, that is, those who are found after death without a cohabitant, has exceeded 10,000.

[0003] As a result, the number of accidents involving falls, which are becoming increasingly common, is increasing. Falls are accidents in which a person falls regardless of his or her will, and they can cause various complications and even death.

[0004] Therefore, a fall detection technique that can accurately detect a fall accident and deal with it quickly is important.

[0005] The fall detection device using a camera detects the elderly person's movements through images and provides an alert to the guardian when a fall is detected, but there are problems such as human rights violations and privacy issues.

[0006] In addition, conventional fall detection devices that use radar signals have the problem that they will determine that a fall has not occurred when one actually has, or that a fall has occurred due to radio wave interference from wind or the like when no fall has actually occurred.

[0007] In addition, conventional wearable fall detection devices had the disadvantage that it was difficult to properly analyze the user's fall because users often forgot to wear the wearable device or lost the wearable device itself.

[0008] In addition, such furniture for single-person living may cause irregular sleep quality when the person lives an irregular lifestyle, which may cause health problems. In order to measure the quality of sleep, a person must make a reservation at a specialized hospital, visit the hospital at a time appropriate to the schedule, and go through the troublesome process of using specialized equipment while sleeping directly, which makes it difficult to measure the quality of sleep regularly. Summary of the Invention [Problem to be solved by the invention]

[0009] The present invention has been made to solve the above problems, and an object of the present invention is to provide a non-face-to-face, non-contact fall monitoring system that can protect the user's privacy while increasing the reliability of fall detection.

[0010] Another object of the present invention is to provide a non-face-to-face, non-contact fall monitoring system that can monitor a user's fall without the user having to wear a sensing device.

[0011] It is still another object of the present invention to provide a non-face-to-face, non-contact fall monitoring system that determines the degree of risk when a fall occurs and notifies guardians and medical staff of the risk.

[0012] Another object of the present invention is to provide a non-contact sleep monitoring system and method that increases the reliability of analysis of a user's state while sleeping by using a radar sensor unit to acquire biological activity information while the user is sleeping, and using a thermal image sensor unit to acquire turning over movements as an auxiliary signal.

[0013] Another object of the present invention is to provide a non-contact sleep monitoring system and method that can protect the privacy of a user by not using thermal imaging data as it is. [Means for solving the problem]

[0014] The non-face-to-face, non-contact fall monitoring system of the present invention may include a radar sensor unit that transmits radar toward a user, receives radar reflected from the user, and generates a time-series Doppler radar signal using a Doppler radar algorithm; a thermal image sensor unit that images the user above the user and generates thermal image data; a radar signal pattern analysis unit that receives the time-series Doppler radar signal from the radar sensor unit and analyzes the user's condition based on the transmitted time-series Doppler radar signal; a thermal image data processing unit that receives the thermal image data generated by the thermal image sensor unit and processes the transmitted thermal image data to analyze the user's condition; and a fall determination unit that determines whether or not a fall has occurred based on the user's condition data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit.

[0015] The radar signal pattern analysis unit may separate a motion change signal and a biological signal from the time series Doppler radar signal transmitted from the radar sensor unit, and analyze the user's condition as a time series motion Doppler radar signal by using the separated motion change signal.

[0016] The radar signal pattern analysis unit may generate first status data indicating that a fall has occurred if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal within a first hour and then the amplitude becomes zero, and may generate second status data indicating that the user is slowly falling and a fall is suspected if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal after the first hour.

[0017] The thermal image data processing unit can process the thermal image data by including the steps of: determining an analysis range in the transmitted thermal image data; determining a user's edge within the determined analysis range; and processing the thermal image data to have a closed curve shape from the determined edge using a preset threshold value to obtain a closed curve image.

[0018] The thermal image data processing unit can form a labeled image by further including a step of performing image labeling on the closed curve image.

[0019] The thermal imaging data processor may further include the step of spectrogramming the labeled image to form a spectrogram.

[0020] The thermal image data processing unit can determine the user's state to be one of a fourth state to a sixth state based on at least one of the closed curve image, the labeled image, and the spectrogram, and form fourth state data to sixth state data, respectively.

[0021] The fourth state data may indicate a state in which the user is positioned on an object at a predetermined height from the floor, the fifth state data may indicate a state in which the user is falling from an object positioned at a predetermined height from the floor, and the sixth state data may be a state in which the user has fallen from an object positioned at a predetermined height from the floor.

[0022] the fall determination unit receives first status data from the radar signal pattern analysis unit, receives the fourth status data, the fifth status data, and the sixth status data from the thermal image data processing unit in sequence, and, when the fourth status data is generated before a generation time of the first status data, determines that the user has fallen and generates a first signal indicating that the user has fallen; receives the first status data determined to be a first status from the radar signal pattern analysis unit, receives the sixth status data, the fifth status data, and the fourth status data from the thermal image data processing unit in sequence, and, when the sixth status data is generated before a generation time of the first status data, determines that the user has not fallen and generates a second signal indicating that a fall has not occurred; receives the second status data indicating that a fall is suspected from the radar signal pattern analysis unit, receives the fourth status data, the fifth status data, and the sixth status data from the thermal image data processing unit in sequence, and, when the first status data is generated, and generate a third signal indicating that the user has fallen, if the fourth status data is generated before the fourth hour, the radar signal pattern analysis unit determines that the user has fallen and generates a third signal indicating that the user has fallen; if the third status data is transmitted from the radar signal pattern analysis unit and only the fourth status data is transmitted from the thermal image data processing unit until the fourth hour from the time when the third status data was transmitted, the radar signal pattern analysis unit generates a fourth signal indicating that the user is at a high risk of falling; if the first status data or the second status data is transmitted from the radar signal pattern analysis unit, the fourth status data, the fifth status data, and the sixth status data are all transmitted sequentially from the thermal image data processing unit, and after the fourth hour, the radar signal pattern analysis unit further transmits the first status data or the second status data, and the sixth status data, the fifth status data, and the fourth status data are transmitted sequentially from the thermal image data processing unit, the radar signal pattern analysis unit determines that the user who has fallen has returned to his / her original position, and generates a fifth signal indicating that the user has returned to his / her original position.

[0023] The radar signal pattern analysis unit can analyze the separated vital sign and generate the sixth signal notifying a dangerous situation when the vital sign differs from a vital sign pattern for a predetermined period by a predetermined range or more.

[0024] The non-contact fall and sleep monitoring system may further include a fall risk determination unit.

[0025] When the sixth signal is continuously received from the radar signal pattern analysis unit for a fifth time period and the first signal is received from the fall determination unit, and the fourth signal has not been received within the fifth time period, the fall risk determination unit may determine that the user's fallen state continues and the biological signal is in a poor state, and generate a seventh signal notifying that the user is at a very high risk due to a fall; when the sixth signal is not received from the radar signal pattern analysis unit for the fifth time period and the fourth signal is not received from the fall determination unit for the fifth time period, the fall risk determination unit may generate an eighth signal indicating that the user's fallen state continues but the biological signal of the user is stable; and when the third signal is received from the fall determination unit, the fall risk determination unit may generate a ninth signal notifying that there is a high possibility that the user will fall.

[0026] The non-face-to-face, non-contact fall monitoring system further includes an alarm unit, which generates a first alarm signal notifying that the user is in an extremely critical state when the seventh signal is transmitted from the fall risk determination unit, and transmits the first alarm signal to both a predetermined guardian and medical staff, generates a second alarm signal notifying that the user has fallen when the eighth signal is transmitted from the fall risk determination unit, and transmits the second alarm signal to the predetermined guardian or medical staff, and generates a third alarm signal notifying that the user is on the verge of falling when the ninth signal is transmitted from the fall risk determination unit, and transmits the third alarm signal to the predetermined guardian or medical staff.

[0027] The non-contact sleep monitoring system and method of the present invention may include a Doppler signal acquisition unit that uses a radar to acquire a Doppler signal including biological activity information; an auxiliary signal processing unit that uses a thermal image sensor to acquire turning over movements as an auxiliary signal; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a pre-set period, determines whether the spectral energy is a periodic acquisition, and if it is determined that the spectral energy is a non-periodic spectral energy, classifies the non-periodic spectral energy using the auxiliary signal; and a sleep section definition unit that defines a sleep state for each section in an entire sleep section using a ratio of respiratory spectral energy and heart rate spectral energy among the spectral energy and a combination of the non-periodic spectral energy.

[0028] The Doppler signals may include a respiratory Doppler signal for obtaining information on the user's breathing, and a heart rate Doppler signal for obtaining information on the user's heartbeat.

[0029] The Doppler signal analysis unit may obtain the spectral energy by performing a fast Fourier transform on the Doppler signal.

[0030] The sleep section definition unit can define a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy is 5:5 or more in the entire sleep section as a deep sleep section, and can define a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy is less than 5:5 as an apnea section.

[0031] The sleep section definition unit can define a section in which the non-periodic spectral energy exists and appears at a magnitude equal to or greater than a preset magnitude as a tossing section, and can define a section in the apnea section in which the non-periodic spectral energy appears at a magnitude equal to or less than a preset magnitude as a snoring section.

[0032] A non-contact sleep monitoring system and method according to another embodiment of the present invention may include a Doppler signal acquisition unit that acquires a Doppler signal including biological activity information using a radar; an auxiliary signal processing unit that acquires a turning over motion as an auxiliary signal using a thermal image sensor; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a pre-set period, determines whether the spectral energy is a periodic acquisition, and classifies the spectral energy using the auxiliary signal; and a sleep zone definition unit that defines a sleep state for the remaining sleep zones using a pre-set percentage range and non-periodic spectral energy based on an average of the spectral energy of a sleep entry zone that satisfies a pre-set criterion among all sleep zones.

[0033] The non-contact sleep monitoring system and method of the present invention may include the steps of: acquiring a Doppler signal including biological activity information in a Doppler signal acquisition unit using a radar; processing an auxiliary signal in an auxiliary signal processing unit using a thermal image sensor to acquire a turning over movement as an auxiliary signal; analyzing the Doppler signal in a Doppler signal analysis unit to acquire spectral energy at a pre-set period, determining whether the spectral energy is periodically acquired, and analyzing the Doppler signal by dividing the spectral energy using the auxiliary signal; and defining a sleep period in a sleep period definition unit using a ratio of respiratory spectral energy and heart rate spectral energy and a combination of non-periodic spectral energy among the spectral energy to define a sleep state for each period in the entire sleep period.

[0034] The Doppler signal may include a respiratory Doppler signal for acquiring information on the user's breathing, and a cardiac Doppler signal for acquiring information on the user's heartbeat.

[0035] The step of analysing the Doppler signal may involve performing a Fast Fourier Transform on the Doppler signal to obtain the spectral energy.

[0036] The step of defining the sleep interval may define an interval in which the ratio of the breathing spectral energy to the heart rate spectral energy in the entire sleep interval is 5:5 or more and in which the non-periodic spectral energy is not present as a deep sleep interval, and may define an interval in which the ratio of the breathing spectral energy to the heart rate spectral energy is less than 5:5 and in which the non-periodic spectral energy is not present as an apnea interval.

[0037] The step of defining the sleep section may define a section in which the non-periodic spectral energy exists and appears at a magnitude equal to or greater than a preset magnitude as a tossing section, and may define a section in the apnea section in which the non-periodic spectral energy appears at a magnitude equal to or less than a preset magnitude as a snoring section.

[0038] The non-contact sleep monitoring system and method of the present invention may include the steps of: acquiring a Doppler signal including biological activity information using a radar in a Doppler signal acquisition unit; processing an auxiliary signal that acquires turning over motion as an auxiliary signal using a thermal image sensor in an auxiliary signal processing unit; analyzing the Doppler signal in a Doppler signal analysis unit to acquire spectral energy at a pre-set period, determining whether the spectral energy is periodically acquired, and classifying the spectral energy using the auxiliary signal; and defining a sleep state for the remaining sleep periods using a pre-set percentage range and non-periodic spectral energy based on an average of the spectral energy of a sleep entry period that satisfies a pre-set criterion among all sleep periods in a sleep period definition unit. Effect of the Invention

[0039] The present invention has the advantage of protecting the user's privacy while increasing the reliability of fall detection.

[0040] In addition, the present invention has an advantage in that it is possible to detect a user's fall in real time without the user having to wear a separate device by using a non-face-to-face, non-contact type that is not a wearable type.

[0041] In addition, the present invention has the advantage that, when a fall is determined, the degree of danger is judged and notified to guardians or medical staff, thereby making it possible to prevent or respond to the danger of a fall in advance.

[0042] In addition, the present invention has an advantage that, when monitoring a fall, the thermal image can be processed to convert it into a closed curve, a labeling image, and a spectrogram, thereby protecting the privacy of the user.

[0043] Furthermore, the present invention has an advantage in that it is possible to analyze the user's risk level based on the user's biosignal and the processed thermal image, and to take appropriate measures.

[0044] Furthermore, the present invention has the advantage that by using a radar to acquire biological activity information of a user while sleeping, the user's body is not affected and the user's sleep is not disturbed.

[0045] In addition, the present invention has an advantage in that it is possible to accurately determine the user's sleep state for each sleep period using the user's respiration and heart rate information.

[0046] In addition, the present invention has the advantage that sleep monitoring is performed using the contour of a thermal image together with a Doppler signal, thereby improving reliability compared to conventional sleep monitoring based on a Doppler signal. [Brief description of the drawings]

[0047] [Figure 1] FIG. 1 is a diagram illustrating a non-face-to-face, non-contact fall monitoring system according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a movement pattern analyzed by a radar signal pattern analyzer according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing a process of analyzing the thermal imaging data and determining the fourth state. [Figure 4] FIG. 4 is a diagram showing a process of analyzing the thermal imaging data and determining the fifth state. [Diagram 5] FIG. 5 is a diagram showing a process of determining the sixth state by analyzing the thermal imaging data. [Figure 6] FIG. 6 is a block diagram illustrating a non-contact sleep monitoring system according to one embodiment of the present invention. [Figure 7] FIG. 7 is a flow chart illustrating a method for non-contact sleep monitoring according to an embodiment of the present invention. [Figure 8] FIG. 8 is a graph showing the results of an actual conspiracy experiment using one embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing a process of processing thermal imaging data in the auxiliary signal processor of the present invention. BEST MODE FOR CARRYING OUT THEINVENTION

[0048] The present invention is characterized by including a radar sensor unit that transmits radar toward a user, receives radar reflected from the user, and generates a time series Doppler radar signal using a Doppler radar algorithm; a thermal image sensor unit that images the user above the user and generates thermal imaging data; a radar signal pattern analysis unit that receives the time series Doppler radar signal from the radar sensor unit and analyzes the user's condition based on the transmitted time series Doppler radar signal; a thermal image data processing unit that receives the thermal imaging data generated by the thermal image sensor unit and processes the transmitted thermal imaging data to analyze the user's condition; and a fall determination unit that determines whether or not a fall has occurred based on the user's condition data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0049] The non-face-to-face, non-contact fall monitoring system according to one embodiment of the present invention may include a radar sensor unit 100, a thermal image sensor unit 200, a radar signal pattern analysis unit 300, a thermal image data processing unit 400, a fall determination unit 500, a fall risk determination unit 600, and an alarm unit 700.

[0050] A non-face-to-face, non-contact fall monitoring system according to an embodiment of the present invention can be installed not only in the space where a user sleeps but also in the space where the user lives.

[0051] 1. Radar sensor part (100) The radar sensor unit 100 transmits radar toward a user, receives radar reflected from the user, and uses a Doppler radar algorithm to form a time series Doppler radar signal as shown in Fig. 2. Here, in the time series Doppler radar signal, the X-axis represents time and the Y-axis represents the magnitude of the Doppler radar signal. In the time series Doppler radar signal, positive and negative data on the Y-axis represent opposite directions, and the magnitude of the amplitude, which is the absolute value of the Y-axis, represents the speed of the user's movement or the movement of a large area.

[0052] 2. Thermal image sensor unit (200) The thermal image sensor unit 200 may be an infrared thermal imaging camera. The infrared thermal imaging camera may capture an image of a user from above the user. Thermal image data of the user is transmitted to the determination unit. Preferably, the infrared thermal imaging camera may be installed on a ceiling above a bed in a bedroom.

[0053] 3. Radar signal pattern analysis unit (300) The radar signal pattern analysis unit 300 may separate a motion change signal and a biological signal from the time series Doppler radar signal transmitted from the radar sensor unit 100. The time series Doppler radar signal separated from the motion change signal is hereinafter referred to as a time series motion Doppler radar signal.

[0054] When analyzing the time-series movement Doppler radar signal, the positive and negative phases of the Y-axis mean opposite directions. Therefore, when the phase change of the Y-axis changes from a positive phase to a negative phase, it means that the movement has changed to a second movement that is opposite to the direction of the first movement, such as a movement from front to back, a movement from back to front, a movement from up to down, or a movement from down to up. Referring to FIG. 2, the Doppler time-series data corresponding to a fall means that the movement of the whole body or part of the body of the user has changed in the opposite direction over a total of four phase changes from positive to negative to positive to negative to positive to positive.

[0055] The magnitude of the amplitude of the time-series motion Doppler radar signal (absolute value on the Y-axis) indicates either the speed or the magnitude of the motion, or both. In other words, a large amplitude indicates fast movement or a large area of ​​motion, such as arm or leg movements that are not biological signals.

[0056] Therefore, when a part of the body with a large surface area such as an arm or a leg is quickly turned, the phase of the large amplitude of the Y-axis can change from positive to negative or from negative to positive. In the present invention, the change in the Y-axis phase from positive to negative or from negative to positive is called a positive / negative change.

[0057] The radar signal pattern analysis unit 300 generates first status data indicating that a fall has occurred when a positive / negative change occurs in data having a predetermined or greater amplitude in the time-series movement Doppler radar signal within one hour, and then the amplitude becomes zero.

[0058] In one embodiment, the radar signal pattern analysis unit 300 generates first status data indicating that a fall has occurred when the amplitude of the time-series movement Doppler radar signal remains constant at 0 after the amplitude is greater than or equal to a predetermined range and a positive / negative change occurs within 200 ms.

[0059] In addition, if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal after the first time, the radar signal pattern analysis unit 300 may generate second status data indicating that the user is slowly falling and is suspected of having fallen.

[0060] In addition, the radar signal pattern analysis unit 300 can generate third status data, which means that the user is not falling but is continuing to move, when the magnitude of the amplitude of the time series movement Doppler radar signal is equal to or greater than a predetermined range and the amplitude of a second time series movement Doppler radar signal occurs at or after a second time after the generation of the first time series movement Doppler radar signal based on the analysis of the time series movement Doppler radar signal.

[0061] When the second status data is generated during the analysis of the time-series movement Doppler radar signal, it is difficult to accurately determine whether the user has fallen or not.

[0062] In addition, the time-series motion Doppler radar signal analysis may erroneously determine that a user has fallen if the user jumps from under the bed onto the bed, since the hands and feet quickly change direction and the amplitude of the motion Doppler radar signal is large and a positive / negative change occurs within a predetermined time, as opposed to a fall.

[0063] In addition, the time-series movement Doppler radar signal analysis is unable to determine that a fall has occurred when a fall has not actually occurred but when the patient is lying on the edge of the bed and there is a high possibility of a fall occurring. It only determines that a fall has occurred after the fall has actually occurred, and therefore has the disadvantage of being unable to prevent falls.

[0064] The present invention utilizes thermal imaging data analysis together with time series motion Doppler radar signal analysis to complement this.

[0065] 4. Thermal image data processing unit (400) The thermal image data processing unit 400 processes the thermal image data transmitted from the thermal image sensor unit 200 to analyze the state of the user.

[0066] The thermal image data processing unit 400 determines an analysis range from the transmitted thermal image data. Then, the thermal image data processing unit 400 determines a user's edge within the determined analysis range. Then, the thermal image data processing unit 400 processes the thermal image data to have a closed curve shape from the determined edge using a preset threshold value, thereby obtaining an image with a closed curve shape.

[0067] Then, the thermal imaging data processing unit 400 may further perform image labeling on the closed curve image to form a labeled image. Thereafter, the thermal imaging data processing unit 400 may spectrogramize the labeled image to form a spectrogram.

[0068] 3 is a diagram showing a process of analyzing the thermal image data transmitted from the thermal image sensor unit 200 and determining the fourth state. Here, the fourth state means a state where the user is positioned on an object at a predetermined height from the floor (e.g., lying on a bed). FIG. 3a shows the thermal image data transmitted from the thermal image sensor unit 200, FIG. 3b shows the thermal image data with the analysis range determined, FIG. 3c shows the edge of the user, FIG. 3d shows the closed curve image of the user, FIG. 3e shows the labeled image of the user, and FIG. 3f shows the spectrogram of the labeled image of the user.

[0069] 4 is a diagram showing a process of analyzing the thermal image data transmitted from the thermal image sensor unit 200 and determining the fifth state. Here, the fifth state means a state in which the user falls from an object (e.g., a bed) located at a predetermined height from the floor. FIG. 4a shows the thermal image data transmitted from the thermal image sensor unit 200, FIG. 4b shows the thermal image data with the analysis range determined, FIG. 4c shows the edge of the user, FIG. 4d shows the closed curve image of the user, FIG. 4e shows the labeled image of the user, and FIG. 4f shows the spectrogram of the labeled image of the user.

[0070] 5 is a diagram showing a process of analyzing the thermal image data transmitted from the thermal image data processing unit 400 and determining the sixth state. Here, the sixth state means a state where the user falls off an object (e.g., a bed) located at a predetermined height from the floor. FIG. 5a shows the thermal image data transmitted from the thermal image sensor unit 200, FIG. 5b shows the thermal image data with the analysis range determined, FIG. 5c shows the edge of the user, FIG. 5d shows the closed curve image of the user, FIG. 5e shows the labeled image of the user, and FIG. 5f shows the spectrogram of the labeled image of the user.

[0071] The thermal image data processing unit 400 judges the state into a fourth state in which the subject is lying on an object of a predetermined height (e.g., a bed), a fifth state in which the subject is falling from an object of a predetermined height (e.g., a bed), and a sixth state in which the subject is falling from an object of a predetermined height (e.g., a bed) based on at least one of the closed curved images 3d, 4d, and 5d, the labeled images 3e, 4e, and 5e, and the spectrograms 3f, 4f, and 5f, and generates fourth state data indicating the fourth state, fifth state data indicating the fifth state, and sixth state data indicating the sixth state. The fourth state data to sixth state data thus generated can be transmitted to the fall determination unit 500 together with the time when the fourth state data to sixth state data were generated.

[0072] Preferably, the thermal imaging data processing unit 400 determines the fourth state to the sixth state through a morphological analysis of the spectrogram, generates fourth state data indicating the fourth state, fifth state data indicating the fifth state, and sixth state data indicating the sixth state, and transmits the thus generated fourth state data to sixth state data to the fall determination unit 500 together with the generation times of the fourth state data to sixth state data.

[0073] 5. Fall detection unit (500) The fall determination unit 500 can match the first to third status data transmitted from the radar signal pattern analysis unit 300 with the fourth to sixth status data transmitted from the thermal imaging data processing unit 400 at the same time, and then match at least one of the first to third status data with the fourth to sixth status data to determine whether or not a fall has occurred and the degree of risk of a fall.

[0074] (1) A fall is detected (the first signal is generated) The fall determination unit 500 receives first status data determined to be a first status from the radar signal pattern analysis unit 300, and sequentially receives fourth status data, fifth status data, and sixth status data from the thermal image data processing unit 400. If the fourth status data is generated before the first status data generation time, the fall determination unit 500 determines that the user has fallen, generates a first signal indicating that a user has fallen, and transmits the first signal to the fall risk determination unit 600.

[0075] (2) It is determined that the person did not fall (a second signal is generated). Conversely, when the first status data determined to be a first status is transmitted from the radar signal pattern analysis unit 300, and the sixth status data, fifth status data, and fourth status data are sequentially transmitted from the thermal imaging data processing unit 400, and the sixth status data is generated before the first status data generation time, it can be determined that the user jumped onto an object at a predetermined height that was not a fall, and a second signal indicating that a fall did not occur can be generated and transmitted to the fall risk determination unit 600.

[0076] (3) It is determined that the fall occurred at a slow speed (a third signal is generated). The fall determination unit 500 receives second status data indicating that a fall is suspected from the radar signal pattern analysis unit 300, and sequentially receives fourth, fifth, and sixth status data from the thermal image data processing unit 400. If the fourth status data is generated before the first status data generation time, the fall determination unit 500 determines that the user has fallen, and generates a third signal indicating that the user has fallen as the user slowly falls over, and transmits the third signal to the fall risk determination unit 600.

[0077] (4) The risk of falling is judged to be high (the fourth signal is generated). When the fall determination unit 500 receives third status data determined to be a third status from the radar signal pattern analysis unit 300 and receives only fourth status data from the thermal imaging data processing unit 400 from the time the third status data is received until the third hour or later, the fall determination unit 500 can determine that there is a high risk of the user falling, generate a fourth signal indicating a high risk of a fall, and transmit the fourth signal to the fall risk determination unit 600.

[0078] (5) Determines that the device has returned to its original position (generates the fifth signal) When the fall determination unit 500 receives the first status data determined to be a first status from the radar signal pattern analysis unit 300, receives the fourth status data, the fifth status data, and the sixth status data from the thermal image data processing unit 400 in sequence, and receives the first status data determined to be the first status from the radar signal pattern analysis unit 300 again after a fourth time, and receives the sixth status data, the fifth status data, and the fourth status data in sequence from the thermal image data processing unit 400, the fall determination unit 500 may determine that the user who has fallen has returned to his / her original position, and generate a fifth signal indicating that the user has returned to his / her original position and transmit the fifth signal to the fall risk determination unit 600.

[0079] (5) Poor biological signals (generates the sixth signal) The radar signal pattern analysis unit 300 may analyze the separated biosignal, and if the biosignal differs from a biosignal pattern for a predetermined period by a predetermined range or more, generate a sixth signal notifying a dangerous situation and transmit the sixth signal to the fall risk determination unit 600. For example, if the heart rate or respiration rate, which is a biosignal, is decreased or increased by 50% or more from an average heart rate or average respiration rate for one month or one week, the radar signal pattern analysis unit 300 may generate a sixth signal notifying that the user's biosignal is bad and the user's condition is dangerous.

[0080] 6. Fall risk determination unit (600) The fall risk determination unit 600 receives a sixth signal from the radar signal pattern analysis unit 300 and the first to fifth signals from the fall determination unit 500, analyzes the risk of the user falling and generates a signal indicating each risk.

[0081] (1) There is a very high risk of falling (generates the seventh signal) If the fall risk determination unit 600 continuously receives the sixth signal from the radar signal pattern analysis unit 300 for a fifth time period and receives the first signal from the fall determination unit 500 and does not receive the fourth signal within the fifth time period, it determines that the user continues to fall (i.e., has fallen and is unable to get up into bed) and that the biological signals have been poor for a certain period of time, and generates a seventh signal notifying that the user is at very high risk of falling and transmits it to the alarm unit 700.

[0082] (2) There is a very high risk of falling (generates the seventh signal) If the fall risk determination unit 600 continuously receives the sixth signal from the radar signal pattern analysis unit 300 for a fifth time period and receives the third signal from the fall determination unit 500, but does not receive the fourth signal within the third time period, it determines that the user continues to fall (i.e., has fallen and is unable to get up into bed) and that the biological signals have been in a poor state for a certain period of time, and generates a seventh signal notifying that the user is at very high risk of falling and transmits the seventh signal to the alarm unit 700.

[0083] 3) The fall continues but the vital signs are stable (signal number 8 is generated) If the fall risk determination unit 600 does not receive the sixth signal from the radar signal pattern analysis unit 300 during the fifth time period, and if the fall determination unit 500 receives the third signal but does not receive the fourth signal within the third time period, the fall risk determination unit 600 can generate an eighth signal to notify the user that the user continues to fall (i.e., has fallen and is unable to get up into bed), but that the vital signs are not bad and the user is not at high risk of falling, and transmit the eighth signal to the alarm unit 700.

[0084] (4) There is a high possibility of a fall occurring (the 9th signal is generated) When the fall risk determination unit 600 receives a fourth signal from the fall determination unit 500, it can generate a ninth signal notifying that the user is likely to fall and transmit the ninth signal to the alarm unit 700.

[0085] (4) No fall occurred (no signal generated) When the fall risk determination unit 600 receives the second signal from the fall determination unit 500, it analyzes that the user is at a low risk of falling and does not generate a separate signal.

[0086] (5) The fall has ended and the biosignals are stable (no signal is generated) If the fall risk determination unit 600 does not receive a sixth signal from the radar signal pattern analysis unit 300 within the five hours, but receives a fourth signal within the five hours, it determines that the user's biological signals are stable and the user's fall has ended (i.e., the user has climbed into bed by himself / herself), so there is no danger to the user's safety, and does not generate a separate signal.

[0087] 7. Alarm section (700) When the alarm unit 700 receives the seventh signal from the fall risk determination unit 600, it can generate a first alarm signal to notify that the user has fallen and that the vital signs are poor and in a very dangerous state, and transmit the first alarm signal to both a predetermined guardian and medical staff.

[0088] When the alarm unit 700 receives the eighth signal from the fall risk determination unit 600, it can generate a second alarm signal notifying that the user is in a fall state but the vital signs are stable, and transmit the second alarm signal to a predetermined guardian or medical staff.

[0089] The alarm unit 700 may generate a third alarm signal notifying that the user is on the verge of falling and transmit the third alarm signal to a predetermined guardian or medical staff when the ninth signal is transmitted from the fall risk determination unit 600. When the third alarm signal is transmitted, the guardian or medical staff may take appropriate measures to prevent the user from falling.

[0090] FIG. 6 is a block diagram showing a non-contact sleep monitoring system according to an embodiment of the present invention. The non-contact sleep monitoring system 1 according to an embodiment of the present invention is installed in a space where a user lives and sleeps, for example, a room, and when the user goes to sleep, it can be configured to detect sleep, obtain a Doppler signal for biological activity, and define the user's sleep state by analyzing the obtained Doppler signal. For this purpose, the present invention can use a biological activity measuring device using a radar that has already been installed in the above-mentioned space. The non-contact sleep monitoring system 1 according to an embodiment of the present invention can be configured to include a Doppler signal acquisition unit 11, an auxiliary signal processing unit 13, a Doppler signal analysis unit 15, and a sleep segment definition unit 17, as shown in FIG. 6.

[0091] The Doppler signal acquisition unit 11 is configured to acquire a Doppler signal including biological activity information from a radar installed in the user's sleep space. The Doppler signal is configured to include a respiratory Doppler signal that is information on the user's breathing, and a heartbeat Doppler signal that is information on the user's heartbeat. The Doppler signal can be defined as a signal that includes a Doppler frequency generated by the movement of the user when the radar transmits radio waves to the user and the transmitted radio waves return. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal may be Doppler signals for changes in the movement of the body organs that move when breathing and the body organs that move when the heart beats.

[0092] The auxiliary signal processor 13 is configured to acquire an auxiliary signal using an auxiliary biological activity information acquisition device further provided in the user's sleep space, and analyze the acquired auxiliary signal to acquire information on the user's body movements (turning over in sleep) other than breathing or heartbeat movements. To this end, the auxiliary signal processor 13 can acquire measurement results from a thermal image sensor, which is an auxiliary biological activity information acquisition device. The thermal image sensor can be provided to measure the user's body movements.

[0093] The auxiliary signal processor 13 can continuously obtain contour information of the user's body using a thermal image sensor. When the thermal image sensor is used to continuously obtain contour information of the user's body, it is possible to obtain changes in the contour information that occur due to movements such as the user turning over in bed.

[0094] Preferably, the auxiliary signal processor 13 can continuously obtain information on the user's movements by analyzing the thermal imaging data obtained by the thermal image sensor. In the present invention, for the sake of user privacy, the thermal imaging data is not used as is, but the user's movements can be obtained through contour information or spectrogram.

[0095] The auxiliary signal processor 13 processes the thermal image data transmitted from the thermal image sensor to analyze the user's movement.

[0096] The auxiliary signal processor 13 determines an analysis range from the transmitted thermal imaging data. Then, the auxiliary signal processor 13 determines a user edge within the determined analysis range. Then, the auxiliary signal processor 13 processes the thermal imaging data to have a closed curve shape from the determined edge using a preset threshold, thereby obtaining an image with a closed curve shape.

[0097] Thereafter, the auxiliary signal processing unit 13 may further perform image labeling on the closed curve image to form a labeled image, and then the auxiliary signal processing unit 13 may spectrogram the labeled image to form a spectrogram.

[0098] 9 is a diagram showing a process of acquiring information on user movement by the auxiliary signal processor 13. 4a is thermal imaging data acquired by the thermal image sensor, 4b is thermal imaging data with the analysis range determined, 4c is an edge of the user, 4d is a closed curve image of the user, 4e is a labeled image of the user, and 4f is a spectrogram of the labeled image of the user.

[0099] The body contour information acquired by the auxiliary signal processing unit 13 can be used in the Doppler signal analysis unit 15 described later. As an example, when the Doppler signal analysis unit 15 acquires non-periodic spectral energy using acquisition time information for a turning over motion represented by non-periodic spectral energy, the body contour information can be used as auxiliary information for determining what biological activity information the energy represents.

[0100] The Doppler signal analysis unit 15 is configured to analyze the Doppler signal acquired by the Doppler signal acquisition unit 11 and acquire spectral energy at a pre-set period. The Doppler signal analysis unit 15 applies a pre-set function to the Doppler signal to acquire spectral energy, and analyzes the acquired spectral energy according to a pre-set criterion to acquire respiratory spectral energy and cardiac spectral energy, which are spectral energies for the respiratory Doppler signal and cardiac Doppler signal.

[0101] In one embodiment of the present invention, the Doppler signal analysis unit 15 may use a fast Fourier transform (FFT) as a pre-set function used to obtain spectral energy from the Doppler signal at a pre-set period. The Fourier transform is a well-known function that means a transformation that resolves a signal sampled in time or space into time frequency or spatial frequency components. By using this, the Doppler signal analysis unit 15 can convert the Doppler signal obtained at a pre-set period into spectral energy of a specific frequency component.

[0102] When the spectral energy of a specific frequency component is acquired, the Doppler signal analysis unit 15 according to an embodiment of the present invention is configured to check whether the acquired spectral energy of the specific frequency component is a periodically occurring spectral energy. The Doppler signal analysis unit 15 may be configured to continuously acquire the spectral energy of a Doppler signal acquired at a previously set period, and determine whether the acquired continuous spectral energy has periodicity. Here, if the spectral energy has periodicity, the spectral energy may be determined to be a spectral energy for respiration or heart rate, and if the spectral energy does not have periodicity, the spectral energy may be determined to not be a spectral energy for respiration or heart rate.

[0103] The Doppler signal analysis unit 15 can be configured to check whether the non-periodic spectral energy has energy greater than a preset value, and if the non-periodic spectral energy has energy greater than a preset value, determine the spectral energy as the spectral energy for the Doppler frequency generated by the user's body movement, or determine the spectral energy as the spectral energy for the Doppler frequency generated by the user's snoring if the non-periodic spectral energy has energy equal to or less than a preset value.

[0104] In addition, the Doppler signal analysis unit 15 according to an embodiment of the present invention may analyze the periodic spectral energy using a pre-set judgment algorithm, and as a result, may separately obtain the respiration spectral energy and the heart rate spectral energy.

[0105] Furthermore, the Doppler signal analysis unit 15 according to an embodiment of the present invention can ascertain the type of non-periodic spectral energy by using the auxiliary signal acquired by the above-mentioned auxiliary signal processing unit 13. Here, the Doppler signal analysis unit 15 can be configured to acquire non-periodic spectral energy corresponding to the user's turning over (body movement) by using thermal image information, for example, and determine the acquired non-periodic spectral energy as a noise signal to remove it from data for analyzing a sleep period, which will be described later.

[0106] In addition, as another embodiment, if a pre-set magnitude of energy capable of distinguishing between body movement and snoring is not defined using non-periodic spectral energy, the Doppler signal unit 15 may define a pre-set magnitude of energy capable of distinguishing between both movements using an auxiliary signal acquired using a thermal image sensor.

[0107] According to an embodiment of the present invention, when the Doppler signal analysis unit 15 continuously acquires the spectral energy of a specific frequency component and determines whether it is a periodic occurrence, the sleep segment definition unit 17 is configured to define a sleep state for each segment in the entire sleep segment using the analysis result.

[0108] The total sleep section may include at least one of a sleep entry section, a deep sleep section, a tossing section, an apnea section, and a snoring section. The sleep entry section refers to a section where a user enters sleep, and is a section that appears statistically in almost all users, and where it is possible to determine whether or not a user has entered sleep. The deep sleep section refers to a sleep section where the user's physical activity is stably maintained, and the tossing section refers to a sleep section where there is movement in the user's physical activity. The apnea section refers to a sleep section where no respiratory activity occurs in the user's physical activity, and the snoring section refers to a sleep section where snoring occurs in the user's physical activity.

[0109] The higher the ratio of deep sleep intervals and the lower the ratio of remaining intervals in a sleep interval, the higher the quality of sleep. Therefore, if the definition of each sleep interval can be performed according to the present invention, not only can information on the sleep quality of the user be obtained, but also a prescription for sleep quality corresponding to each user can be provided later using the information.

[0110] The sleep segment definition unit 17 is configured to define a sleep state for each segment in the entire sleep segment using the analysis result of the Doppler signal analysis unit 15. As described above, each segment in the entire sleep segment includes at least one of a sleep entry segment, a deep sleep segment, a tossing and turning segment, an apnea segment, and a snoring segment. The sleep segment definition unit 17 obtains the analysis result from the Doppler signal analysis unit 15 and confirms which sleep segment the analysis result represents using a state determination criterion that has already been set.

[0111] The previously set state determination criterion may be a previously set value using the ratio of the respiration spectrum energy and the heart rate spectrum energy acquired by the Doppler signal analysis unit 15. As described above, since the magnitude of the physical activity generated by the respiration movement is generally greater than the magnitude of the physical activity generated by the heartbeat movement, the magnitude of the respiration spectrum energy appears greater than the magnitude of the heart rate spectrum energy in the stable sleep section. Also, since high frequency vibrations are captured in the snoring section, additional spectra other than the respiration spectrum energy and the heart rate spectrum energy may appear. Also, since no respiration movement occurs or occurs only weakly in the apnea section, the magnitude of the respiration spectrum energy may be reduced by the magnitude of the heart rate spectrum energy.

[0112] The sleep segment definition unit 17 of one embodiment of the present invention can define a segment in which the spectral ratio, which is the ratio of respiratory spectrum energy to heart rate spectrum energy, is 5:5 or more, as a deep sleep segment.

[0113] In addition, the sleep section definition unit 17 can define a section in which the non-periodic spectral energy is acquired at a level equal to or less than a preset level among sections in which the spectral ratio is less than 5:5 as a snoring section, and can define the section as an apnea section if the non-periodic spectral energy is not acquired for a preset time period.

[0114] In another embodiment of the present invention, the sleep segment definition unit 17 may define a sleep segment using a sleep entry segment other than the spectrum ratio. Since tossing and turning, apnea, and snoring do not generally occur in the sleep entry segment, similar to the deep sleep segment, the sleep segment definition unit 17 of the present invention may be configured to first obtain the sleep entry segment and determine the deep sleep segment based on the obtained sleep entry segment.

[0115] Meanwhile, a flow chart of a non-contact sleep monitoring method according to an embodiment of the present invention is shown in Fig. 7. For convenience of explanation, the non-contact sleep monitoring method of the present invention will be described below as being performed using the system of Fig. 6, but the present invention is not necessarily limited thereto.

[0116] The non-contact sleep monitoring method 10 according to an embodiment of the present invention may be configured to be installed in a space where a user lives and sleeps, such as a room, and to detect sleep when the user goes to sleep, obtain a Doppler signal for biological activity, and define the user's sleep state by analyzing the obtained Doppler signal. For this purpose, the present invention may use a biological activity measuring device using a radar that has already been installed in the above-mentioned space. As shown in FIG. 7, the non-contact sleep monitoring method 10 according to an embodiment of the present invention may be configured to include step S11 of obtaining a Doppler signal, step S13 of processing an auxiliary signal, step S15 of analyzing the Doppler signal, and step S17 of defining a sleep period.

[0117] The step S11 of acquiring the Doppler signal is formed so that the Doppler signal acquisition unit 11 acquires a Doppler signal including biological activity information from a radar installed in the user's sleep space. The Doppler signal is formed to include a respiratory Doppler signal that is information on the user's breathing and a heartbeat Doppler signal that is information on the user's heartbeat. The Doppler signal may be defined as a signal including a Doppler frequency generated by the user's movement when the radar transmits radio waves to the user and the transmitted radio waves return. In one embodiment of the present invention, the respiratory Doppler signal and the heartbeat Doppler signal may be Doppler signals for changes in the movement of the body organs that move when breathing and the body organs that move when the heart beats.

[0118] The step S13 of processing the auxiliary signal is configured to acquire the auxiliary signal using an auxiliary biological activity information acquisition device further provided in the user's sleep space, and to analyze the acquired auxiliary signal to acquire information on the user's body movements (turning over in sleep) other than the breathing or heartbeat movements. For this purpose, the step S13 of processing the auxiliary signal may acquire a measurement result from a thermal image sensor, which is the auxiliary biological activity information acquisition device. The thermal image sensor may be provided to measure the user's body movements.

[0119] In step S13 of processing the auxiliary signal, the thermal image sensor can be used to continuously obtain contour information of the user's body. When the thermal image sensor is used to continuously obtain contour information of the user's body, changes in the contour information caused by movements such as the user turning over in bed can be obtained.

[0120] The body contour information obtained in step S13 of processing the auxiliary signal can be used in step S15 of analyzing the Doppler signal, which will be described later. As an example, when non-periodic spectral energy is obtained in step S15 of analyzing the Doppler signal using acquisition time information for turning over or snoring, which is represented by non-periodic spectral energy, the body contour information can be used as auxiliary information for determining what biological activity information the energy represents.

[0121] The step S15 of analyzing the Doppler signal is formed to analyze the Doppler signal acquired by the step S11 of acquiring the Doppler signal in a Doppler signal analyzer to acquire spectral energy at a pre-set period. The step S15 of analyzing the Doppler signal may acquire spectral energy by applying a pre-set function to the Doppler signal, and analyze the acquired spectral energy according to a pre-set criterion to acquire respiratory spectral energy and cardiac spectral energy, which are spectral energies for the respiratory Doppler signal and cardiac Doppler signal.

[0122] In one embodiment of the present invention, the step S15 of analyzing the Doppler signal can utilize a fast Fourier transform (FFT) as a pre-defined function used to obtain spectral energy from the Doppler signal at a pre-defined period. The Fourier transform is a well-known function that means a transformation that decomposes a signal sampled in time or space into time frequency or spatial frequency components. By utilizing this, the step S15 of analyzing the Doppler signal can convert the Doppler signal obtained at a pre-defined period into spectral energy of a specific frequency component.

[0123] When the spectral energy of a specific frequency component is acquired, step S15 of analyzing the Doppler signal according to an embodiment of the present invention is configured to check whether the acquired spectral energy of the specific frequency component is a periodically occurring spectral energy. Step S15 of analyzing the Doppler signal may be configured to continuously acquire the spectral energy of the Doppler signal acquired at a previously set period, and determine whether the acquired continuous spectral energy has periodicity. Here, if the spectral energy has periodicity, it may be determined that the spectral energy is a spectral energy for respiration or heart rate, and if the spectral energy does not have periodicity, it may be determined that the spectral energy is not a spectral energy for respiration or heart rate.

[0124] The step S15 of analyzing the Doppler signal can be configured to check whether the non-periodic spectral energy has energy greater than a preset value, and if the non-periodic spectral energy has energy greater than a preset value, the non-periodic spectral energy can be determined to be a spectral energy for a Doppler frequency generated by the user's body movement, or if the non-periodic spectral energy has energy equal to or less than a preset value, the non-periodic spectral energy can be determined to be a spectral energy for a Doppler frequency generated by the user's snoring.

[0125] In addition, in step S15 of analyzing the Doppler signal according to an embodiment of the present invention, a pre-set judgment algorithm is used to analyze the periodic spectral energy, and as a result, breathing spectral energy and heart rate spectral energy can be obtained separately.

[0126] Also, in step S15 of analyzing the Doppler signal according to an embodiment of the present invention, the type of non-periodic spectral energy can be identified using the auxiliary signal acquired in the above-mentioned auxiliary signal processing step S13. Here, step S15 of analyzing the Doppler signal can be configured to acquire non-periodic spectral energy for the user's turning over (body movement) using thermal image information, for example, and determine the acquired non-periodic spectral energy as a noise signal to remove it from data for sleep period analysis, which will be described later.

[0127] In addition, in another embodiment, step S15 of analyzing the Doppler signal can be configured to define a pre-set magnitude of energy capable of distinguishing between body movement and snoring using an auxiliary signal acquired using a thermal image sensor if a pre-set magnitude of energy capable of distinguishing between both movements is not defined using non-periodic spectral energy.

[0128] In one embodiment of the present invention, in step S15 of analyzing the Doppler signal, the spectral energy of a specific frequency component is continuously obtained, and once it is determined whether or not it is a periodic occurrence, step S17 of defining a sleep section is formed to define a sleep state for each section in the entire sleep section using the analysis results.

[0129] The total sleep section may include at least one of a sleep entry section, a deep sleep section, a tossing section, an apnea section, and a snoring section. The sleep entry section refers to a section where a user enters sleep, and is a section that appears statistically in almost all users, and where it is possible to determine whether or not a user has entered sleep. The deep sleep section refers to a sleep section where the user's physical activity is stably maintained, and the tossing section refers to a sleep section where there is movement in the user's physical activity. The apnea section refers to a sleep section where no respiratory activity occurs in the user's physical activity, and the snoring section refers to a sleep section where snoring occurs in the user's physical activity.

[0130] The higher the ratio of deep sleep segments to the rest of the sleep segment, the higher the quality of sleep. Therefore, if the definition of each sleep segment can be performed according to the present invention, it is possible to obtain information on the sleep quality of the user, and to later provide a prescription for sleep quality for each user using the information.

[0131] The step S17 of defining a sleep section is formed by using the analysis result of the step S15 of analyzing the Doppler signal to define a sleep state for each section in the entire sleep section in a sleep section definition unit. As described above, each section in the entire sleep section includes at least one of a sleep entry section, a deep sleep section, a tossing section, an apnea section, and a snore section. The step S17 of defining a sleep section obtains the analysis result from the step S15 of analyzing the Doppler signal, and checks what kind of sleep section the analysis result represents using a state judgment criterion that has already been set.

[0132] The previously set state determination criterion may be a previously set value using the ratio of the respiration spectrum energy and the heart rate spectrum energy acquired in step S15 of analyzing the Doppler signal. As described above, since the magnitude of the physical activity generated by the respiration movement is generally greater than the magnitude of the physical activity generated by the heartbeat movement, the magnitude of the respiration spectrum energy appears greater than the magnitude of the heart rate spectrum energy in the stable sleep section. Also, in the snoring section, high frequency vibrations are captured, and other spectra other than the respiration spectrum energy and the heart rate spectrum energy may appear. Also, in the apnea section, respiration movement does not occur or occurs weakly, so the magnitude of the respiration spectrum energy may be reduced by the magnitude of the heart rate spectrum energy.

[0133] In step S17 of defining a sleep section according to an embodiment of the present invention, a section in which the ratio of the spectrum between the respiration spectrum energy and the heart rate spectrum energy is 5:5 or more can be defined as a deep sleep section.

[0134] In addition, in step S17 of defining the sleep section, a section in which the non-periodic spectral energy is acquired at or below a preset level among sections in which the spectral ratio is less than 5:5 can be defined as a snoring section, and if the non-periodic spectral energy is not acquired for a preset time period, the section can be defined as an apnea section.

[0135] In another embodiment of the present invention, step S17 of defining a sleep zone may define a sleep zone using a sleep entry zone other than the spectrum ratio. Since tossing and turning, apnea, and snoring do not generally occur in a sleep entry zone, similar to a deep sleep zone, step S17 of defining a sleep zone of the present invention may be configured to first obtain a sleep entry zone and determine a deep sleep zone based on the obtained sleep entry zone.

[0136] Meanwhile, Fig. 8 is a graph showing the results of an actual conspiracy experiment using an embodiment of the present invention. Referring to Fig. 8, the entire sleep period can be represented by A. Period B is a sleep entry period, which occurs when most users statistically enter sleep as described above, and can be a period in which users actually start to sleep, appearing as an average of 20 to 30 minutes out of a 10 to 40 minute period. C is a deep sleep period, which is a stable sleep period, D is a snoring period, E is a tossing and turning period, and F is an apnea period.

[0137] Section C is a stable deep sleep section, in which the magnitude of the respiration spectrum energy a1 appears to be sufficiently greater than the magnitude of the heart rate spectrum energy b1. In addition, looking at the periodicity of the spectrum energy, it can be seen that the spectrum energy does not include irregular peaks and is repeated at a regular cycle. Therefore, in one embodiment of the present invention, such a sleep section can be defined as a deep sleep section. In this case, as described above, in the present invention, a section in which the ratio of the respiration spectrum energy a1 to the heart rate spectrum energy b1 is 5:5 or more can be defined as a deep sleep section, and in another embodiment, a section that has a periodicity greater than or equal to a previously set similarity based on section B, which is a sleep entry section, and at the same time has an energy less than or equal to a previously set magnitude can be defined as a deep sleep section.

[0138] Section D is a snoring section, where the magnitude of the respiration spectrum energy a2 is greater than the magnitude of the heartbeat spectrum b2, but where snoring spectrum energy c2, which does not appear in other sections, is also measured. It can be seen that section D is a section where high frequency spectrum energy, such as c2, which is a frequency higher than the respiration / heartbeat frequency, is measured. Therefore, in the sleep section definition unit and the step of defining the sleep section of the present invention, section D can be defined as a snoring section where the user is snoring.

[0139] Section F is an apnea section, and the magnitude of the breathing spectrum energy a3 appears to be similar to the magnitude of the heart rate spectrum energy b3. More specifically, the ratio of the breathing spectrum energy a3 to the heart rate spectrum energy b3 appears to be less than 5:5, and unlike section D, this is a section in which the snoring spectrum energy c2 is not measured. This is because the magnitude of the acquired breathing spectrum energy a3 is significantly reduced since the breathing activity of the user's body is weak or nonexistent in the process of acquiring a Doppler signal using a radar. Therefore, when the magnitude of the breathing spectrum energy a3 is compared with the magnitude of the heart rate spectrum energy b3 as in the spectrum analysis result for section F in Fig. 8, if the ratio appears to be less than 5:5 when the magnitude of the breathing spectrum energy a3 is compared with the magnitude of the heart rate spectrum energy b3, the sleep section definition unit and the step of defining the sleep section according to an embodiment of the present invention can define section F as an apnea section.

[0140] Section E is a section where turning over occurs, and where spectral energy having a higher magnitude than the spectral energy detected in sections B, C, D, F, etc. is acquired. This is a biological activity having a much higher energy than the respiratory spectral energy or the heart rate spectral energy, and the present invention is configured to analyze such biological activity as turning over.

[0141] That is, the device configuration and method of Figures 6 and 7 are configured to obtain and analyze Doppler signals of a user's biological activity using a radar, and define a sleep state for each sleep section using respiration spectrum energy and heart rate spectrum energy therein, and experimental results thereof are shown in Figure 8. By using the present invention described in Figures 6 to 8, it is possible to confirm a user's sleep state, and provide content for responding to an emergency situation or improving the user's sleep quality using the confirmed sleep state.

[0142] Although one embodiment of the present invention has been described above, the concept of the present invention is not limited to the embodiment presented in this specification, and a person skilled in the art who understands the concept of the present invention can easily propose other embodiments by adding, changing, deleting, or adding components within the scope of the same concept, which can also be said to fall within the scope of the concept of the present invention. [Industrial Applicability]

[0143] The non-face-to-face, non-contact fall monitoring system and non-contact sleep monitoring system and method of the present invention are characterized in that they protect the privacy of the user while increasing the reliability of fall detection, can detect the user's fall in real time without the user having to wear a separate device in a non-face-to-face, non-contact type that is not a wearable type, and when a fall is determined, judge the degree of danger and notify the guardian or medical staff to prevent the danger of the fall in advance or to take measures, can process the thermal image to be a closed curve, a labeling image, and a spectrogram during fall monitoring and sleep monitoring to protect the user's privacy, analyze the user's danger based on the user's biosignal and the processed thermal image, and provide appropriate measures, and can obtain the user's bioactivity information during sleep using a radar, so as not to affect the user's body, and can accurately judge the user's sleep state for each sleep section using the user's breathing and heart rate information without disturbing the user's sleep.

Claims

1. a radar sensor unit that transmits a radar signal toward a user, receives the radar signal reflected from the user, and generates a time-series Doppler radar signal using a Doppler radar algorithm; A thermal image sensor unit that captures an image of the user from above the user and generates thermal image data; a radar signal pattern analysis unit configured to receive the time series Doppler radar signal from the radar sensor unit and analyze a state of a user based on the received time series Doppler radar signal; a thermal image data processing unit that receives the thermal image data generated by the thermal image sensor unit and processes the received thermal image data to analyze the state of the user; A fall determination unit that determines whether or not a fall has occurred based on the user's condition data transmitted from the radar signal pattern analysis unit and the thermal image data processing unit; A non-face-to-face, non-contact fall detection system comprising:

2. The radar signal pattern analysis unit is The non-face-to-face, non-contact fall detection system of claim 1, characterized in that a motion change signal and a biological signal are separated from the time series Doppler radar signal transmitted from the radar sensor unit, and the separated motion change signal is used as a time series motion Doppler radar signal to analyze the user's condition.

3. The radar signal pattern analysis unit is generating first status data indicating that a fall has occurred when a positive / negative change occurs in data having a magnitude of amplitude equal to or greater than a predetermined value within the first hour in the time-series movement Doppler radar signal and then the magnitude of the amplitude becomes zero; The non-face-to-face, non-contact fall detection system of claim 2, characterized in that if a positive / negative change in data having an amplitude greater than or equal to a predetermined value occurs in the time-series movement Doppler radar signal after the first time, second status data is generated, indicating that the user is slowly falling and a fall is suspected.

4. The thermal image data processing unit includes: determining an analysis range based on the transmitted thermal imaging data; determining a user edge within the determined analysis range; The non-face-to-face, non-contact fall detection system of claim 3, further comprising a step of processing the thermal image data so that the thermal image data has the form of a closed curve from the determined edge using a preset threshold value to obtain an image that has been converted into a closed curve.

5. The thermal image data processing unit includes: The non-face-to-face, non-contact fall detection system according to claim 4 , further comprising a step of performing image labeling on the closed curve image to form a labeled image.

6. The non-face-to-face, non-contact fall detection system of claim 5 , wherein the thermal image data processing unit further includes a step of spectrographically converting the labeled image into a spectrogram to form a spectrogram.

7. The thermal image data processing unit includes: determining a state of the user as one of a fourth state to a sixth state based on at least one of the closed curve image, the labeled image, and the spectrogram, and forming fourth state data to sixth state data, respectively; the fourth state data indicates a state in which the user is positioned on an object at a predetermined height from a floor, The non-face-to-face, non-contact fall detection system of any one of claims 3 to 5, characterized in that the fifth status data indicates a state in which the user has fallen from an object located at a predetermined height from the floor, and the sixth status data indicates a state in which the user has fallen from an object located at a predetermined height from the floor.

8. The fall determination unit is The first status data is transmitted from the radar signal pattern analysis unit, and the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit. When the fourth status data is generated before the first status data generation time, it is determined that the user has fallen, and a first signal indicating that the user has fallen is generated. The first status data determined to be the first status is transmitted from the radar signal pattern analysis unit, and the sixth status data, the fifth status data, and the fourth status data are sequentially transmitted from the thermal image data processing unit. When the sixth status data is generated before the first status data generation time, it is determined that the user has not fallen and a second signal indicating that a fall has not occurred is generated. The second status data indicating that a fall is suspected is transmitted from the radar signal pattern analysis unit, and the fourth status data, the fifth status data, and the sixth status data are sequentially transmitted from the thermal image data processing unit. When the fourth status data is generated before the first status data generation time, it is determined that the user has fallen, and a third signal indicating that the user has fallen and the user has fallen slowly is generated. When third status data is transmitted from the radar signal pattern analysis unit and only fourth status data is transmitted from the thermal image data processing unit until a fourth time after the time when the third status data is transmitted, a fourth signal is generated indicating that the user is at high risk of falling; The non-face-to-face, non-contact fall detection system of claim 7, characterized in that when the first status data or the second status data is transmitted from the radar signal pattern analysis unit, the fourth status data, the fifth status data, and the sixth status data are all transmitted sequentially from the thermal image data processing unit, and after a fourth time, the first status data or the second status data is further transmitted from the radar signal pattern analysis unit, and the sixth status data, the fifth status data, and the fourth status data are transmitted sequentially from the thermal image data processing unit, it is determined that the user who fell has returned to his or her original position, and a fifth signal is generated indicating that the user has returned to his or her original position.

9. The non-face-to-face, non-contact fall detection system of claim 8, wherein the radar signal pattern analysis unit analyzes the separated vital sign and generates the sixth signal notifying of a dangerous situation when the vital sign differs from a vital sign pattern for a predetermined period by a predetermined range or more.

10. The non-contact fall and sleep monitoring system further includes a fall risk determination unit, The fall risk determination unit is if the sixth signal is continuously received from the radar signal pattern analysis unit for a fifth time period, the first signal is received from the fall determination unit, and the fourth signal has not been received within the fifth time period, the seventh signal is generated to notify that the user is in a very high danger due to a fall by determining that the user continues to have fallen and that the vital signs are in a poor state, generating an eighth signal indicating that the user continues to fall but that the biological signal of the user is stable when the sixth signal is not received from the radar signal pattern analysis unit for the fifth time period and the fourth signal is not received from the fall determination unit for the fifth time period; The non-face-to-face, non-contact fall detection system described in claim 9, characterized in that when a third signal is transmitted from the fall discrimination unit, a ninth signal is generated to notify that there is a high possibility that the user will fall.

11. The non-face-to-face, non-contact fall monitoring system further includes an alarm unit, The alarm unit includes: When the seventh signal is transmitted from the fall risk determination unit, a first alarm signal is generated to notify that the user is in a very critical state, and the first alarm signal is transmitted to a predetermined guardian and medical staff. When the eighth signal is transmitted from the fall risk determination unit, a second alarm signal is generated to notify a fall state of the user, and the second alarm signal is transmitted to a predetermined guardian or medical staff. The non-face-to-face, non-contact fall detection system of claim 10, characterized in that when the ninth signal is transmitted from the fall risk determination unit, a third alarm signal is generated to notify that the user is on the verge of falling, and transmitted to a predetermined guardian or medical staff.

12. a Doppler signal acquisition unit that acquires a Doppler signal including biological activity information using a radar; an auxiliary signal processing unit that acquires a turning motion as an auxiliary signal using a thermal image sensor; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a preset period, determines whether the spectral energy is acquired periodically, and classifies the non-periodic spectral energy using the auxiliary signal if the spectral energy is determined to be non-periodic; a sleep segment definition unit that defines a sleep state for each segment in the entire sleep segment by using a ratio of a respiratory spectral energy and a heart rate spectral energy among the spectral energies and a combination of the non-periodic spectral energies; A non-contact sleep monitoring system comprising:

13. The non-contact sleep monitoring system of claim 12, wherein the Doppler signals include a respiratory Doppler signal for acquiring information on the user's breathing, and a heart rate Doppler signal for acquiring information on the user's heartbeat.

14. The Doppler signal analysis unit The non-contact sleep monitoring system of claim 13, wherein the spectral energy is obtained by performing a fast Fourier transform on the Doppler signal.

15. The sleep segment definition unit 15. The non-contact sleep monitoring system of claim 14, wherein a section in which the ratio of the respiratory spectral energy to the heart rate spectral energy in the entire sleep section is 5:5 or more and in which the non-periodic spectral energy is not present is defined as a deep sleep section, and a section in which the ratio of the respiratory spectral energy to the heart rate spectral energy in the entire sleep section is less than 5:5 and in which the non-periodic spectral energy is not present is defined as an apnea section.

16. The sleep segment definition unit The non-contact sleep monitoring system of claim 15, wherein a section in which the non-periodic spectral energy exists and appears at a level equal to or greater than a preset level is defined as a tossing section, and a section in the apnea section in which the non-periodic spectral energy appears at a level equal to or less than a preset level is defined as a snoring section.

17. a Doppler signal acquisition unit that acquires a Doppler signal including biological activity information using a radar; an auxiliary signal processing unit that acquires a turning motion as an auxiliary signal using a thermal image sensor; a Doppler signal analysis unit that analyzes the Doppler signal to acquire spectral energy at a preset period, determines whether the spectral energy is acquired periodically, and classifies the spectral energy using the auxiliary signal; a sleep section definition unit for defining a sleep state for the remaining sleep sections using a pre-defined ratio range and aperiodic spectral energy based on an average of the spectral energy of a sleep entry section that satisfies a pre-defined criterion among all sleep sections; A non-contact sleep monitoring system comprising:

18. acquiring a Doppler signal including biological activity information by a Doppler signal acquisition unit using a radar; A step of processing an auxiliary signal by an auxiliary signal processing unit, the auxiliary signal being acquired as an auxiliary signal based on a turning motion using a thermal image sensor; A Doppler signal analysis unit analyzes the Doppler signal to acquire spectral energy at a preset period, determines whether the spectral energy is periodically acquired, and analyzes the Doppler signal by dividing the spectral energy using the auxiliary signal; a sleep segment definition unit for defining a sleep segment by using a ratio of a respiratory spectral energy and a heart rate spectral energy among the spectral energies and a combination of non-periodic spectral energies to define a sleep state for each segment in the entire sleep segment; A non-contact sleep monitoring method comprising:

19. The method of claim 18, wherein the Doppler signal includes a respiratory Doppler signal for acquiring information on the user's respiration, and a cardiac Doppler signal for acquiring information on the user's heartbeat.

20. 20. The method of claim 19, wherein analyzing the Doppler signal comprises performing a Fast Fourier Transform on the Doppler signal to obtain the spectral energy.

21. 21. The non-contact sleep monitoring method of claim 20, wherein the step of defining the sleep section defines a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy in the entire sleep section is 5:5 or more as a deep sleep section, and defines a section in which the non-periodic spectral energy does not exist among sections in which the ratio of the respiratory spectral energy to the heart rate spectral energy is less than 5:5 as an apnea section.

22. 22. The non-contact sleep monitoring method of claim 21, wherein the step of defining the sleep section defines a section in which the non-periodic spectral energy exists and appears at a magnitude equal to or greater than a preset magnitude as a tossing section, and defines a section in the apnea section in which the non-periodic spectral energy appears at a magnitude equal to or less than a preset magnitude as a snoring section.

23. acquiring a Doppler signal including biological activity information using a radar in a Doppler signal acquisition unit; A step of processing an auxiliary signal by an auxiliary signal processing unit, the auxiliary signal being acquired as an auxiliary signal based on a turning motion using a thermal image sensor; A Doppler signal analysis unit analyzes the Doppler signal to acquire spectral energy at a preset period, and determines whether the spectral energy is periodically acquired, and classifies the spectral energy using the auxiliary signal; defining a sleep state for the remaining sleep sections using a pre-defined ratio range and aperiodic spectral energy based on the average of the spectral energy of the sleep entry sections that satisfy a pre-defined criterion among all the sleep sections in a sleep section definition unit; A non-contact sleep monitoring method comprising:

Images