Emergency prediction and emergency notification system through integration of smart wearable device and robot

Abstract

An emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot is disclosed, and the emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention comprises: a smart wearable device that collects biosignal information or impact detection information of an elderly person and measures GPS location data; a robot connected to the smart wearable device via a wireless communication network to receive information of the elderly person and transmit the received information externally; and a smartphone linked with the smart wearable device and the robot to receive information from the smart wearable device and provide information to the robot. The invention includes an emergency control server that receives and controls emergency situation information from the robot, wherein the smart wearable device transmits the elderly person's biosignal information or shock detection information and real-time GPS coordinate information in real time to the robot, and the robot receives the biosignal information or shock detection information from the smart wearable device and determines whether the elderly person is in an emergency situation, and if it is determined to be an emergency situation, transmits an alarm regarding the elderly person's emergency situation to the emergency control server, and at the same time transmits real-time GPS location tracking information that is updated to track the elderly person's movement path in real time to the emergency control server.

Description

Technology Field

[0001] The present invention relates to an emergency prediction and emergency notification system, which is a technology for safety management of elderly people living alone in an aging society, that combines biometric data collected from smart wearable devices with voice and video analysis functions of a robot to predict emergency situations in advance and provides a rapid alarm through multiple cross-verification when an actual emergency occurs. Background Technology

[0002] Amid the global trend of population aging, South Korea is entering a super-aged society at an unprecedented speed, with the proportion of the elderly population exceeding 20% ​​by 2025. This rapid change in population structure and the explosive increase in single-person households are causing serious problems such as social isolation and lonely deaths among the elderly living alone.

[0003] To address such problems, various ICT-based welfare technologies have evolved over time.

[0004] First-generation technology is an analog-based passive response system in the form of an emergency bell or pendant, which requires the user to press a physical button directly in an emergency situation to operate.

[0005] The second generation technology is an IoT-based non-contact monitoring method using motion detection sensors (PIR) or smart meters, which estimates an emergency retrospectively if there is no movement for a certain period of time.

[0006] Third-generation technology is AI speakers and social robots, which are currently becoming widespread, and they recognize and respond to speech such as "save me" through voice recognition.

[0007] However, the aforementioned conventional technologies have fatal limitations. Heart disease and cerebrovascular disease, which are major causes of death in elderly people aged 65 or older, are accompanied by a rapid loss of consciousness or physical paralysis immediately after onset, resulting in an 'involuntary state' in which the user cannot operate the device or fire on their own.

[0008] Despite the golden time being 4 minutes for cardiac arrest and 3 hours for stroke, existing systems employ reactive mechanisms that rely on user intervention immediately after an emergency occurs, rendering them useless in actual critical situations where consciousness is lost. Furthermore, while most emergencies occur during nighttime sleep, existing sensors fail to accurately distinguish between normal sleep and immobility caused by loss of consciousness, leading to frequent malfunctions.

[0009] Therefore, in order to solve the problems of the conventional technology described above, there is a growing need for a system capable of identifying emergency occurrences in advance and providing emergency prediction and notification.

[0010] The technology forming the background of the present invention is described in Korean Published Patent Application No. 10-2020-0116241 (October 12, 2020). Prior art literature

[0011] (Patent Document 0001) KR 10-2020-0116241 A The problem to be solved

[0012] The present invention aims to solve the problems of the aforementioned conventional technology. The first objective of the present invention is to establish a proactive emergency safety network that analyzes the vital signs of the elderly and infirm in real time, predicts emergency situations, and automatically performs unconscious reporting (emergency alarms).

[0013] The second problem that the present invention aims to solve is to provide an algorithm that establishes multiple paths and performs mutual cross-verification between a smart wearable device, a smartphone for the elderly, and a robot in order to overcome the limitations of a single device and prevent sensor malfunction.

[0014] In addition, the third task that this institute aims to solve is to provide a diagnostic mechanism that can precisely detect early warning signs of brain diseases, such as stroke, through short-term daily conversations by applying Personalized-Text-To-Speech (P-TTS) technology in reverse to overcome the problem of the massive data requirements of existing AI prediction models.

[0015] The fourth task is to provide a smartphone tethering-based hybrid location tracking system to overcome the spatial constraints of robots fixed within the home and to support uninterrupted emergency rescue even in sparsely populated outdoor areas.

[0016] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0017] As a technical means for achieving the above-mentioned technical problem, an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention comprises: a smart wearable device that collects biosignal information or impact detection information of an elderly person and measures GPS location data; a robot that is connected to the smart wearable device via a wireless communication network to receive information about the elderly person and transmits the received information externally; and a smartphone that is linked with the smart wearable device and the robot, receives information from the smart wearable device, and provides information to the robot. The invention includes an emergency control server that receives and controls emergency situation information from the robot, wherein the smart wearable device transmits the elderly person's biosignal information or shock detection information and real-time GPS coordinate information in real time to the robot, and the robot receives the biosignal information or shock detection information from the smart wearable device and determines whether the elderly person is in an emergency situation, and if it is determined to be an emergency situation, transmits an alarm regarding the elderly person's emergency situation to the emergency control server, and at the same time transmits real-time GPS location tracking information that is updated to track the elderly person's movement path in real time to the emergency control server.

[0018] In addition, the smart wearable device transmits the elderly person's biosignal information or shock detection information and time GPS coordinate information in real time to the smartphone, and the smartphone transmits the biosignal information or shock detection information received from the smart wearable device to the robot in real time, and the robot includes a control unit, and if the control unit determines that the elderly person is in an emergency situation based on the biosignal information or shock detection information directly received from the wearable device, it compares and analyzes the information received via the smartphone with the information directly received from the smart wearable device, and performs a multiple cross-verification algorithm to confirm the presence or absence of consciousness through interaction by inducing a voice response from the elderly person through the robot's speaker, thereby finally determining whether an emergency situation exists.

[0019] In addition, the robot further includes a P-STT model trained based on the unique voice data of the elderly person; and a disease prediction module that predicts a disease by identifying precursor symptoms of the elderly person's disease through comparative analysis of voice data collected during the interaction process with the elderly person with reference data of the personalized voice learning model, and the control unit can determine whether the elderly person is in an emergency situation by further considering the type of disease predicted by the disease prediction module.

[0020] In addition, the robot may further include a disease prediction precision correction unit that corrects the precision of disease prediction by analyzing the correlation between the PPG (photocirculation)-based biomarker measured by the wearable device and the voice data variability through the analysis of the learned P-STT model.

[0021] In addition, the robot may further include an accident type analysis module that performs an accident type analysis algorithm to suppress the false transmission of emergency alarms by combining and analyzing impact data received from the smart wearable device and heart rate data of PPG (photoplethysmosis)-based biomarkers, and distinguishing between a simple mechanical impact and a fainting state due to loss of consciousness.

[0022] In addition, when the communication connection between the smart wearable device and the robot is disconnected, tethering with the smartphone can be performed to continuously transmit the location and biometric data of the elderly person to the emergency control server through the wide-area communication network of the smartphone.

[0023] Additionally, the robot includes an alarm transmitting unit that transmits an alarm regarding an emergency situation of the elderly person to the emergency control server, and a GPS location signal transmitting unit that transmits real-time GPS location tracking information to the emergency control server. The GPS location signal transmitting unit receives a low-power short-range communication signal from the smart wearable device, converts it into a broadband network, and transmits the signal. The GPS location signal transmitting unit may further include a GPS information error correction unit that performs a composite positioning algorithm to correct errors in the GPS information of the elderly person by analyzing surrounding Wi-Fi signals and beacon information in combination when the reception of real-time GPS location tracking information is unstable.

[0024] In addition, when an emergency situation is received by the emergency control server, the emergency control server transmits a rescue request to the terminal of a rescue worker at the emergency center, and the robot can transmit streaming data to the emergency control server by activating the camera module and microphone equipped on the robot so that the rescue worker can check the condition of the elderly person in real-time video and the elderly person can perform voice communication with the rescue worker at the emergency center.

[0025] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0026] According to the solution to the problem of the present invention described above, even in the event of an acute disease that is impossible for the elderly to operate, the system actively detects the crisis through the bio-signals of a smart wearable device and transmits an emergency alarm to an emergency control server, thereby ensuring that the golden time for rescuing the elderly is secured.

[0027] In addition, according to the solution to the problem of the present invention, the reliability of the system can be maximized by fundamentally blocking false alarms through triple data linkage connecting smart wearable devices, smartphones, and robots, multiple cross-verification through voice response confirmation, and accident type analysis combining impact force and PPG data, thereby clearly distinguishing between a simple device drop or a sleeping state and a fainting state.

[0028] Furthermore, according to the solution provided by this institute, by fusing voice analysis—which applies P-TTS (Personalized Text-to-Speech) technology in reverse—with biomarker analysis, it is possible to detect early warning signs of fatal cerebrovascular diseases or cognitive decline and provide emergency alarms considering the type of disease, thereby enabling more proactive preventive measures.

[0029] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist. Brief explanation of the drawing

[0030] FIG. 1 is a schematic diagram showing an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention. FIG. 2 is a control configuration diagram showing the configuration of a robot in more detail in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention. FIGS. 3(a) and (b) are drawings that more specifically illustrate the information transmission configuration of a smart wearable device, a smartphone, and a robot in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention. FIG. 4 is a flowchart specifically illustrating a disease prediction method performed by a P-STT model and a disease prediction module in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention. FIG. 5 is a flowchart specifically illustrating a method for determining the type of an accident by data collection and analysis performed in an accident type analysis module in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention. Specific details for implementing the invention

[0031] Embodiments of the present invention are described below with reference to the attached drawings to enable those skilled in the art to easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0032] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" or "indirectly connected" with other elements interposed between them.

[0033] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0034] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0035] This invention relates to an active emergency prediction and emergency notification system that organically links smart wearable devices and robots to prevent lonely deaths among the elderly living alone in an aging society and to secure the golden time in the event of an emergency.

[0036] FIG. 1 is a schematic diagram showing an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention.

[0037] Referring to FIG. 1 of the present invention, an emergency prediction and emergency notification system (100) through the linkage of a smart wearable device and a robot according to one embodiment of the present invention may include: a smart wearable device (110) that collects biosignal information or shock detection information of an elderly person and measures GPS location data; a robot (120) that is connected to the smart wearable device (110) via a wireless communication network to receive information of the elderly person and transmits the received information to the outside; a smartphone (130) that is linked with the smart wearable device (110) and the robot (120) to receive information from the smart wearable device (110) and provide information to the robot (120); and an emergency control server (140) that receives and controls emergency situation information from the robot (120).

[0038] Additionally, the robot (120) receives the biosignal information or the shock detection information from the smart wearable device (110) and determines whether the elderly person is in an emergency situation. If it is determined to be an emergency situation, it can send an alarm regarding the emergency situation of the elderly person to the emergency control server (140) and, at the same time, send real-time GPS location tracking information that is updated to track the movement path of the elderly person in real time to the emergency control server (140).

[0039] In one embodiment of the present invention, the smart wearable device (110) may be, for example, a smart watch, but is not limited thereto, and may include other types of devices as long as they are wearable devices capable of collecting biosignal information or shock detection information of an elderly person (user). The smart wearable device (110) can monitor the pulse, body temperature, oxygen saturation, and activity level (accelerometer) of the elderly person in real time. By doing so, the condition of the elderly person, such as a sudden drop in heart rate or a fall, can be monitored, and the information can be transmitted to the robot (120).

[0040] In addition, in one embodiment of the present invention, the robot (120) may be a care robot, and, for example, may be a robot having a doll-like shape friendly to the elderly.

[0041] The robot (120) is connected to a smart wearable device (110) via a wireless communication network to receive the biometric signals and location data of the elderly person, and can transmit the received data to an external source (e.g., an emergency control server (140)). The specific configuration of the robot (120) will be examined in more detail through FIG. 2.

[0042] In one embodiment of the present invention, the smartphone (130) may receive information from the smart wearable device (110) and provide information to the robot (120). The smart wearable device (110) and the smartphone (130) may be devices that have been registered in advance by an elderly person (user).

[0043] In addition, when the communication connection between the smart wearable device (110) and the robot (120) is disconnected, tethering between the smartphone (130) and the smart wearable device (110) is performed so that the location and biometric data of the elderly person can be continuously transmitted to the emergency control server (140) through the wide-area communication network of the smartphone (130).

[0044] For example, when the elderly person (user) is at home, the smart wearable device is directly connected to the robot's Wi-Fi router or Bluetooth so that the robot acts as the main control tower. However, when the elderly person goes out and moves out of communication range with the robot, the smart device is immediately tethered to the user's smartphone, and location information can be tracked using the smartphone's wide-area network (LTE / 5G) and GPS.

[0045] Accordingly, even when there is a problem with the communication connection between the robot (120) and the smart wearable device (110) when the elderly person goes out, etc., the location and biometric data of the elderly person can be transmitted to the emergency control server (140) via the smartphone (130), and by providing an emergency situation notification to the emergency control server (140), a significant effect can be achieved in that the emergency alarm and real-time GPS location information can be reliably transmitted to the emergency control server.

[0046] Meanwhile, to maximize the battery efficiency of a smart wearable device, the smart wearable device according to an embodiment of the present invention may be equipped with Adaptive Sampling Rate Control technology, which maintains a low-power mode during normal operation but shortens the data transmission cycle in real time when it is determined that the elderly person is in an emergency situation due to shock detection or abnormal heart rate. Accordingly, the smart wearable device can be kept at low power during normal operation to save battery power, and an effect can be achieved to resolve the problem of the battery running out when the elderly person is in an emergency situation.

[0047] In one embodiment of the present invention, the emergency control server (140) may refer to a server built to receive an emergency situation and transmit it to a rescue worker at an emergency center. The emergency control server (140) may receive an emergency alarm and real-time GPS location tracking information from the robot (120), provide an alarm for emergency measures to a rescue worker at an emergency center, and transmit streaming data to a rescue worker's terminal at the emergency center.

[0048] Specifically, the emergency control server (140) may be connected to a rescue worker's terminal (150) at the emergency center. For example, since there may be multiple rescue workers at the emergency center, there may be multiple rescue worker terminals (150) (151, 152, 153,…). For example, the terminals (151, 152, 153,…) may be mobile phones, but are not limited thereto, and may include other devices that can receive alarms and receive streaming data.

[0049] When an emergency situation is received by the emergency control server (140) from the robot (120) (the robot sends an alarm regarding the emergency situation of the elderly person), the emergency control server (140) can transmit a rescue request to the terminal of a rescue worker at the emergency center. Meanwhile, the robot (120) can transmit streaming data to the emergency control server (140) by activating the camera module and microphone equipped in the robot (120) so that the rescue worker can check the condition of the elderly person in real-time video and the elderly person can communicate with the rescue worker at the emergency center via voice.

[0050] In addition, the robot (120) may be connected not only to the emergency control server (140) but also to the terminal of the guardian of the elderly person, and may provide an emergency alarm to the guardian along with transmitting an emergency alarm to the emergency control server (140), thereby enabling a more effective response to emergency situations.

[0051] Meanwhile, the real-time GPS location tracking information may be a location synchronization link (URL) or a data stream. Here, the URL method provides an intuitive access path to rescue personnel at an emergency center, enabling immediate location identification even in a general-purpose terminal environment, whereas the data stream method has the advantage of ensuring data continuity in situations where the condition of an elderly person wandering outdoors or an elderly person inside a moving ambulance needs to be tracked in real time, and minimizing latency between the control screen and the actual location of the elderly person.

[0052] The robot (120) can receive biosignal information or shock detection information of the elderly person from the smart wearable device (110) and determine whether the elderly person is in an emergency situation.

[0053] Additionally, the smart wearable device (110) can transmit the biosignal information or shock detection information of the elderly person and real-time GPS coordinate information to the smartphone (130), and the smartphone (130) can transmit the biosignal information or shock detection information received from the smart wearable device (110) to the robot (120).

[0054] Due to this configuration, the robot (120) can more accurately determine whether the elderly person is in an emergency situation by comparing the elderly person's biosignal information or shock detection information received from the smart wearable device (110) with the elderly person's biosignal information or shock detection information received from the smartphone (130).

[0055] The specific method for determining emergency situations will be examined in more detail in the explanation of Figure 2.

[0056] According to the embodiment of the present invention as described above, by determining whether an emergency situation exists for an elderly person based on their vital sign information or shock detection information, and if an emergency situation is determined, transmitting an alarm regarding the emergency situation to an emergency control server, and simultaneously transmitting real-time GPS location tracking information that is updated to track the elderly person's movement path in real time to the emergency control server, a significant effect can be achieved in that the emergency situation of the elderly person can be quickly responded to in cases where it is difficult to receive assistance, such as when the elderly person is alone at home.

[0057] FIG. 2 is a control configuration diagram showing the configuration of a robot in more detail in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention.

[0058] Referring to FIG. 2, a robot (120) according to one embodiment of the present invention may include a control unit (121). Specifically, the control unit (121) may play a role in controlling the overall function of the robot (120).

[0059] The control unit (121) can receive biosignal information or shock detection information of an elderly person from a smart wearable device (110) and determine whether the elderly person is in an emergency situation. Additionally, the control unit (121) can receive the relevant information from a smartphone (130) that has received biosignal information or shock detection information of an elderly person from the smart wearable device (110), and if it is determined that the elderly person is in an emergency situation based on the information from the smart wearable device (110), it can determine whether the emergency situation is an emergency by comparing and analyzing the information received via the smartphone (130) with the information received directly from the smart wearable device (110).

[0060] Additionally, the robot (120) may include a camera module (122). The camera module (121) may perform the role of observing the condition of the elderly and recognizing images for transmitting video to an emergency control server.

[0061] Additionally, the camera module (122) can provide image information for verifying the identity of the elderly person. For example, identity can be verified by performing facial recognition of the elderly person (user). Furthermore, when the robot (120) converses with the elderly person (user) through the camera module (122), it can perform facial recognition to authenticate the identity of the device wearer in real time, and can output personalized health guidelines in voice according to the authenticated information.

[0062] Additionally, a robot (120) according to one embodiment of the present invention may include a speaker (123) and a microphone (124). The speaker (123) and the microphone (124) may serve to enable the robot (120) to communicate with the elderly.

[0063] For example, when a wearer (an elderly person) of a smart wearable device (110) looks at the robot (120), the robot (120) can perform facial recognition by inducing a natural conversation, such as “Grandma, please make eye contact with me,” through the speaker (123) to verify the identity. Additionally, the robot (120) can recognize the voice pattern of the elderly person by conversing with the elderly person through the speaker (123) and microphone (124), thereby learning the voice pattern of the elderly person and determining the condition of the elderly person.

[0064] In addition, when the control unit (121) of the robot (120) compares and analyzes information received directly from the smart wearable device (110) with information received via the smartphone (130) to determine whether there is an emergency situation, it can finally determine whether there is an emergency situation by performing a multiple cross-verification algorithm that confirms whether there is consciousness through interaction by inducing a voice response from the elderly person through the speaker (123) of the robot (120).

[0065] Through such a multiple cross-verification algorithm, a significant effect can be achieved in that information errors are minimized and the accuracy of the information is greatly improved by undergoing a step of mutually cross-checking information obtained through different paths.

[0066] In addition, according to an embodiment of the present invention, when an emergency occurs, the camera module (122) and microphone (124) equipped in the robot (120) can be activated so that a rescue worker can check the condition of the elderly person in real-time video, and the elderly person can communicate with the rescue worker at the emergency center through the speaker (23) and microphone (124).

[0067] Referring to FIG. 2, a robot (120) according to one embodiment of the present invention may further include a P-STT model (125) and a disease prediction module (126).

[0068] The P-STT model (125) is also called a P-STT (Personalized-Speech To Text) model that applies the Personalized-Text To Speech (P-TTS) model in reverse, and it may be a model that stores language as text (data) in a robot or wearable device and uses it as prediction data. That is, the robot (120) plays the role of learning the unique voice patterns (pronunciation accuracy, speed, tone, intonation, dialect, etc.) of each individual elderly person through machine learning by having conversations with the elderly person in their daily life, and through this learning, it can play the role of setting the normal range (Baseline) of the elderly person.

[0069] Additionally, the disease prediction module (126) can identify warning signs of disease, such as speech disorders caused by stroke or cognitive decline, by comparing and analyzing speech data (pronunciation accuracy, speech speed, pitch variability, etc.) collected during the interaction with the elderly person with reference data of the P-STT model. For example, if subtle slurred speech, inappropriate word repetition, or significantly slow speed are detected in the speech of the elderly person, these can be identified as warning signs of stroke (cerebral infarction) or dementia. Furthermore, a time series analysis algorithm can be used to distinguish between natural voice changes due to aging and rapid variability caused by disease (dementia, stroke, etc.) to identify warning signs of disease.

[0070] In this way, when the robot (120) detects early symptoms of disease in the elderly person, it can proactively ask questions such as, “Grandma, you seem to be having a little more trouble speaking than usual today. Shall I contact the hospital?” or send a ‘speech pattern abnormality detection’ report to the elderly person’s guardian so that preventive measures can be taken before the disease worsens to a severe state.

[0071] According to an embodiment of the present invention, in order to overcome the problem of the massive data requirements of existing AI prediction models, a diagnostic mechanism can be provided that precisely detects warning signs of brain diseases, such as stroke, through short-term daily conversations alone by utilizing a P-STT model that applies existing personalized speech synthesis technology in reverse.

[0072] The P-STT technology according to the embodiment of the present invention can read text in a natural voice by mimicking a person's voice (timbre, intonation, speech habits, etc.) using only a few minutes or a few seconds of specific person's voice data.

[0073] Furthermore, while conventional personalized text-to-speech (P-TTS) technology aimed to promote emotional healing by restoring the voice of a third party, emergency situation monitoring applying P-STT technology according to the embodiment of the present invention analyzes the accuracy, speed, and wavelength of pronunciation based on the user's (elderly / infirm) normal voice data (first person), thereby enabling early detection of signs of speech disorders associated with conditions such as cerebral infarction and dementia, and allowing for preventive measures.

[0074] This is a reverse approach that transforms third-party application technology into first-person user diagnosis technology, and it can have a significant effect in that it enables another path to diagnose emergency situations independently of emergency situation judgment through data transmission.

[0075] In addition, the control unit (121) of the robot (120) can determine whether there is an emergency situation for the elderly by additionally considering the type of disease predicted by the disease prediction module (126).

[0076] Additionally, referring to FIG. 2, a robot (120) according to one embodiment of the present invention may include an accident type analysis module (127).

[0077] The smart wearable device (110) can transmit impact data using an acceleration sensor to the robot (120), and the accident type analysis module (127) of the robot (120) can collect impact data received from the smart wearable device (110) in the impact data collection unit (127A). In addition, the heart rate data collection unit (127B) of the accident type analysis module (127) can receive heart rate data measured by the smart wearable device (110) and collect heart rate data of the elderly.

[0078] Based on the collected data, the accident type analysis module (127) can perform an accident type analysis algorithm that combines and analyzes the received impact data and the collected heart rate data of PPG (photoplethysmosis)-based biomarkers, and distinguishes between a simple mechanical shock and a fainting state due to loss of consciousness, thereby suppressing the mistransmission of an emergency alarm.

[0079] Due to the function of the accident type analysis module (127) as described above, it is possible to determine whether an emergency situation exists more accurately and suppress the transmission of an emergency situation alarm, thereby enabling effective detection of emergency situations and management of emergency request alarms.

[0080] Additionally, referring to FIG. 2, a robot (120) according to one embodiment of the present invention may include a signal transmitting unit (128). Additionally, the signal transmitting unit (128) may include an alarm transmitting unit (128A) and a GPS position signal transmitting unit (128B).

[0081] Specifically, when the control unit (121) determines that the elderly person is in an emergency situation, the alarm transmission unit (128A) can transmit an emergency situation notification to the emergency control server. Additionally, when the control unit (121) determines that the elderly person is in an emergency situation, the GPS location signal transmission unit (128B) can transmit real-time GPS location tracking information, which is updated to track the elderly person's movement path in real time, to the emergency control server.

[0082] In addition, the GPS location signal transmitting unit (128B) can receive a low-power short-range communication signal from the smart wearable device (110), convert it into a broadband network, and transmit the signal.

[0083] Meanwhile, the GPS location signal transmitting unit (128B) may further include a GPS information error correction unit (128B'). The GPS information error correction unit (128B') can perform a composite positioning algorithm to correct errors in the GPS information of the elderly person by complexly analyzing surrounding Wi-Fi signals and beacon information when the reception of real-time GPS location tracking information is unstable.

[0084] Due to the function of the GPS information error correction unit (128B'), even when the reception of real-time GPS location tracking information is unstable, the GPS location information can be measured more accurately and the location information can be transmitted, thereby enabling a significant effect of being able to stably transmit accurate GPS location information in the event of an emergency.

[0085] Meanwhile, according to an embodiment of the present invention, when an emergency situation is received by an emergency control server, the emergency control server can transmit a rescue request to a rescue worker's terminal at the emergency center.

[0086] In this case, the robot (120) can activate the camera module (122) and microphone (124) equipped on the robot to transmit streaming data to the emergency control server so that a rescue worker can check the condition of the elderly person in real-time video and the elderly person can communicate with the rescue worker at the emergency center via voice.

[0087] Additionally, referring to FIG. 2, a robot (120) according to one embodiment of the present invention may further include a charging dock (129).

[0088] For the elderly, charging the smart wearable device (110) can be cumbersome and there is a high possibility of losing the device. Therefore, a magnetic charging dock dedicated to the smart band is installed on the wrist area or the top of the main body of the robot (120), so that when the elderly person sleeps or takes off the band, they can say to the robot, “Please hold the watch,” and charge the smart wearable device (110) by placing it on the dock.

[0089] In addition, the robot can monitor the remaining battery level of a smart wearable device and output a voice guide instructing charging when the level is below a threshold, and send a loss prevention alert to the elderly person's smartphone when the device is removed.

[0090] Meanwhile, in order to prevent the elderly from giving up on wearing the device due to the inconvenience of having to plug the wearable device into a charging plug every time, the charging dock may be in the form of a magnetic charging dock that can attach and detach a smart wearable device using magnetic force on the hardware body (upper part or wrist part) of the robot.

[0091] In this way, not only can the hassle of charging smart wearable devices be reduced, but the risk of loss can also be fundamentally prevented by providing a fixed storage location for smart wearable devices. Additionally, the robot can monitor the charging status of the device and induce charging by providing a voice guidance saying, “Please charge the device now.”

[0092] With the configuration of the robot (120) of the present invention as described above, a significant effect can be achieved, which can improve the monitoring effect for the elderly and infirm, stably collect the elderly and infirm's biometric information and shock detection information, accurately determine emergency situations, and effectively transmit alarms to an emergency control server and GPS location information.

[0093] FIGS. 3(a) and (b) are drawings that more specifically illustrate the information transmission configuration of a smart wearable device, a smartphone, and a robot in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention.

[0094] FIG. 3(a) is a diagram showing the method of transmitting information (biometric information and shock detection information) of an elderly person (user) when the communication connection between the smart wearable device (110) and the robot (120) is good.

[0095] Specifically, referring to FIG. 3(a), biometric information and shock detection information from the smart wearable device (110) can be transmitted to the robot (120) and also to the smartphone (130) of the elderly person. The information transmitted to the smartphone (130) can then be transmitted back to the robot (120) so that the robot (120) can compare the information transmitted directly from the smart wearable device (110) with the information transmitted from the smartphone (130) when determining whether there is an emergency situation.

[0096] For example, in the case where it is determined that an emergency situation is present based on information from a smart wearable device (110), it may be re-determined whether it is an emergency situation through information transmitted from a smartphone (130). In this case, the final determination of whether it is an emergency situation may be made by additionally performing a process of checking whether the elderly person is conscious through interaction between the robot (120) and the elderly person.

[0097] If it is determined that an emergency situation is occurring based on the above judgment, the robot (120) can send an alarm about the emergency situation to the emergency control center and at the same time send real-time GPS location tracking information (location information) of the elderly person to the emergency control center.

[0098] Meanwhile, in an embodiment of the present invention, all sensitive data may be transmitted via end-to-end encryption (E2EE) technology after local encryption, and identification information may be anonymized and stored in the cloud.

[0099] FIG. 3(b) is a diagram showing how information (biometric information and shock detection information) of an elderly person (user) is transmitted when the communication connection between the smart wearable device (110) and the robot (120) is not good.

[0100] Specifically, referring to FIG. 3(b), when the communication connection between the smart wearable device (110) and the robot (120) is disconnected, the smart wearable device (110) can tether with the smartphone (130) to transmit information to the smartphone (130), and the smartphone (130) can make an emergency situation determination based on the information received from the smart wearable device (110).

[0101] When it is determined that an elderly person is in an emergency situation, the smartphone (130) can send an emergency situation notification to an emergency control server via a wide-area communication network, and can also continuously send the real-time location and biometric data of the elderly person to the emergency control server.

[0102] Through such an embodiment, a significant effect can be achieved in that it enables the elderly and infirm to respond to sudden emergency situations even when they are outdoors, and allows for rapid rescue and emergency treatment.

[0103] Meanwhile, in another embodiment of the present invention, when the external network between the robot (120) and the emergency control center (140) is disconnected, the robot (120) may be configured to perform a local fail-safe mode that operates an emergency alert on its own. In this way, even in a situation where an emergency alert cannot be provided to the emergency control center, the robot (120) can directly generate an alarm to provide an emergency alert to people in the vicinity.

[0104] FIG. 4 is a flowchart specifically illustrating a disease prediction method performed by a P-STT model and a disease prediction module in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention.

[0105] Referring to FIG. 4, a disease prediction method (S110) in an emergency prediction and emergency notification system according to an embodiment of the present invention may include a step (S111) of performing learning based on unique voice data of an elderly person. This learning may involve learning the unique voice patterns (pronunciation accuracy, speed, tone, intonation, dialect, etc.) of each individual elderly person using machine learning, and the data learned in this way may be stored in a personalized voice learning model.

[0106] Next, a step (S112) of comparing the response data collected from interactions with the elderly with reference data stored in a personalized speech learning model may be performed.

[0107] Through such comparison, warning signs of a disease can be identified (S113). According to an embodiment of the present invention, by learning an individual's unique voice pattern and identifying warning signs of a disease based thereon, a significant effect can be achieved in effectively identifying warning signs of a disease by considering the individual's tendencies.

[0108] Next, a step (S114) of analyzing the correlation between voice data variability through the analysis of PPG (photocirculation)-based biomarkers measured by a wearable device and a learned P-STT model may be performed.

[0109] In this way, the accuracy of disease prediction can be corrected by analyzing the correlation between PPG (photocirculation)-based biomarkers and negative data variability (S115).

[0110] In other words, according to an embodiment of the present invention, in addition to comparing the unique voice patterns of individual elderly people with a voice learning model, a significant effect can be achieved in that disease prediction for the elderly can be performed more precisely by further analyzing the correlation between PPG (photoplethysmosis)-based biomarkers and voice data variability.

[0111] FIG. 5 is a flowchart specifically illustrating a method for determining the type of an accident by data collection and analysis performed in an accident type analysis module in an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to one embodiment of the present invention.

[0112] Referring to FIG. 5, the method for determining the type of accident (S120) in an emergency prediction and emergency notification system according to an embodiment of the present invention may include a step of collecting impact data of an acceleration sensor (S121) and a step of collecting heart rate fluctuation data (S122).

[0113] Specifically, in the event of a sudden change in the movement of an elderly person (user), such as a fall, impact data and heart rate data of the elderly person can be measured through the accelerometer of a smart wearable device. This information is transmitted to a robot, and impact data and heart rate variability data can be collected by the data collection unit of the robot.

[0114] Afterwards, by combining and analyzing the impact data and heart rate fluctuation data (S123), it can be determined whether the device malfunctioned, whether the elderly person was asleep, or whether the elderly person fainted (S124).

[0115] In addition, this judgment may be made by adding image data collected through the robot's camera. That is, by considering the condition of the elderly person recognized by the camera in addition to impact and heart rate variability data, the fainting state (emergency situation) of the elderly person can be determined.

[0116] If the elderly person is in a state of unconsciousness due to loss of consciousness (S124 'YES'), the robot may determine that it is an emergency situation and send out an emergency alarm (S125), and if it is not (S125 'NO'), it may determine that it is not an emergency situation and suppress the emergency alarm (S126).

[0117] In this way, it is possible to prevent the erroneous transmission of emergency alarms when they are unnecessary, and accordingly, a significant effect can be achieved that greatly improves the reliability of the emergency alarm system.

[0118] Meanwhile, the disclosed embodiment may also be implemented as a computer-readable program, etc., on a computer-readable recording medium for execution on a computer. The computer-readable recording medium may be a computer-readable non-transient recording medium, such as a data storage device capable of storing data that can be read by a processor / microprocessor.

[0119] Examples of computer-readable recording media include hard disk drives (HDDs), solid-state drives (SSDs), silicon disk drives (SDDs), read-only memory (ROM), CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices.

[0120] According to the embodiments of the present invention as described above, an emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot can be provided, which can maximize the effect of emergency rescue by actively detecting a crisis through the biosignal of a smart wearable device and transmitting an emergency alarm to an emergency control server.

[0121] In addition, according to the emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot according to the embodiment of the present invention, a significant effect can be achieved in improving the reliability of the system by accurately determining the emergency situation through data linkage of the smart wearable device, the robot, and the smartphone, and through multiple cross-verification including confirmation of the voice response of the elderly person by the robot.

[0122] In addition, according to an embodiment of the present invention, a significant effect can be achieved in fundamentally blocking false alarms by clearly distinguishing between a simple device drop or a sleep state and a fainting state through accident type analysis combining impact data collected from a smart wearable device and PPG data.

[0123] Furthermore, according to an embodiment of the present invention, by fusing voice analysis based on P-STT technology, which applies personalized voice synthesis technology in reverse, with biomarker analysis, it is possible to detect early warning signs of fatal cerebrovascular disease or cognitive decline and provide an emergency alarm considering the type of disease, thereby enabling the effect of taking preemptive preventive measures.

[0124] In addition, it is possible to go beyond simple post-incident response to enable proactive disease prediction, and through triple verification between smart wearable devices, robots, and smartphones, it can reduce the fatigue of control centers and dramatically improve the reliability of rescue in actual emergency situations.

[0125] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0126] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0127] 100: Emergency prediction and emergency notification system through the integration of smart wearable devices and robots 110: Smart wearable devices 120: Robot 121: Control unit 122: Camera Module 123: Speaker 124: Microphone 125: P-STT model 126: Disease Prediction Module 127: Accident Type Analysis Module 128: GPS location signal transmitter 129: Charging dock holder 130: Smartphone 140: Emergency Control Server 150 (151, 152, 153): Rescue worker's terminal

Claims

Claim 1 An emergency prediction and emergency notification system through the linkage of a smart wearable device and a robot, comprising: a smart wearable device that collects biosignal information or impact detection information of an elderly person and measures GPS location data; a robot connected to the smart wearable device via a wireless communication network to receive information about the elderly person and transmit the received information externally; and a smartphone linked with the smart wearable device and the robot to receive information from the smart wearable device and provide information to the robot.and includes an emergency control server that receives and controls emergency situation information from the robot, wherein the smart wearable device transmits the elderly person's biosignal information or impact detection information and real-time GPS coordinate information in real time to the robot, and the robot receives the biosignal information or impact detection information from the smart wearable device and determines whether the elderly person is in an emergency situation, and if it is determined to be an emergency situation, sends an alarm regarding the elderly person's emergency situation to the emergency control server and simultaneously transmits real-time GPS location tracking information, which is updated to track the elderly person's movement path in real time, to the emergency control server, and the smart wearable device transmits the elderly person's biosignal information or impact detection information and real-time GPS coordinate information in real time to the smartphone, and the smartphone transmits the biosignal information or impact detection information received from the smart wearable device to the robot in real time, and the robot includes a control unit, and if the control unit determines that the elderly person is in an emergency situation based on the biosignal information or impact detection information directly received from the wearable device, the information via the smartphone and An emergency prediction and emergency notification system characterized by comparing and analyzing information directly received from the smart wearable device, and performing a multiple cross-verification algorithm to confirm the presence or absence of consciousness through interaction by inducing a voice response from the elderly person through the robot's speaker, thereby finally determining whether an emergency situation exists. Claim 2 delete Claim 3 An emergency prediction and emergency notification system according to claim 1, wherein the robot further comprises: a P-STT model (personalized speech learning model) learned based on the unique voice data of the elderly person; and a disease prediction module that predicts a disease by comparing and analyzing voice data collected during the interaction process with the elderly person with reference data of the P-STT model to identify precursor symptoms of the elderly person's disease, and wherein the control unit determines whether the elderly person is in an emergency situation by further considering the type of disease predicted by the disease prediction module. Claim 4 An emergency prediction and emergency notification system according to claim 3, wherein the robot further comprises a disease prediction precision correction unit that corrects the precision of disease prediction by analyzing the correlation between voice data variability through the analysis of PPG (photocircuitry flow)-based biomarkers measured by the wearable device and the learned P-STT model. Claim 5 An emergency prediction and emergency notification system according to claim 4, wherein the robot further comprises an accident type analysis module that performs an accident type analysis algorithm to suppress false transmission of emergency alarms by combining and analyzing impact data received from the smart wearable device and heart rate data of PPG (photoplethysmosis)-based biomarkers, and distinguishing between a simple mechanical shock and a fainting state due to loss of consciousness. Claim 6 An emergency prediction and emergency notification system characterized by, in any one of claims 1 and 3 to 5, performing tethering with the smartphone to continuously transmit the location and biometric data of the elderly person to the emergency control server through the wide-area communication network of the smartphone when the communication connection between the smart wearable device and the robot is disconnected. Claim 7 An emergency prediction and emergency notification system according to any one of claims 1 and 3 to 5, wherein the robot comprises an alarm transmitting unit that transmits an alarm regarding an emergency situation of the elderly person to the emergency control server, and a GPS location signal transmitting unit that transmits real-time GPS location tracking information to the emergency control server, wherein the GPS location signal transmitting unit receives a low-power short-range communication signal of the smart wearable device, converts it into a broadband network, and transmits the signal, and the GPS location signal transmitting unit further comprises a GPS information error correction unit that performs a composite positioning algorithm to correct errors in the GPS information of the elderly person by complexly analyzing surrounding Wi-Fi signals and Beacon information when the reception of real-time GPS location tracking information is unstable. Claim 8 An emergency prediction and emergency notification system according to any one of claims 1 and 3 to 5, wherein when an emergency situation is received by the emergency control server, the emergency control server transmits a rescue request to a terminal of a rescue worker at an emergency center, and the robot transmits streaming data to the emergency control server so that the rescue worker can check the condition of the elderly person in real-time video and the elderly person can perform voice communication with the rescue worker at the emergency center by activating the camera module and microphone equipped in the robot.

Images