Intelligent senior care system based on voice analysis

The voice analysis-based system predicts wandering risk and ensures uninterrupted location tracking and accurate emergency judgment by using a Local Large Language Model and Bluetooth Low Energy relay, addressing conventional system limitations.

KR103047142B1Active Publication Date: 2026-11-11DAVINCO CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
KR1020260093253
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Priority Date
2026-05-12
Filing Date
2026-05-22
Publication Date
2026-11-11
Estimated Expiration
2046-05-22

AI Technical Summary

Technical Problem

Conventional senior care systems fail to accurately predict wandering risk before an elderly person leaves home, rely on reactive location-based methods that lose 'golden time' for intervention, struggle with communication blind spots, and expose personal information to external servers.

Method used

A voice analysis-based system using a Local Large Language Model (LLM) on an elderly terminal assesses wandering risk by analyzing pre-departure voice, switches to precision tracking in communication blind spots via Bluetooth Low Energy (BLE) relay, and calculates a crisis score for emergency determination.

Benefits of technology

Secures 'golden time' for intervention by predicting wandering risk, prevents personal information exposure, and ensures uninterrupted location tracking and accurate emergency judgment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure R1020260093253_ABST
    Figure R1020260093253_ABST
Patent Text Reader

Abstract

The present invention relates to a voice analysis-based intelligent senior care system, and more specifically, to a voice analysis-based intelligent senior care system that determines the degree of wandering risk of an elderly person by analyzing voice collected before the elderly person goes out using a Local Large Language Model (LLM) configured in the elderly person's terminal, and protects the elderly person by monitoring those at risk of wandering and tracking their location. Unlike conventional senior care methods that rely on post-event notifications after wandering or disappearance, the present invention supports securing a "golden time" for preemptive intervention by guardians or managers by predicting risk in advance based solely on subtle changes in voice at a stage before the elderly person actually enters a state of wandering. At the same time, it has the effect of fundamentally blocking the risk of external exposure of sensitive personal information by processing the elderly person's personal information and speech history directly within the elderly person's terminal without transmitting them to an external server.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to a voice analysis-based intelligent senior care system, and more specifically, to a voice analysis-based intelligent senior care system that determines the degree of wandering risk of an elderly person by analyzing voice collected before the elderly person goes out using a Local Large Language Model (LLM) configured in an elderly person's terminal, and protects the elderly person by monitoring the elderly person at risk of wandering and tracking the location of the elderly person. Background Technology

[0002] With the recent entry into an aging society, there is an increasing demand for senior care systems to protect the safety of elderly people living at home and to respond to emergency situations.

[0003] Conventional senior care systems widely utilize a method that uses GPS (Global Positioning System) or cellular location data from a device carried by an elderly person to determine whether they have left a pre-set geo-fencing zone, and provides an alarm to a guardian or control server if such departure is detected.

[0004] In addition, a method has been applied to detect falls or abnormal behaviors in the elderly through radar, cameras, and motion sensors placed within the home, and to determine emergency situations by sending a voice confirmation message to the elderly upon detection of abnormal signs to verify their response.

[0005] Meanwhile, to enhance personal information security and improve response speed, an on-premise edge computing method is also being applied, in which primary data processing is performed on terminals for the elderly or on edge devices within the home, and only the necessary information is transmitted to a control server.

[0006] However, these conventional senior care systems have the following problems.

[0007] First, conventional location-based systems employ a reactive approach that provides alarms to guardians or control servers only after the elderly person has already left the safe zone while out and about. This results in the loss of the "golden time" from the moment wandering or disappearance occurs, making it difficult to provide prompt protection for the elderly.

[0008] Second, conventional voice verification methods determine normality or abnormality based on whether the elderly respond simply or rely on STT (Speech-to-Text) converted text responses; therefore, there is a problem in that they fail to accurately identify emergency situations when elderly individuals with cognitive or hearing impairments respond with meaningless utterances or when background noise, such as TV noise, is mistaken for a positive response.

[0009] Third, conventional location-based tracking methods have a problem in that they cannot locate the elderly person when the elderly person enters a shadow area where GPS signals are cut off, such as a basement or subway station, or when the main communication network is disconnected, making it impossible to track the elderly person's location in the most critical situations.

[0010] Fourth, conventional cloud-based analysis methods, which transmit sensitive personal information such as the elderly's speech content, medical history, and cognitive status to external analysis servers for processing, carry a persistent risk of such information being exposed to external servers. Meanwhile, conventional edge computing methods, which attempt to avoid this by performing all processing locally, have limitations in that the efficiency of emergency response is reduced because analysis results and emergency situation information are difficult to transmit to the control server in a timely manner. As such, conventional voice analysis technology in the senior care field faces the problem of failing to simultaneously satisfy both personal information protection and emergency response efficiency.

[0011] Therefore, there is a need for a new type of senior care technology that can assess the risk of wandering in advance before an elderly person leaves the house, rather than relying on post-outside responses; analyze the elderly person's voice without transmitting their personal information externally; and enable location tracking even in areas with poor communication coverage. Prior art literature

[0012] Korean Registered Patent No. 10-2742759 The problem to be solved

[0013] The present invention was devised to solve the problems of the conventional technology described above. Its purpose is to provide a voice analysis-based intelligent senior care system that supports securing a "golden time" for the protection of the elderly without relying on post-event response after wandering or disappearance by analyzing the voice of the elderly collected before they go out through a Local Large Language Model (LLM) configured within the elderly terminal to determine the degree of wandering risk in advance, while also preventing the leakage of the elderly's personal information by processing it within the elderly terminal without transmitting it to an external analysis server.

[0014] Furthermore, the present invention aims to provide a voice analysis-based intelligent senior care system capable of tracking the location of an elderly person even in communication blind spots or when the main communication network is disconnected, by switching the tracking mode of the elderly person's terminal to a precision tracking mode when the pre-determined level of wandering risk of the elderly person exceeds a preset standard and transmitting location information of the elderly person to a control server through multiple available communication means, and by utilizing a Bluetooth Low Energy (BLE) communication mode to transmit the location information of the elderly person to a control server via mesh routing through surrounding terminals as relay nodes in blind spots where GPS (Global Positioning System) signals are disconnected or when the main communication network is disconnected.

[0015] Furthermore, the present invention aims to provide a voice analysis-based intelligent senior care system capable of improving the accuracy of emergency situation determination by comprehensively reflecting the elderly's cognitive state and medical history, as well as their response and voice state, unlike conventional emergency situation determination methods that rely on the elderly's simple response. This is achieved by transmitting a voice confirmation message to the elderly when abnormal signs are detected in the elderly, classifying the types of responses collected as a response, calculating a crisis score by synthesizing the classified response types with the elderly's cognitive decline score and underlying disease index as weights, and determining whether an emergency situation has occurred based on this score. means of solving the problem

[0016] In an intelligent senior care system based on voice analysis comprising an elderly terminal and a control server according to an embodiment of the present invention, the elderly terminal is configured to include a Local Large Language Model (LLM) that communicates with the control server via a communication network and operates by referencing the elderly person's personal information and speech history information stored in the elderly terminal; the system detects an outing trigger event corresponding to the elderly person's outing based on at least one of the elderly person's location information, internal sensor information, external sensor information, and wireless connection status information for a wireless access point pre-registered in the elderly terminal; generates voice input information based on an audio signal generated based on sound received by the elderly terminal during a pre-set past time based on the detection time of the outing trigger event, and inputs it into the Local LLM; and compares and analyzes a normal pattern including the elderly person's voice characteristic pattern and speech pattern generated according to the elderly person's personal information and speech history information through the Local LLM with the voice characteristic information and speech content information according to the voice input information, and determines the degree of wandering risk of the elderly person based on the deviation from the normal pattern according to the comparative analysis. It may be characterized by calculating, generating wandering risk information regarding the degree of wandering risk, transmitting the wandering risk information to the control server when the wandering risk information satisfies a preset risk condition, and switching the tracking mode of the elderly terminal to a precision tracking mode.

[0017] As an example related to the present invention, the elderly terminal may be characterized by detecting that the going-out trigger event has occurred when a state in which the elderly person leaves a preset residential area is detected based on location information generated by measuring the location of the elderly person, or when a door opening / closing signal, which is the external sensor information, is received from a front door opening / closing sensor unit provided in the elderly person's residential space, or when a movement start pattern is detected in which the acceleration value, which is the internal sensor information measured through an acceleration sensor provided in the elderly terminal, changes above a preset threshold from an acceleration value corresponding to a stationary state, or when a state of disconnection of the wireless connection to the preset wireless connection point is detected based on the wireless connection status information generated through a terminal communication unit provided in the elderly terminal for communication of the elderly terminal.

[0018] As an example related to the present invention, the elderly terminal may be characterized by generating a voice signal of the elderly person by separating it from the audio signal through a preset voice signal separation algorithm, measuring at least one of pitch, jitter, and shimmer from the voice signal to generate voice characteristic information including at least one of pitch variation rate information, jitter information, and shimmer information, converting the voice signal into text information to generate speech content information, and inputting the voice input information including the voice characteristic information and the speech content information into the local LLM.

[0019] As an example related to the present invention, the local LLM may be characterized by calculating an anxiety index by converting the degree of psychological anxiety of the elderly person into a quantitative value of a preset scale based on the voice characteristic information, calculating a speech confusion index by converting at least one of the frequency of occurrence of self-referential speech corresponding to soliloquy and the degree of contextual confusion into a quantitative value of a preset scale from the speech content information, calculating a normal anxiety index distribution and a normal speech confusion index distribution based on the voice characteristic pattern and speech pattern included in the normal pattern, calculating the deviation based on the degree to which the calculated anxiety index deviates from the normal anxiety index distribution and the degree to which the calculated speech confusion index deviates from the normal speech confusion index distribution, and then calculating the degree of wandering risk based on the deviation.

[0020] As an example related to the present invention, the elderly terminal transmits a voice confirmation message to the elderly when the occurrence of an abnormal sign event, including at least one of a fall event, an impact sound event, and an abnormal movement event, is detected based on at least one of the internal sensor information and the external sensor information; after transmitting the voice confirmation message, response information including an audio signal generated based on sound received by the elderly terminal during a preset response waiting time is input into a preset VAD (Voice Activity Detection) model, and the response is classified into one of a normal response state, a static state, a groaning sound detection state, an impact sound detection state, and a general living noise detection state through the VAD model, and response type information corresponding to the classified response type is generated; upon detection of the occurrence of the abnormal sign event, speech history information and personal information stored in the elderly terminal are input into the local LLM, and a baseline value is calculated based on a cognitive decline score that quantifies the degree of cognitive decline of the elderly through the local LLM and an underlying disease index for the elderly's underlying disease, and the abnormal sign event The method may be characterized by generating voice characteristic information based on audio signals corresponding to sounds received by the elderly terminal during a preset past time based on the time of occurrence, inputting this information into the local LLM to calculate an anxiety index through the local LLM, calculating a crisis score based on a preset crisis weight corresponding to a specific response type according to the response type information, the calculated anxiety index, and the baseline value, and generating abnormal situation occurrence information and transmitting it to the control server if the calculated crisis score is greater than or equal to a preset threshold.

[0021] As an example related to the present invention, the elderly terminal may be characterized by calculating an amplification factor by multiplying the anxiety index by the crisis weight corresponding to the specific response type when the response type information corresponds to any one response type excluding a normal response state, or when the anxiety index is greater than or equal to a preset anxiety threshold value, and calculating the crisis score by multiplying the baseline value by an exponential function value with the natural constant as the base and the amplification factor as the exponent.

[0022] As an example related to the present invention, the elderly terminal may be characterized in that, when switched to the precision tracking mode, it transmits the location information of the elderly person to the control server according to a preset second period shorter than the first period applied in the general tracking mode among the tracking modes, activates Bluetooth Low Energy (BLE) communication, and if the state of communication connection with the control server or the state of receiving an external positioning signal through a location measurement unit configured in the elderly terminal satisfies a preset communication restriction condition, it changes at least one of the scanning period or advertising period of the BLE communication to a fourth period shorter than the third period applied in the general tracking mode, performs scanning or advertising of the BLE communication according to the fourth period, and transmits tracking information including the recent location information of the elderly person and identification information of the elderly terminal to a nearby terminal discovered through the BLE communication, thereby causing the tracking information to be transmitted to the control server through the nearby terminal.

[0023] As an example related to the present invention, the elderly terminal may be characterized by acquiring location information of a nearby terminal that communicates via BLE communication when reception of an external positioning signal through the location measuring unit is not possible in the precision tracking mode, generating location information of the elderly based on the acquired location information of the nearby terminal, and generating tracking information including the generated location information of the elderly.

[0024] As an example related to the present invention, the control server receives together surrounding terminal identification information for identifying the surrounding terminal from the surrounding terminal transmitting the tracking information and the tracking information for the elderly person transmitted from the elderly terminal to the surrounding terminal via BLE communication; if the elderly person identification information for identifying the elderly terminal included in the received tracking information and the surrounding terminal identification information are inconsistent, the control server determines the surrounding terminal corresponding to the received surrounding terminal identification information as a compensation target; calculates a relay contribution based on at least one of the number of transmissions of tracking information and the size of transmission data based on one or more tracking information received from the surrounding terminal that is the compensation target during a pre-set compensation period through a pre-set compensation calculation algorithm; generates compensation information for the surrounding terminal based on the relay contribution; and transmits the compensation information to a pre-set external compensation management server so that compensation for the surrounding terminal is provided according to the compensation information.

[0025] As an example related to the present invention, the elderly terminal may be characterized by generating normal movement pattern information by time period based on movement history according to location information of the elderly accumulated and stored over a past period based on the current time point, comparing the elderly's current movement path information according to changes in the elderly's location information with the normal movement pattern information by time period, and determining that the elderly has deviated from the normal movement path if the comparison result satisfies a pre-set deviation condition, and generating movement anomaly information and transmitting it to the control server.

[0026] As an example related to the present invention, the control server may be characterized by determining the dispatch priority between one or more other elderly people and the elderly person by comparing the wandering risk information received from the elderly person terminal with the wandering risk information received from other elderly person terminals, generating dispatch request information including the dispatch priority for the elderly person and dispatch location information generated based on the location information received from the elderly person terminal, and transmitting the dispatch request information to a pre-set guardian terminal or a pre-set administrator terminal corresponding to the elderly person.

[0027] As an example related to the present invention, the elderly terminal monitors resource status information of the elderly terminal, and when resource status information regarding the usage or availability status of the operational resources constituting the elderly terminal satisfies a preset offloading condition, transmits voice analysis request information including the audio signal to a preset external voice analysis server via an encrypted communication network, receives analysis result information including voice characteristic information and speech content information calculated by the external voice analysis server based on the audio signal from the external voice analysis server, and generates voice input information including the voice characteristic information and speech content information included in the analysis result information and inputs it into the local LLM. Effects of the invention

[0028] The present invention inputs the voice of an elderly person collected during a certain period immediately prior to when the elderly person goes out into a local LLM (Local Large Language Model) configured within the elderly person's terminal to determine in advance the possibility that the elderly person may be in a wandering state based on deviations from a personalized normal pattern based on usual voice characteristics and speech content. If the possibility exceeds a preset level, the elderly person's terminal immediately notifies a control server in advance, and the control server transmits a notification to a preset guardian or manager while simultaneously initiating location tracking of the elderly person. Unlike conventional senior care methods that rely on post-notification after wandering or disappearance, this invention supports securing a golden time for preemptive intervention by a guardian or manager by predicting danger in advance based only on subtle changes in voice at a stage before the elderly person actually enters a wandering state. At the same time, it has the effect of fundamentally blocking the risk of external exposure of sensitive personal information by processing the elderly person's personal information and speech history directly within the elderly person's terminal without transmitting them to an external server.

[0029] Furthermore, the present invention has the effect of overcoming the structural limitations of conventional location tracking methods, which rely on a single communication network and a single positioning signal source and thus suffer from interrupted location tracking in underground spaces or communication blind spots, by enabling uninterrupted location tracking of the elderly person in any communication-restricted environment. This is achieved by having the elderly person's terminal transmit the elderly person's location to the control server at a shorter interval than usual when it is determined that the elderly person is highly likely to wander, and by transmitting the elderly person's location to the control server via a bypass transmission method through other terminals near the elderly person detected via BLE (Bluetooth Low Energy) short-range wireless communication in areas where GPS signal reception is blocked or mobile communication networks are disconnected.

[0030] Furthermore, the present invention has the effect of significantly improving the accuracy of emergency situation judgment and minimizing the waste of response resources by guardians and managers due to false alarms by comprehensively reflecting the response state and voice state as well as the cognitive state and medical history of the elderly, compared to conventional emergency notification methods that relied solely on simple acceleration changes or impact sound thresholds, by notifying a control server of the occurrence of an emergency only when abnormal signs such as falls, impact sounds, or movement abnormalities are detected in the elderly, after the elderly terminal transmits a voice message inquiring about the elderly's well-being and then analyzes the voice collected in response to calculate a crisis score by synthesizing the classified response state (normal response, silence, groaning, additional impact sound, or general daily noise, etc.), a reference value reflecting the elderly's usual degree of cognitive decline and underlying diseases, and the level of psychological anxiety extracted from the response voice, and by notifying the control server of the occurrence of an emergency only when the crisis score is above a preset reference value. Brief explanation of the drawing

[0031] FIG. 1 is a configuration diagram of a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 2 is a detailed configuration diagram of an elderly terminal and a control server constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 3 is a flowchart of the detection of an outing trigger event of an elderly terminal constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention, the generation of wandering risk information based on the detection of the outing trigger event, and the operation of switching to a precision tracking mode. FIG. 4 is a flowchart of an operation flowchart for an embodiment of the process of generating voice input information of an elderly terminal and generating wandering risk information through a local LLM, which constitutes a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 5 is a flowchart of the operation in the precision tracking mode of an elderly terminal constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 6 is an example diagram of a tracking information transmission operation using a surrounding terminal via BLE communication in a precision tracking mode of an elderly terminal constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 7 is an example diagram of the operation of generating and transmitting movement anomaly information based on the comparison of normal movement pattern information by time period and current movement path information of an elderly terminal constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 8 is a flowchart of the operation for detecting abnormal signs of an elderly terminal and generating information on the occurrence of an abnormal situation therefrom, which constitutes a voice analysis-based intelligent senior care system according to an embodiment of the present invention. FIG. 9 is an example diagram of the operation of a control server constituting a voice analysis-based intelligent senior care system according to an embodiment of the present invention. Specific details for implementing the invention

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0033] FIG. 1 is a configuration diagram of a voice analysis-based intelligent senior care system (hereinafter, senior care system) according to an embodiment of the present invention.

[0034] As described above, the senior care system according to an embodiment of the present invention may be configured to include an elderly terminal (100), a control server (200), etc.

[0035] At this time, the senior care system may be implemented with more components than those shown in Fig. 1, or with fewer components.

[0036] In addition, the senior care system according to an embodiment of the present invention may further comprise, in addition to the elderly terminal (100) and the control server (200), an abnormality detection device (300) provided inside the living space (residence or address) of the elderly person to detect abnormal signs of the elderly person, a front door opening / closing sensor unit (105) provided at the entrance door of the living space of the elderly person to detect the opening / closing status of the entrance door, one or more surrounding terminals located near the elderly terminal (100) when the elderly person leaves the living space and goes out, a guardian terminal carried by a guardian corresponding to the elderly person, and a manager terminal carried by a manager who manages the senior care system.

[0037] In addition, the elderly terminal (100), the control server (200), the abnormality detection device (300), the front door opening / closing sensor unit (105), one or more surrounding terminals, the guardian terminal, the administrator terminal, etc. can communicate with each other through a communication network.

[0038] At this time, the communication network described in the present invention may include wired / wireless communication networks, and examples of such wireless communication networks include Wireless LAN (WLAN), DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), WiMAX (World Interoperability for Microwave Access), GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), and IEEE 802.16. Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), Wireless Mobile Broadband Service (WMBS), 5G mobile communication service, Bluetooth, BLE (Bluetooth Low Energy), LoRa (Long Range), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, Near Field Communication (NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, etc. may be included. In addition, wired communication networks may include wired LAN (Local Area Network), wired WAN (Wide Area Network), Power Line Communication (PLC), USB communication, Ethernet, serial communication, optical / coaxial cables, etc.

[0039] In addition, the elderly terminal (100) according to the embodiment of the present invention is a terminal carried or worn by an elderly person (user), and may be referred to as a senior terminal, a user terminal, etc.

[0040] In addition, the elderly terminal (100) may be configured, for example, as a wearable terminal in the form of a smart watch, a portable terminal in the form of a smartphone, or a combination of the wearable terminal and the portable terminal.

[0041] In addition, the elderly terminal (100) may be configured in various forms other than the form described above, such as an installed terminal installed inside the living space of the elderly person, a smart speaker type terminal, and two or more of the wearable terminal, the portable terminal, the installed terminal, and the smart speaker type terminal may be interconnected to form a single elderly terminal (100).

[0042] In addition, the control server (200) according to an embodiment of the present invention is a server that monitors the condition of the elderly person based on information received from the elderly person terminal (100) (100) and performs tracking and dispatch management for the elderly person, and may be referred to as a control device, a management server, etc.

[0043] In addition, the abnormality detection device (300) according to an embodiment of the present invention is a device provided inside the living space of the elderly person to detect abnormal signs such as falls, impact sounds, or abnormal movements of the elderly person, and may include, for example, an IoT (Internet of Things) radar-based abnormality detection device (300), a camera-based abnormality detection device (300), or an acoustic sensor-based abnormality detection device (300).

[0044] In addition, the front door opening / closing sensor unit (105) according to an embodiment of the present invention is a device comprising a sensor that generates a door opening / closing signal corresponding to the opening / closing state of the door, provided at the entrance door of the living space of the elderly person, and may include various types of sensors such as a magnetic sensor, a contact type sensor, an infrared sensor, etc., and may be provided with a communication module for communicating with the elderly person terminal (100).

[0045] In addition, the one or more surrounding terminals according to the embodiment of the present invention are terminals located near the elderly terminal (100) when the elderly person goes out, and may include, for example, a smartphone or wearable terminal carried by a general person who is not the elderly person.

[0046] In addition, the guardian terminal and the administrator terminal according to the embodiment of the present invention may be implemented as various types of terminals, such as, for example, a smartphone, a tablet PC, a laptop, a desktop PC, a monitor, etc.

[0047] Additionally, in the configuration described above, the elderly terminal (100) may be configured to include a Local Large Language Model (LLM) that is pre-configured to communicate with the control server (200) via a communication network and to operate by referring to the elderly person's personal information and speech history information corresponding to the elderly terminal (100) stored in the elderly terminal (100); and a function to detect an outing trigger event (or outing event) corresponding to the elderly person's outing based on at least one of the elderly person's location information, internal sensor information, external sensor information, and wireless connection status information for a wireless connection point pre-registered in the elderly terminal (100); and a voice characteristic pattern of the elderly person generated according to the elderly person's personal information and speech history information through the Local LLM (109) by generating voice input information based on audio signals received by the elderly terminal (100) during a pre-configured past time based on the detection time of the outing trigger event. Various functions can be performed, such as a function to calculate the degree of wandering risk of the elderly person based on the deviation from the normal pattern according to the comparison analysis and the normal pattern including the speech pattern and speech characteristic information and speech content information according to the voice input information, and a function to generate wandering risk information regarding the degree of wandering risk, and a function to transmit the wandering risk information to the control server (200) and switch the tracking mode of the elderly terminal (100) to a precision tracking mode when the wandering risk information satisfies a preset risk condition.

[0048] Through this, the senior care system supports predicting the risk of wandering in advance by analyzing the deviation from the normal pattern based on the elderly person's usual voice characteristics and usual speech content through the local LLM (109) based on the audio signal collected immediately before the elderly person goes out; supports minimizing the exposure of personal information or speech content to an external server by processing the elderly person's personal information and speech history information internally within the elderly person terminal (100); and supports switching to the precision tracking mode only when the risk of wandering is detected to enable precise location tracking by the control server (200), thereby establishing an intelligent senior care environment that can ensure the safety of the elderly person, the rapid response of the guardian, and the efficient control of the manager. The detailed operational configuration of the elderly person terminal (100) and the control server (200) performing these functions will be described in detail below with reference to the drawings.

[0049] FIG. 2 is a detailed configuration diagram of an elderly terminal (100) and a control server (200) according to an embodiment of the present invention.

[0050] As described above, the elderly terminal (100) may be configured to include a terminal communication unit (101), a terminal storage unit (102), a terminal input unit (103), a terminal display unit (104), a sensor unit (105), a location measurement unit (106), a sound receiving unit (107), a sound output unit (108), a local LLM (109), a terminal control unit (110), etc., and the control server (200) may be configured to include a server communication unit (210), a server storage unit (220), a server control unit (230), etc.

[0051] At this time, the elderly terminal (100) and the control server (200) may be implemented with more components than those shown in FIG. 2, or the elderly terminal (100) and the control server (200) may be implemented with fewer components.

[0052] First, the detailed configuration of the above-mentioned elderly terminal (100) is explained.

[0053] The terminal communication unit (101) can communicate with the control server (200), the abnormality detection device (300), the front door opening / closing sensor unit (105), one or more surrounding terminals, the guardian terminal, the administrator terminal, etc. through a communication network.

[0054] In addition, the terminal communication unit (101) may be configured to include a cellular communication module that supports cellular communication, a wireless internet communication module that supports wireless internet communication, a short-range communication module that supports short-range wireless communication, etc.

[0055] At this time, the cellular communication described in the present invention may refer to a communication method that divides a geographical area into a plurality of cells or cell sectors and provides wide-area mobile communication services through wireless connection between a base station responsible for each cell or cell sector and a terminal, and may include, for example, GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), HSPA (High Speed ​​Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), 5G NR (New Radio), etc.

[0056] In addition, the wireless internet communication described in the present invention may refer to a communication method for wirelessly accessing the internet or an IP-based network, and may include various wireless access methods such as Wireless LAN (WLAN), Wi-Fi, Wireless Broadband (WiBro), and Worldwide Interoperability for Microwave Access (WiMAX).

[0057] In addition, the above-mentioned short-range wireless communication may refer to a communication method for transmitting and receiving short-range data between terminals or peripheral devices, and may include, for example, Bluetooth, BLE (Bluetooth Low Energy), ZigBee, UWB (Ultra Wideband), NFC (Near Field Communication), RFID (Radio Frequency Identification), infrared communication (IrDA), Wi-Fi Direct, etc.

[0058] To this end, the short-range communication module may be configured to include a BLE communication module that supports BLE (Bluetooth Low Energy) communication, an NFC communication module that supports NFC (Near Field Communication), etc.

[0059] In addition, the terminal storage unit (102) can store various data and programs necessary for the operation of the terminal control unit (110).

[0060] For example, the terminal storage unit (102) can store personal information stored in advance for the elderly person, speech history information of the elderly person, location information of the elderly person, movement history information based on the location information of the elderly person, etc.

[0061] In addition, the terminal input unit (103) can receive input from a user, generate input information, and provide the input information to the terminal control unit (110).

[0062] Additionally, the terminal display unit (104) can display various information according to the control of the terminal control unit (110), and the terminal input unit (103) and the terminal display unit (104) may be configured as components that constitute a touch screen configured in the elderly terminal (100).

[0063] In addition, the sensor unit (105) can be provided in the elderly terminal (100) to generate the internal sensor information.

[0064] For example, the sensor unit (105) may include an acceleration sensor to generate internal sensor information including acceleration measurement values, and may also include various types of sensors such as a gyro sensor, a geomagnetic sensor, and a heart rate sensor in addition to the acceleration sensor.

[0065] Additionally, the above-mentioned position measuring unit (106) can generate location information of the elderly terminal (or elderly person) (100) by measuring the location of the elderly terminal (elderly person) (100) based on an external positioning signal received from a GPS (Global Positioning System) satellite, and may be referred to as a GPS unit, etc.

[0066] In addition, the sound receiving unit (107) can receive sound generated in the vicinity of the elderly terminal (100) and generate an audio signal corresponding to the sound, and may be configured to include, for example, a microphone.

[0067] In addition, the sound output unit (108) can output various sounds according to the control of the terminal control unit (110), and may be configured to include, for example, a speaker.

[0068] Additionally, the local large language model (109) may be a large language model that operates by referring to personal information and speech history information of the elderly person that is stored in advance, and may be referred to as, for example, a lightweight LLM or an on-device LLM.

[0069] Additionally, the local LLM (109) may be configured as separate hardware (hardware module) inside the elderly terminal (100) as shown in FIG. 2, or as another embodiment, may be configured as software that is stored in the terminal storage unit (102) and executed by the terminal control unit (110).

[0070] Additionally, the local LLM (109) may be configured in the form of a pre-configured agent to automatically perform analysis processing for an elderly person corresponding to the elderly terminal (100).

[0071] Additionally, the local LLM (109) may have a system prompt pre-configured during the initial configuration of the local LLM (109) applied (configured), and the system prompt may include content instructing the local LLM (109) to analyze voice characteristic information and speech content information according to the voice input information by referring to the personal information and speech history information that are input into the local LLM (109) or stored in the elderly terminal (100), and to calculate the degree of wandering risk based on the deviation from the normal pattern.

[0072] Through this, the local LLM (109) can operate by automatically referencing the personal information and speech history information that are input along with the voice input information or stored in the terminal storage unit (102) of the elderly terminal (100) according to the preset system prompt whenever the voice input information is received from the terminal control unit (110).

[0073] For example, the local LLM (109) can automatically perform various functions, such as generating a normal pattern including the voice characteristic pattern and speech pattern of the elderly person by referring to the personal information and speech history information stored in the terminal storage unit (102) according to a system prompt pre-set in the local LLM (109), a function of calculating the degree of wandering risk of the elderly person based on the deviation from the normal pattern by comparing and analyzing input voice input information with the normal pattern, and a function of extracting the voice signal of the elderly person from the input audio signal to calculate voice characteristic information or speech content information.

[0074] Additionally, the local LLM (109) may be configured to process information in the form of an audio signal in addition to text information, and for example, after extracting the voice signal of the elderly person from the input audio signal, at least one of the pitch (or pitch variation rate), jitter, and shimmer of the voice signal may be measured to calculate the voice characteristic information including at least one of the pitch variation rate information, jitter information, and shimmer information, or the voice signal may be converted into text to calculate the speech content information.

[0075] In addition, the local LLM (109) can enhance the protection of the elderly person's personal information by performing the analysis processing internally within the elderly person terminal (100) without transmitting the personal information and speech history information to an external server of the elderly person terminal (100).

[0076] In addition, the terminal control unit (110) can perform overall control functions of the elderly terminal (100) and may include RAM, ROM, CPU, GPU, and bus, and the RAM, ROM, CPU, GPU, etc. may be connected to each other through the bus.

[0077] Additionally, the terminal control unit (110) can execute overall control functions of the elderly terminal (100) using programs and data previously stored in the terminal storage unit (102), and can control the terminal communication unit (101), the terminal storage unit (102), the terminal input unit (103), the terminal display unit (104), the sensor unit (105), the location measurement unit (106), the sound receiving unit (107), the sound output unit (108), the local LLM (109), etc.

[0078] Additionally, at least one of the terminal communication unit (101), the terminal storage unit (102), the terminal input unit (103), the terminal display unit (104), the sensor unit (105), the location measurement unit (106), the sound receiving unit (107), the sound output unit (108), and the local LLM (109) may be configured to be included in the terminal control unit (110).

[0079] Additionally, the terminal storage unit (102) may be configured to include application-related data for transmitting and receiving various information with the control server (200) and processing and displaying the transmitted and received information, and the terminal control unit (110) may operate as an application unit by executing the application-related data.

[0080] The operation of the terminal control unit (110) or the elderly terminal (100) described below may be performed by the application unit.

[0081] Next, the detailed configuration of the above-mentioned control server (200) is described.

[0082] The above server communication unit (210) can communicate through a communication network with the elderly terminal (100), a surrounding terminal located near the elderly when the elderly goes out, the guardian terminal, the administrator terminal, etc.

[0083] In addition, the server storage unit (220) can store various data and programs necessary for the operation of the server control unit (230), and, for example, can store location information, wandering risk information, movement abnormality information, abnormal situation occurrence information received from the elderly terminal (100).

[0084] At this time, the server storage unit (220) may be configured as a DB or may include one or more DBs.

[0085] For example, the server storage unit (220) may be configured to include a member DB in which member information of the elderly person is stored, and location information, wandering risk information, movement abnormality information, abnormal situation occurrence information, etc. corresponding to the elderly person that is matched with the member information may be stored in the member DB.

[0086] Alternatively, the server storage unit (220) may be configured as a separate database server to interact (communicate) with the control server (200).

[0087] Additionally, the server control unit (230) can perform overall control functions of the control server (200) and may include RAM, ROM, CPU, GPU, and a bus, and the RAM, ROM, CPU, GPU, etc. may be connected to each other through a bus.

[0088] Additionally, the server communication unit (210) and the server storage unit (220) may be configured to be included in the server control unit (230).

[0089] Based on the configuration described above, the detailed operation configuration of the elderly terminal (100) and the control server (200) will be explained below with reference to the drawings.

[0090] FIG. 3 is a flowchart of the detection of an outing trigger event of an elderly terminal (100) constituting a senior care system according to an embodiment of the present invention, the generation of wandering risk information based on the detection of the outing trigger event, and the operation of switching to a precision tracking mode.

[0091] As described above, the terminal control unit (110) configured in the elderly terminal (100) can detect an outing trigger event corresponding to the elderly person's outing based on at least one of the location information of the elderly person measured through the location measurement unit (106), internal sensor information generated through the sensor unit (105), external sensor information received through the terminal communication unit (101), and wireless connection status information generated through the terminal communication unit (101) (S1).

[0092] At this time, the terminal control unit (110) can continuously determine whether the outing trigger event occurs (S2).

[0093] For example, the terminal control unit (110) can detect (determine) that the going-out trigger event has occurred when the elderly person is detected leaving a preset residential area (inside the home area) based on the location information measured through the location measuring unit (106).

[0094] Additionally, the terminal control unit (110) can detect (determine) that the going-out trigger event has occurred when the door opening / closing signal is received from the front door opening / closing sensor unit (105) provided in the living space of the elderly person as the external sensor information.

[0095] Additionally, the terminal control unit (110) can detect (determine) that the outing trigger event has occurred when the acceleration value measured through the acceleration sensor included in the sensor unit (105) as internal sensor information is detected as a movement start pattern that changes above a preset threshold from the acceleration value corresponding to the stationary state.

[0096] At this time, the above-mentioned pre-set residential area may include, for example, the living space of the elderly person (residential space) and an area within a pre-set radius from the living space.

[0097] Additionally, the terminal control unit (110) can detect (determine) that the outing trigger event has occurred when a wireless connection disconnection state for the previously registered wireless connection point is detected based on the wireless connection status information generated through the wireless internet communication module of the terminal communication unit (101).

[0098] At this time, the wireless access point described in the present invention may refer to a wireless access point capable of wireless access through the wireless internet communication module, and may include, for example, a wireless sharing device, a wireless router, etc. installed within or near the living space of the elderly person.

[0099] At this time, the wireless access point can be identified by wireless access point identification information including, for example, SSID (Service Set Identifier), BSSID (Basic Service Set Identifier), etc.

[0100] Additionally, the wireless access point corresponding to the living space of the elderly person may be pre-registered in the elderly person terminal (100), and, for example, the wireless access point identification information of the pre-registered wireless access point may be pre-stored in the terminal storage unit (102).

[0101] Additionally, the wireless connection status information described in the present invention may refer to information generated by the terminal control unit (110) detecting the wireless connection status (connection or disconnection) of the pre-registered wireless connection point through the wireless internet communication module of the terminal communication unit (101).

[0102] To this end, the terminal control unit (110) can generate wireless connection status information by continuously monitoring the wireless connection availability and wireless connection status (connection or disconnection) of the pre-registered wireless connection point through the wireless internet communication module of the terminal communication unit (101), and can check the connection status (connection or disconnection) of the pre-registered specific wireless connection point based on the wireless connection status information.

[0103] For example, the terminal control unit (110) can detect (determine) that the wireless connection disconnection status of the previously registered wireless connection point has occurred by detecting (determining) that the going-out trigger event has occurred based on the wireless connection status information generated through the wireless internet communication module of the terminal communication unit (101) when the elderly person, who was located within or near the elderly person’s living space, moves out of the living space and moves out of the communication range of the previously registered wireless connection point.

[0104] In addition, the terminal control unit (110) can detect (determine) that the outing trigger event has occurred based on at least one of the location information, the internal sensor information, and the wireless connection status information, even in an environment where an external sensor generating the external sensor information, such as the front door opening / closing sensor unit (105), is not installed inside the living space of the elderly person.

[0105] Additionally, the terminal control unit (110) can detect the occurrence of the going-out trigger event when it corresponds to at least one of the above-described state of leaving the residential area, state of receiving a door opening / closing signal, state of detecting a movement start pattern, and state of disconnection of the wireless connection to the previously registered wireless connection point, and can also detect the occurrence of the going-out trigger event when two or more of the state of leaving the residential area, state of receiving a door opening / closing signal, state of detecting a movement start pattern, and state of disconnection of the wireless connection are detected simultaneously or sequentially.

[0106] Additionally, when the above terminal control unit (110) detects the above outing trigger event (S2), it can generate the voice input information based on an audio signal (or one or more audio signals) generated based on the sound received through the sound receiver (107) during a preset past time based on the time of detection of the above outing trigger event (S3).

[0107] That is, when the above terminal control unit (110) detects the above outing trigger event, it can extract one or more audio signals corresponding to the sound received through the sound receiver (107) during a preset past time based on the time of detection of the above outing trigger event from the terminal storage unit (102), and generate the voice input information based on the extracted one or more audio signals.

[0108] At this time, the sound receiving unit (107) can continuously receive sound occurring in the vicinity of the elderly terminal (100) and generate an audio signal, and the terminal control unit (110) can accumulate and store audio signals corresponding to the sound received through the sound receiving unit (107) in the terminal storage unit (102) at preset time intervals.

[0109] Additionally, the terminal control unit (110) can store the audio signal received from the sound receiving unit (107) in the terminal storage unit (102) by matching it with time information (or timestamp information) regarding the time when the audio signal was received or the time when the audio signal was generated.

[0110] In addition, the above-mentioned preset past time can be set to various time lengths, such as 5 minutes or 10 minutes past the time of detection of the above-mentioned outing trigger event.

[0111] At this time, the terminal control unit (110) can store the audio signal generated through the sound receiver (107) so that it is maintained in the terminal storage unit (102) for a preset maintenance time, and manage the audio signal that has elapsed the preset maintenance time so that it is sequentially deleted from the terminal storage unit (102).

[0112] Additionally, the above-mentioned preset retention time may be set to be longer than the above-mentioned past time so that the audio signal during the above-mentioned past time can be maintained in the terminal storage unit (102) at the time of detection of the above-mentioned outing trigger event.

[0113] Additionally, the audio signal may be sound information (or audio data or sound data converted into a digital signal) generated by digitally converting an analog signal corresponding to sound generated externally through the sound receiver (107) or the terminal control unit (110).

[0114] Additionally, according to one embodiment of the present invention, the terminal control unit (110) may input the audio signal itself, generated through the sound receiver (107) that received the external sound, into the local LLM (109) as voice input information. In this case, the local LLM (109) may extract the voice signal of the elderly person from the input audio signal and then directly calculate the voice characteristic information and the speech content information.

[0115] In another embodiment, the terminal control unit (110) may extract the voice signal of the elderly person from the audio signal generated through the sound receiver (107), directly calculate the voice characteristic information and the speech content information from the voice signal, and input the voice input information including the calculated voice characteristic information and the speech content information to the local LLM (109).

[0116] That is, the terminal control unit (110) may extract the voice signal of the elderly person from the audio signal generated through the sound receiver (107), measure at least one of the pitch, jitter, and shimmer of the voice signal to directly calculate the voice characteristic information, convert the voice signal into text information to directly calculate the speech content information, and then input the voice input information including the calculated voice characteristic information and the speech content information into the local LLM (109).

[0117] At this time, the specific operation of the terminal control unit (110) or the local LLM (109) to calculate the voice characteristic information and the speech content information from the audio signal will be described later with reference to FIG. 4.

[0118] In another embodiment, the terminal control unit (110) monitors resource status information of the elderly terminal (100), and when the resource status information satisfies a preset offloading condition, transmits voice analysis request information including the audio signal generated through the sound receiver (107) to a preset external voice analysis server via an encrypted communication network, and after receiving analysis result information including voice characteristic information and speech content information calculated by the external voice analysis server based on the audio signal from the external voice analysis server, generates voice input information including the voice characteristic information and speech content information included in the analysis result information and inputs it to the local LLM (109).

[0119] At this time, the offloading described in the present invention may mean transferring at least a portion of the analysis processing that is to be performed internally by the elderly terminal (100) to the external voice analysis server for processing.

[0120] In addition, the terminal control unit can generate the resource status information when the outing trigger event occurs and determine whether the resource status information satisfies a preset offloading condition.

[0121] Additionally, the resource status information of the elderly terminal (100) is information regarding the usage or availability status of the operating resources constituting the elderly terminal (100), and may include, for example, information on the remaining battery level of the elderly terminal (100), information on the load rate of the CPU or GPU of the terminal control unit (110), information on the memory usage of the terminal storage unit (102), and information on the self-analysis processing time in the elderly terminal (100).

[0122] At this time, the terminal control unit (110) can periodically monitor the remaining amount of a battery (not shown) provided in the elderly terminal (100), the load rate of the CPU or GPU included in the terminal control unit (110), the memory usage for the terminal storage unit (102), and the time required to independently calculate the voice characteristic information and the speech content information from the audio signal in the terminal control unit (110) or the local LLM (109), and generate the resource status information including the remaining amount of the battery, the load rate information of the CPU or GPU, the memory usage information, and the self-analysis processing time information.

[0123] Additionally, the above-mentioned preset offloading conditions may include, for example, at least one of a condition in which the remaining battery level according to the battery level information decreases below a preset threshold, a condition in which the load rate according to the CPU or GPU load rate information increases above a preset threshold, a condition in which the memory usage according to the memory usage information increases above a preset threshold, and a condition in which the self-analysis processing time according to the self-analysis processing time information becomes longer than a preset threshold.

[0124] At this time, the terminal control unit (110) may determine that the offloading condition is satisfied if at least one of the following conditions applies: the condition that the remaining battery level decreases below a preset threshold, the condition that the load rate of the CPU or GPU increases above a preset threshold, the condition that the memory usage increases above a preset threshold, and the condition that the self-analysis processing time becomes longer than a preset threshold, or if two or more of these conditions are satisfied simultaneously or sequentially.

[0125] Additionally, the terminal control unit (110) may transmit the voice analysis request information to the external voice analysis server only when the offloading condition is satisfied, and in normal circumstances when the offloading condition is not satisfied, the voice characteristic information and the speech content information may be generated by independently calculating the voice input information within the elderly terminal (100) through the local LLM (109) or the terminal control unit (110).

[0126] In addition, the voice analysis request information may further include, for example, elderly person identification information and information on the time of generation of the voice analysis request information in addition to the audio signal, and may be configured so that the elderly person's personal information and the elderly person's speech history information are not included in the voice analysis request information.

[0127] At this time, the elderly person identification information may be the identification information of the elderly person terminal.

[0128] Additionally, the terminal control unit (110) may transmit the voice analysis request information to the external voice analysis server using the encrypted communication network through the terminal communication unit (101), and the encrypted communication network may include, for example, a communication network protected by an encryption communication protocol such as TLS (Transport Layer Security) or SSL (Secure Sockets Layer).

[0129] Additionally, the external voice analysis server may extract a voice signal of the elderly person from the audio signal included in the voice analysis request information received from the elderly person terminal (100), measure at least one of the pitch, jitter, and shimmer of the extracted voice signal to calculate the voice characteristic information, convert the extracted voice signal into text information to calculate the speech content information, and then generate the analysis result information including the calculated voice characteristic information and the speech content information and transmit it to the elderly person terminal (100).

[0130] At this time, the external voice analysis server according to an embodiment of the present invention communicates with the elderly terminal (100) through a communication network and generates analysis result information including voice characteristic information and speech content information based on voice analysis request information received from the elderly terminal (100), and transmits it to the elderly terminal (100). For example, it can be implemented as various types of servers, such as a cloud-based server operating in a cloud environment or a data center-based server installed in a separate data center.

[0131] In addition, the external voice analysis server may be a server configured separately from the control server (200).

[0132] Additionally, the external voice analysis server may be configured to immediately discard the received voice analysis request information and the audio signal included in the voice analysis request information after transmitting the analysis result information to the elderly terminal (100).

[0133] Through this, the elderly terminal (100) can reliably secure the voice characteristic information and the speech content information by utilizing the external voice analysis server even in exceptional situations where it is difficult to analyze the audio signal due to a lack of available resources of the elderly terminal, and the exposure of the elderly's personal information to the external voice analysis server is prevented, and the external exposure of the voice signal can be minimized through the immediate disposal of data from the external voice analysis server.

[0134] In addition, the terminal control unit (110) can input the generated voice input information to the local LLM (109) (S4).

[0135] At this time, the terminal storage unit (102) may store speech history information in advance to manage the voice characteristic history and speech content history of the elderly person according to changes in time by accumulating and storing the voice characteristic information and speech content information by time whenever the terminal control unit (110) generates voice characteristic information and speech content information.

[0136] Accordingly, the terminal control unit (110) can update the speech history information by directly generating the voice characteristic information and speech content information or by obtaining the voice characteristic information and speech content information generated through the local LLM (109) from the local LLM (109) and accumulating it in the speech history information previously stored in the terminal storage unit (102), and the speech history information may include voice characteristic information and speech content information accumulated and stored by time (by time point).

[0137] At this time, the terminal control unit (110) can delete voice characteristic information and speech content information for which a preset storage period has elapsed from the speech history information, so that the elderly person's recent voice characteristic pattern and speech pattern are reflected in the speech history information.

[0138] Additionally, when the voice input information is input from the terminal control unit (110), the local LLM (109) can generate a normal pattern including the voice characteristic pattern and speech pattern of the elderly person by referring to the personal information of the elderly person and the speech history information of the elderly person that are previously stored in the terminal storage unit (102) according to the preset system prompt, and can calculate the deviation from the normal pattern based on the comparison analysis by comparing and analyzing the voice characteristic information and speech content information according to the voice input information with the normal pattern.

[0139] In addition, the local LLM (109) can calculate the degree of wandering risk of the elderly person based on the calculated deviation (S5).

[0140] Additionally, the terminal control unit (110) may generate wandering risk information based on the degree of wandering risk calculated through the local LLM (109), and the wandering risk information may include the degree of wandering risk and may include elderly person identification information corresponding to the elderly person terminal (100).

[0141] At this time, the normal pattern may be a pattern generated by the local LLM (109) based on the elderly person's personal information and speech history information, and may include the voice characteristic pattern and the speech pattern.

[0142] For example, the local LLM (109) can statistically process the voice characteristic information of the elderly person accumulated in the speech history information by time to generate the voice characteristic pattern corresponding to the elderly person's usual voice characteristics, and statistically process the speech content information of the elderly person accumulated in the speech history information by time to generate the speech pattern corresponding to the elderly person's usual speech content.

[0143] Additionally, the local LLM (109) can generate the normal pattern as a personalized pattern that reflects the characteristics of the elderly person, rather than the average voice and speech characteristics of the general group to which the elderly person belongs, by referring to the personal information when generating the normal pattern.

[0144] For example, the local LLM (109) can set the average distribution information of generally known voice characteristics according to the age and gender of the elderly person included in the personal information as the initial reference value of the voice characteristic pattern, and then correct the voice characteristic pattern by reflecting the voice characteristic information accumulated in the speech history information.

[0145] Additionally, the local LLM (109) can adjust the normal range of the voice characteristic pattern or the speech pattern to match the physical and cognitive characteristics of the elderly person by referring to the elderly person's medical history information (e.g., information on cognitive impairment, Parkinson's disease, laryngeal disease, etc.) or cognitive ability level information included in the personal information.

[0146] Additionally, the normal pattern may be configured in a form that is generated in advance by the local LLM (109) and stored as separate information in the terminal storage unit (102), or, as another embodiment, may be configured in a form that is recorded inside the local LLM (109) after the local LLM (109) learns based on the personal information and the speech history information.

[0147] At this time, the form recorded inside the local LLM (109) may be, for example, a form encoded as a learning parameter of the local LLM (109) or a form retained as in-context learning information of the local LLM (109).

[0148] In addition, according to one embodiment of the present invention, when inputting the voice input information into the local LLM (109), the terminal control unit (110) may read the personal information of the elderly person and the most recently updated speech history information, which are previously stored in the terminal storage unit (102), from the terminal storage unit (102) and input them into the local LLM (109) together with the voice input information.

[0149] Through this, the local LLM (109) can generate the normal pattern by referring to the input personal information and speech history information according to the preset system prompt, and then compare and analyze it with the voice input information.

[0150] In particular, since the above speech history information is information that is accumulated and updated in the terminal storage unit (102) over time by the terminal control unit (110), the terminal control unit (110) can ensure that the local LLM (109) can always perform analysis based on the latest speech history by reading the latest updated speech history information from the terminal storage unit (102) and inputting it together whenever the voice input information is input to the local LLM (109).

[0151] Additionally, when inputting the voice input information to the local LLM (109), the terminal control unit (110) may generate input prompt information for requesting the calculation of the degree of loitering risk based on the personal information and speech history information stored in advance in the terminal storage unit (102) and the voice input information, and input it to the local LLM (109). In this case, the local LLM (109) may generate the normal pattern at each analysis point based on the input personal information and speech history information and utilize it for the comparative analysis.

[0152] In addition, the above degree of wandering risk may refer to a value that quantifies the likelihood of the elderly person wandering after going out, and may be represented, for example, as a numerical value on a preset scale.

[0153] In addition, the above loitering risk information may include, for example, a numerical value of the degree of loitering risk, risk grade information corresponding to the degree of loitering risk, and time information corresponding to the time of calculation of the degree of loitering risk.

[0154] In addition, the specific operation of the local LLM (109) calculating the deviation from the normal pattern and the specific operation of calculating the degree of wandering risk will be described later with reference to FIG. 4.

[0155] In addition, the terminal control unit (110) can determine whether the generated wandering risk information satisfies a preset risk condition (S6).

[0156] At this time, the above risk condition may include, for example, a condition regarding whether the numerical value of the degree of wandering risk corresponds to or greater than a preset threshold.

[0157] Meanwhile, in the configuration described above, the external voice analysis server may be configured to include a Large Language Model (LM) that performs the function of the local LLM.

[0158] Accordingly, the terminal control unit (110) may determine whether the resource status information satisfies the preset offloading condition when an outing trigger event occurs, and if the result of the determination determines that the offloading condition is satisfied and the computational performance of the elderly terminal (100) is insufficient below a preset standard, it may generate voice analysis request information including the audio signal and the elderly person's personal information and speech history information stored in the terminal storage unit (102) and transmit it to the external voice analysis server. The external voice analysis server may, upon receiving the voice analysis request information, generate voice input information based on the audio signal included in the voice analysis request information, input it into the LLM together with the elderly person's personal information and speech history information, and through the LLM, compare and analyze the normal pattern including the elderly person's voice characteristic pattern and speech pattern generated according to the elderly person's personal information and speech history information with the voice characteristic information and speech content information according to the voice input information, and calculate the degree of wandering risk of the elderly person based on the deviation from the normal pattern according to the comparison analysis. It can be performed collectively on behalf of the terminal (100).

[0159] Additionally, the external voice analysis server may transmit analysis result information including the calculated degree of wandering risk to the elderly terminal (100), and the terminal control unit (110) may generate wandering risk information based on the degree of wandering risk included in the analysis result information.

[0160] At this time, the generation of the voice input information may be performed by the terminal control unit (110), and voice analysis request information including the voice input information may be transmitted to the external voice analysis server, so that the voice input information according to the voice analysis request information is input into the LLM together with the elderly person's personal information and speech history information to calculate the degree of wandering risk.

[0161] Through this, the senior care system can support the stable provision of a wandering risk prediction function for the elderly regardless of the computational performance of the elderly terminal (100) by having the external voice analysis server collectively perform the inference calculation including the calculation of the wandering risk level, even when the elderly terminal (100) is a low-performance terminal that is difficult to perform the inference calculation of the local LLM (109) on its own, such as an old terminal or an ultra-low-cost terminal.

[0162] In addition, the senior care system may selectively apply an embodiment in which the external voice analysis server performs only the calculation of the voice characteristic information and the speech content information according to the resource status of the elderly terminal (100), and an embodiment in which the external voice analysis server performs inference operations including the calculation of the degree of wandering risk, thereby allowing for wide application to elderly terminals (100) of various performance capabilities.

[0163] At this time, the terminal control unit (110) transmits the voice analysis request information, which includes the elderly person's personal information and speech history information, to the external voice analysis server through the encrypted communication network, and the external voice analysis server is configured to immediately discard the received voice analysis request information after transmitting the analysis result information, thereby minimizing the exposure of the elderly person's personal information to the outside during the process of performing the inference operation.

[0164] Meanwhile, the terminal control unit (110) can transmit the wandering risk information to the control server (200) through the terminal communication unit (101) and switch the tracking mode of the elderly terminal (100) to the precision tracking mode (S7) when the numerical value of the wandering risk level according to the wandering risk information is greater than or equal to the threshold value and satisfies the risk condition (S6).

[0165] At this time, the tracking mode may include, for example, a mode that is automatically activated when an outing trigger event occurs, a general tracking mode that transmits the location information of the elderly person according to a preset general transmission cycle when the risk condition is not satisfied, and a precision tracking mode that transmits the location information of the elderly person according to a precision transmission cycle shorter than the general transmission cycle when the risk condition is satisfied.

[0166] Additionally, the terminal control unit (110) may operate in the general tracking mode when the wandering risk information does not satisfy the risk condition (S6).

[0167] In addition, the detailed operation of the terminal control unit (110) in the above precision tracking mode will be described later with reference to FIGS. 5 and FIGS. 6.

[0168] Meanwhile, FIG. 4 is a flowchart of an operation for an example of the process of generating voice input information and generating wandering risk information through the local LLM (109) of a senior care system according to the embodiment of the present invention described above.

[0169] As described above, the terminal control unit (110) can separate (extract) the voice signal of the elderly person from the audio signal received from the sound receiver (107) (generated through the sound receiver (107)) using a preset voice signal separation algorithm (or voice signal extraction algorithm) (S11).

[0170] At this time, the voice signal separation algorithm may be an algorithm that separates from the audio signal a voice signal corresponding to the voice of the elderly person and a non-voice signal that is not the voice of the elderly person, such as TV sound, noise from the operation of home appliances, and environmental noise occurring in the living space of the elderly person.

[0171] In addition, the above-mentioned speech signal separation algorithm may include, for example, a spectral subtraction algorithm, a Wiener filter algorithm, an adaptive filter algorithm, an Independent Component Analysis (ICA) algorithm, a Non-negative Matrix Factorization (NMF) algorithm, a deep learning-based speech separation algorithm, etc.

[0172] Additionally, the terminal control unit (110) can measure at least one of pitch (or pitch variation rate), jitter, and shimmer from the generated voice signal according to a preset voice characteristic measurement algorithm, and generate voice characteristic information including at least one of pitch variation rate information, jitter information, and shimmer information (S12).

[0173] For example, the pitch variation rate information may refer to information regarding the fundamental frequency variation rate of the voice signal, the jitter information may refer to information regarding the degree of frequency fluctuation of the voice signal, and the shimmer information may refer to information regarding the degree of amplitude fluctuation of the voice signal.

[0174] In addition, the terminal control unit (110) can convert the generated voice signal into text information to generate the speech content information (S13).

[0175] At this time, the terminal control unit (110) may use, for example, a STT (Speech to Text) conversion algorithm to convert the voice signal into text information.

[0176] Additionally, the operation of generating voice characteristic information (S12) and the operation of generating speech content information (S13) may be performed in parallel or sequentially by the terminal control unit (110).

[0177] Additionally, the terminal control unit (110) can input the voice input information, including the generated voice characteristic information and the generated speech content information, into the local LLM (109) (S14).

[0178] In addition, the local LLM (109) can calculate an anxiety index by converting the degree of psychological anxiety of the elderly person into a quantitative value of a preset scale based on the voice characteristic information included in the input voice input information (S15).

[0179] At this time, the preset scale may be, for example, a quantitative scale of 1 to 100, and the anxiety index may be calculated as a numerical value of the preset scale.

[0180] Additionally, the local LLM (109) can calculate at least one of the frequency of occurrence of self-referential speech corresponding to self-talk and the contextual confusion from the speech content information included in the input voice input information, and convert it into a quantitative value of a preset scale to calculate a speech confusion index (S16).

[0181] For example, the self-referential utterances mentioned above are utterances corresponding to monologues spoken by the elderly person toward themselves without an external conversation partner, and may include, for example, utterances such as "Where am I?", "I have to go see Mom", and "The door won't open."

[0182] Additionally, the frequency of occurrence of the self-referential utterance may refer to the frequency with which the self-referential utterance occurred among the utterance content information during the preset past time.

[0183] In addition, the above contextual confusion may refer to a value that quantifies the degree of lack of contextual consistency among utterances included in the utterance content information.

[0184] Additionally, the operation of calculating the anxiety index (S15) and the operation of calculating the speech confusion index (S16) may be performed in parallel by the local LLM (109) or sequentially.

[0185] Additionally, the local LLM (109) can compare and analyze the calculated anxiety index and the calculated speech confusion index with the normal pattern to calculate the deviation from the normal pattern, calculate the degree of wandering risk corresponding to the deviation, and generate the wandering risk information for the degree of wandering risk (S17).

[0186] For example, the local LLM (109) can calculate a normal anxiety index distribution corresponding to the elderly person's normal voice characteristics based on the voice characteristic pattern included in the normal pattern, and calculate a normal speech confusion index distribution corresponding to the elderly person's normal speech content based on the speech pattern included in the normal pattern.

[0187] In addition, the deviation from the normal pattern can be calculated, for example, based on the degree to which the calculated anxiety index deviates from the usual anxiety index distribution and the degree to which the calculated speech confusion index deviates from the usual speech confusion index distribution.

[0188] In addition, as another example, the deviation from the normal pattern may be calculated by comparing the product value obtained by multiplying the calculated anxiety index and the calculated speech confusion index with the usual product value calculated based on the usual anxiety index distribution and the usual speech confusion index distribution.

[0189] In addition, as another example, the deviation from the normal pattern may refer to a value calculated by using the difference between the calculated anxiety index and the normal anxiety index reference value and the difference between the calculated speech confusion index and the normal speech confusion index reference value, after calculating a reference value for a normal anxiety index corresponding to the elderly person's usual voice characteristics based on the voice characteristic pattern included in the normal pattern and calculating a reference value for a normal speech confusion index corresponding to the elderly person's usual speech content based on the speech pattern included in the normal pattern.

[0190] For example, the deviation from the normal pattern can be calculated by summing or weighting the difference between the above-calculated anxiety index and the above-calculated normal anxiety index reference value, and the difference between the above-calculated speech confusion index and the above-calculated speech confusion index reference value.

[0191] Additionally, the local LLM (109) can calculate the degree of wandering risk in proportion to the magnitude of the deviation from the normal pattern, and for example, if the calculated anxiety index deviates from the normal range of the normal anxiety index distribution or the calculated speech confusion index deviates from the normal range of the normal speech confusion index distribution, it can determine that the deviation from the normal pattern is large and calculate the degree of wandering risk as high.

[0192] Additionally, the operation of FIG. 4 described above is an operation flow according to one embodiment of the present invention in which the terminal control unit (110) preprocesses the audio signal to directly calculate the voice characteristic information and the speech content information and then inputs them to the local LLM (109). In another embodiment, when the terminal control unit (110) inputs the audio signal itself to the local LLM (109), the operations of steps S11 to S13 of FIG. 4 may be performed by the local LLM (109).

[0193] In addition, the operation according to FIG. 4 described above may correspond to an example of the voice input information generation step (S3) to the loitering risk information generation step (S5) of FIG. 3.

[0194] Meanwhile, FIG. 5 is a flowchart of the operation in the precision tracking mode of an elderly terminal (100) constituting a senior care system according to an embodiment of the present invention.

[0195] As described above, the terminal control unit (110) can activate the tracking mode at the time the outing trigger event is detected and set the tracking mode to a preset general tracking mode.

[0196] In addition, the terminal control unit (110) can switch the tracking mode from a general tracking mode to a precision tracking mode when the wandering risk information satisfies a preset risk condition, as described above.

[0197] In addition, the terminal control unit (110) manages the activation and switching of the tracking mode and can determine whether the tracking mode of the elderly terminal (100) has been switched to the precision tracking mode (S21).

[0198] Additionally, when the tracking mode is switched to the precision tracking mode (S21), the terminal control unit (110) can shorten the transmission period of the location information of the elderly person to a preset second period that is shorter than the preset first period applied in the general tracking mode, and activate all of the cellular communication module, wireless internet communication module, and short-range communication module supporting BLE communication included in the terminal communication unit (101) (S22).

[0199] At this time, the first cycle and the second cycle may be set such that, for example, the first cycle is set to a relatively long time length such as 30 seconds, 1 minute, or 5 minutes, and the second cycle is set to a shorter time length than the first cycle such as 1 second, 3 seconds, or 5 seconds.

[0200] In addition, the terminal control unit (110) can transmit the location information of the elderly person to the control server (200) according to the second cycle through the terminal communication unit (101) (S23).

[0201] At this time, the terminal control unit (110) can generate tracking information including the location information of the elderly person and pre-set identification information of the elderly person (or identification information of the elderly person terminal (100)) and transmit it to the control server (200).

[0202] Additionally, the terminal control unit (110) can determine whether the state of communication connection with the control server (200) through the terminal communication unit (101) or the state of receiving an external positioning signal through the location measurement unit (106) configured in the elderly terminal (100) satisfies a preset communication restriction condition (S24).

[0203] At this time, the communication restriction condition may include a condition in which the communication connection with the control server (200) is disconnected or the reception of an external positioning signal through the location measuring unit (106) is impossible.

[0204] For example, the above communication restriction condition may be satisfied when the elderly person enters an underground space, a shadow area, or a communication shadow area.

[0205] Additionally, when the communication connection status with the control server (200) or the external positioning signal reception status through the location measurement unit (106) satisfies the communication restriction condition (S24), the terminal control unit (110) can control the terminal communication unit (101) to change at least one of the scanning cycle or advertising cycle of the BLE communication to a preset fourth cycle that is shorter than the preset third cycle applied in the general tracking mode (S25).

[0206] At this time, the third period and the fourth period may be set such that, for example, the third period is set to a relatively long time length such as 5 seconds or 10 seconds, and the fourth period is set to a shorter time length than the third period, such as within 1 second.

[0207] Additionally, the terminal control unit (110) can perform scanning or advertising via BLE communication according to the fourth cycle, receive location information of each of the one or more surrounding terminals discovered via BLE communication via BLE communication, and generate location information of the elderly terminal (100) based on the received location information of the surrounding terminals (S26).

[0208] At this time, the specific operation of the terminal control unit (110) generating location information of the elderly terminal (100) based on location information of the surrounding terminal will be described later with reference to FIG. 6.

[0209] In addition, the terminal control unit (110) can generate tracking information including location information generated using identification information of the elderly terminal (100) and location information of the surrounding terminal (S27).

[0210] At this time, the tracking information may include, for example, identification information of the elderly terminal (100), recent location information of the elderly, information on the time of creation of the tracking information, etc.

[0211] In addition, the identification information may include, for example, a terminal ID of the elderly terminal (100) or a member ID corresponding to the elderly.

[0212] Additionally, the terminal control unit (110) can transmit the generated tracking information to each of the one or more surrounding terminals discovered via the BLE communication, and the surrounding terminal that receives the tracking information can operate as a relay node and transmit the tracking information received from the elderly terminal (100) to the control server (200) (S28).

[0213] At this time, the relay node may refer to a node that receives the tracking information from the elderly terminal (100) via the BLE communication and then transmits the tracking information to the control server (200) using the relay node's own communication network connection.

[0214] Additionally, if the communication connection status with the control server (200) and the external positioning signal reception status do not satisfy the communication restriction condition (S24), the terminal control unit (110) may not perform steps S25 to S28 described above, and may continuously or periodically perform the operation of transmitting the location information of the elderly person to the control server (200) according to the second period through the terminal communication unit (101) (S23).

[0215] FIG. 6 is an example diagram of the operation of transmitting tracking information using a surrounding terminal via BLE communication in the precision tracking mode of the elderly terminal (100) described above.

[0216] As illustrated in FIG. 6, when the elderly person enters a shadow area (e.g., an area where external positioning signals cannot be received and the communication network is disconnected), the reception of external positioning signals through the location measuring unit (106) may be blocked (①).

[0217] At this time, as illustrated in FIG. 6, the communication network connection between the terminal communication unit (101) of the elderly terminal (100) and the control server (200) may also be disconnected.

[0218] Additionally, when the terminal control unit (110) detects that the communication restriction condition is satisfied as described above, it can perform scanning or advertising of the BLE communication to search for one or more surrounding terminals located around the elderly terminal (100) and receive location information of the one or more surrounding terminals through the BLE communication.

[0219] Additionally, the terminal control unit (110) can generate location information of the elderly terminal (100) according to a preset algorithm based on the received location information of one or more surrounding terminals, and then generate tracking information (②) including the generated location information and identification information of the elderly terminal (100).

[0220] At this time, the terminal control unit (110) can support location tracking of the elderly person even in a shadow area where the external positioning signal cannot be received, by generating location information of the elderly person terminal (100) using location information of one or more surrounding terminals received through the BLE communication, even when it is difficult to measure the location of the elderly person terminal (100) itself in a state where the external positioning signal cannot be received.

[0221] Additionally, the terminal control unit (110) can periodically transmit the generated tracking information to each of the one or more surrounding terminals discovered through the BLE communication, and can transmit the tracking information to the control server (200) via the surrounding terminal operating as the relay node (③).

[0222] At this time, the surrounding terminal operating as the relay node may be, for example, a surrounding terminal located outside the shaded area and in a state where a communication network connection is normally established.

[0223] In addition, the relay node can support the tracking information received from the elderly terminal (100) via the BLE communication to the control server (200) through the relay node's own communication network, thereby enabling the tracking information to reach the control server (200) even when the communication network connection of the elderly terminal (100) is disconnected.

[0224] For example, as illustrated in FIG. 6, when a plurality of surrounding terminals (e.g., surrounding terminal #1, surrounding terminal #2, surrounding terminal #3, etc.) are located around the elderly terminal (100), the terminal control unit (110) can search for the plurality of surrounding terminals through scanning according to the BLE communication and communicate with each of the plurality of surrounding terminals through the BLE communication to receive location information of each of the plurality of surrounding terminals.

[0225] Additionally, the terminal control unit (110) can generate location information of the elderly terminal (100) based on the location information of the plurality of surrounding terminals, generate tracking information including the generated location information, and then transmit an advertisement packet including the tracking information through advertising of the BLE communication, so that each of the plurality of surrounding terminals that receive the advertisement packet operates as a relay node and transmits the tracking information to the control server (200).

[0226] In addition, as the advertisement packet is received by each of the plurality of peripheral terminals, even if the tracking information cannot be transmitted to the control server (200) due to the communication status of any one of the plurality of peripheral terminals, the tracking information can be transmitted to the control server (200) via another peripheral terminal, thereby improving the transmission reliability of the tracking information.

[0227] Through this, the elderly terminal (100) can support the tracking information to be transmitted seamlessly to the control server (200) through a BLE communication network of a mesh topology formed by the plurality of surrounding terminals.

[0228] In the configuration described above, when the communication network connection with the control server (200) is maintained while the elderly terminal (100) satisfies the external positioning signal reception state, the terminal control unit (110) may generate tracking information through BLE communication with a plurality of surrounding terminals as described above and then directly transmit the tracking information to the control server (200) through the terminal communication unit (101).

[0229] In addition, if the elderly terminal (100) is configured as a watch-type wearable terminal, the sensor unit (105) may further include a biosignal measurement sensor for measuring the biosignal of the elderly.

[0230] At this time, the above-mentioned biosignal measurement sensor may include, for example, a heart rate sensor, an oxygen saturation sensor, a body temperature sensor, etc.

[0231] In addition, the terminal control unit (110) can generate biometric measurement information regarding the heart rate, oxygen saturation, body temperature, etc. of the elderly person based on sensing information measured through the biometric signal measurement sensor and include it in the tracking information.

[0232] Additionally, the terminal control unit (110) can apply the audio signal generated through the sound receiver (107) from the time of switching to the precision tracking mode to a preset voice analysis algorithm for voice state analysis, and analyze at least one of the elderly person's speech frequency, speech volume, and speech tremor degree through the voice analysis algorithm, and generate voice state information including at least one of the elderly person's speech frequency, speech volume, and speech tremor degree through the voice analysis algorithm, and can include the voice state information in the tracking information.

[0233] At this time, the voice analysis algorithm may be composed of, for example, a deep learning-based voice analysis model that is pre-trained to generate voice state information from the audio signal.

[0234] Through this, the control server (200) can monitor the physical condition and voice condition of the elderly person in addition to the location information of the elderly person based on the biometric information or voice condition information included in the tracking information received from the elderly person terminal (100).

[0235] Meanwhile, in the operation according to FIGS. 5 and 6 described above, when the control server (200) receives tracking information about the elderly person from the surrounding terminal, it can support compensation for the surrounding terminal or the user of the surrounding terminal by calculating compensation information for the surrounding terminal that contributed to the relay of the tracking information and transmitting it to the external compensation management server.

[0236] To this end, when the elderly terminal (100) transmits tracking information for the elderly person through BLE communication via the short-range communication module of the terminal communication unit (101) in the precision tracking mode, the tracking information may include elderly identification information for identifying the elderly terminal (100).

[0237] At this time, the elderly person identification information may include, for example, user identification information corresponding to the elderly person, terminal identification information corresponding to the elderly person terminal (100) (MDN (Mobile Directory Number), mobile IP, mobile MAC (media access control), MAC address, unique SIM (subscriber identity module) card information, serial number), etc.

[0238] Additionally, when the surrounding terminal receives the tracking information from the elderly terminal (100) via the BLE communication, it may transmit the surrounding terminal identification information for identifying the surrounding terminal, along with the received tracking information, to the control server (200) according to a preset operation of the application pre-installed on the surrounding terminal.

[0239] Additionally, the control server (200) can receive the surrounding terminal identification information from the surrounding terminal that transmits the tracking information through the server communication unit (210), and the tracking information for the elderly person transmitted from the elderly person terminal (100) to the surrounding terminal via the BLE communication.

[0240] Additionally, the server control unit (230) of the control server (200) can compare the elderly person identification information and the received surrounding terminal identification information included in the received tracking information, and if the elderly person identification information and the surrounding terminal identification information are inconsistent, determine that the surrounding terminal contributed to the relay of the tracking information by determining that the received tracking information is information of the elderly person terminal (100) rather than information of the surrounding terminal itself, and thus determine the surrounding terminal corresponding to the received surrounding terminal identification information as a compensation target.

[0241] Additionally, the server control unit (230) can accumulate and store the tracking information in the server storage unit (220) by matching it with the surrounding terminal identification information of the surrounding terminal whenever it receives tracking information transmitted via a relay from the surrounding terminal determined as the compensation target.

[0242] Additionally, the server control unit (230) may accumulate and store relay execution information regarding at least one of the number of transmissions of tracking information and the size of transmission data in the server storage unit (220) whenever tracking information is received from a surrounding terminal, or update existing relay execution information stored in the server storage unit (220) based on the received tracking information, and the corresponding relay execution information may be matched with the surrounding terminal identification information and stored in the server storage unit (220).

[0243] At this time, the surrounding terminal may be registered as a member in advance in the control server (200), and, for example, surrounding terminal identification information for identifying the surrounding terminal may be stored in advance in the server storage unit (220) of the control server (200). In addition, the surrounding terminal identification information may include, for example, user identification information, terminal identification information, etc. corresponding to the surrounding terminal.

[0244] Additionally, the server control unit (230) can check at least one of the number of transmissions of tracking information (cumulative number of transmissions) and the size of transmission data (cumulative (total) size of transmission data) based on one or more tracking information received from a surrounding terminal that is the target of the compensation during a pre-set compensation period or relay execution information stored in the server storage unit (220) through a pre-set compensation calculation algorithm, calculate the relay contribution based on the confirmed information, and then generate compensation information for the surrounding terminal based on the calculated relay contribution.

[0245] At this time, the above-mentioned pre-set compensation period can be set in various period units, such as daily, weekly, monthly, or quarterly.

[0246] In addition, the above-mentioned preset compensation calculation algorithm may be preset to calculate a relay contribution as a quantitative value by applying a preset weight to at least one of the cumulative number of transmissions of tracking information for the surrounding terminal and the cumulative transmission data size, for example, and to calculate a compensation value corresponding to the compensation information by applying a preset conversion factor to the calculated relay contribution.

[0247] In addition, the above-mentioned compensation information may include, for example, information regarding volunteer hours (accumulation of volunteer hours), local currency points (accumulation of local currency operated by a local government), carbon mileage (accumulation of mileage for eco-friendly activities), public points (accumulation of points usable when using public services), medical vouchers (accumulation of vouchers usable when using medical services), etc., and may include the above-mentioned surrounding terminal identification information.

[0248] Additionally, the server control unit (230) can transmit the generated reward information corresponding to the surrounding terminal to the external reward management server through the server communication unit (210), thereby enabling the external reward management server to provide a reward according to the reward information to the surrounding terminal or the user of the surrounding terminal.

[0249] Through this, the senior care system can provide an environment that induces voluntary participation in senior care by ordinary citizens and establishes a community-level senior safety net by ensuring that when a surrounding terminal contributes to the transmission of tracking information about the elderly person as a relay node in a communication blind spot or a communication-unavailable area of ​​the elderly terminal (100), a public reward is paid to the surrounding terminal or the user of the surrounding terminal according to the relay contribution.

[0250] Meanwhile, FIG. 7 is an example diagram of the operation of generating and transmitting abnormal movement information based on the comparison of normal movement pattern information by time period and current movement path information of an elderly terminal (100) constituting a senior care system according to an embodiment of the present invention.

[0251] As illustrated in FIG. 7, the terminal control unit (110) can compare the current movement path information of the elderly person based on the change in location information of the elderly person measured through the location measurement unit (106) with the normal movement pattern information for the elderly person set in advance for each time period, and if the pre-set deviation condition is satisfied as a result of the comparison, it can generate movement abnormality information regarding the movement abnormality of the elderly person and transmit it to the control server (200) through the terminal communication unit (101).

[0252] At this time, the above-mentioned normal movement pattern information by time period is information that is pre-set or updated based on movement history information according to location information accumulated and stored for the elderly person, and may include, for example, information on destinations frequently visited by the elderly person by time period and normal routes from the elderly person's residence to the destinations (①).

[0253] For example, the above-mentioned normal movement pattern information by time period may include, as shown in FIG. 7, a normal morning route from a residence to a park during the morning hours (e.g., 06:00 to 12:00), a normal afternoon route from a residence to a supermarket during the afternoon hours (e.g., 12:00 to 18:00), and a normal evening route from a residence to a hospital during the evening hours (e.g., 18:00 to 22:00).

[0254] In this case, the normal routes for the morning, afternoon, and evening hours may be routes traveled by the elderly on mutually different dates.

[0255] To this end, the terminal control unit (110) can accumulate and store location information measured through the location measurement unit (106) in the tracking mode in the terminal storage unit (102), and periodically generate normal movement pattern information by time period through a pre-set movement pattern analysis algorithm based on the movement history according to the location information (one or more location information) of the elderly person accumulated and stored during a pre-set past period based on the current time.

[0256] At this time, the above movement pattern analysis algorithm may include, for example, a clustering-based pattern analysis algorithm, a time series analysis algorithm, a statistical distribution analysis algorithm, a deep learning-based movement pattern learning algorithm, etc.

[0257] Alternatively, the terminal control unit (110) may generate location input information based on location information (one or more location information) of the elderly person accumulated and stored during the past period, and then input it into the local LLM (109) to generate normal movement pattern information by time period through the local LLM (109).

[0258] In addition, the terminal control unit (110) can store the generated time-based normal movement pattern information in the terminal storage unit (102).

[0259] In addition, the terminal control unit (110) can generate current movement path information of the elderly person by tracking changes over time in the location information of the elderly person measured through the location measurement unit (106) (②).

[0260] In addition, the terminal control unit (110) can compare the normal path according to the time-specific normal movement pattern information corresponding to the time period to which the current time belongs with the generated current movement path information to determine whether the current movement path information satisfies the preset deviation condition from the normal path.

[0261] At this time, the above deviation conditions may include, for example, a condition in which the location of the elderly person according to the current movement path information deviates from a preset allowable radius of the normal path according to the time-based normal movement pattern information, a condition in which the direction of travel of the current movement path information deviates from the direction of travel of the normal path by more than a preset angle, and a condition in which the location of the elderly person according to the current movement path information enters an area other than the destination according to the time-based normal movement pattern information.

[0262] For example, as illustrated in FIG. 7, the terminal control unit (110) compares the current movement path information with the normal movement pattern information for each time period, and if the elderly person moves along a different path other than the normal path corresponding to the time period to which the current time belongs, and the location of the elderly person enters an area outside the allowable radius pre-set based on the normal path, the terminal control unit (110) can determine that the deviation condition is satisfied (③).

[0263] Additionally, the terminal control unit (110) can determine that an abnormality has occurred in the elderly person's movement state when the comparison result shows that the current movement path information satisfies the deviation condition, generate movement abnormality information regarding the movement abnormality (movement state abnormality), and transmit the generated movement abnormality information to the control server (200) through the terminal communication unit (101).

[0264] At this time, the above movement anomaly information may include, for example, elderly person identification information for identifying the elderly person, information on the time of occurrence of the movement anomaly or information on the time of generation of the movement anomaly information, the elderly person's current location information, the current movement path information, the normal movement pattern information by time period, and information on the result of determining whether the deviation condition is satisfied.

[0265] In addition, the operation of generating and transmitting the movement abnormality information based on the comparison of the normal movement pattern information by time period and the current movement path information of the terminal control unit (110) according to FIG. 7 described above can be performed in both the general tracking mode and the precision tracking mode during the operation of the tracking mode of the elderly terminal (100).

[0266] Meanwhile, FIG. 8 is a flowchart of the operation for detecting abnormal signs of an elderly terminal (100) constituting a senior care system according to an embodiment of the present invention and generating information on the occurrence of an abnormal situation therefrom.

[0267] As described above, the terminal control unit (110) can detect the occurrence of an abnormal sign event including at least one of a fall event, an impact sound event, and a movement abnormality event for the elderly person based on at least one of the internal sensor information and the external sensor information (S31).

[0268] For example, the terminal control unit (110) can detect the occurrence of the fall event, which is an event suspected of being a fall of the elderly person, when a fall pattern is detected in which the acceleration value measured through the acceleration sensor included in the sensor unit (105) changes above a preset threshold from the range of acceleration values ​​corresponding to a preset normal activity state in the terminal control unit (110) or the terminal storage unit (102).

[0269] Additionally, the terminal control unit (110) can detect the occurrence of the impact sound event, which is an event suspected of being an impact to the elderly person, when an impact sound having a sound pressure level greater than a preset threshold is detected through a preset impact sound detection algorithm in the audio signal generated through the sound receiver (107).

[0270] In addition, the terminal control unit (110) can detect the occurrence of the movement abnormality event when information corresponding to the movement abnormality of the elderly person is received from the abnormality detection device (300) as the external sensor information.

[0271] For example, if the above anomaly detection device (300) is configured as a radar-based anomaly detection device (300), the radar-based anomaly detection device (300) receives a reflected wave of a radar signal transmitted into the living space to track the movement of the elderly person, and if the pattern of the tracked movement deviates from a preset normal movement pattern by more than a preset standard, or if the movement of the elderly person is not detected for more than a preset time, it can generate information corresponding to the movement anomaly of the elderly person and transmit it to the elderly person terminal (100).

[0272] At this time, the radar-based anomaly detection device (300) may include, for example, a mmWave (millimeter wave) radar or an FMCW (Frequency Modulated Continuous Wave) radar.

[0273] In addition, the terminal control unit (110) can transmit (output) a pre-set voice confirmation message to the elderly person through the sound output unit (108) when the occurrence of the abnormal sign event is detected (S31) (S32).

[0274] For example, the voice confirmation message may include guidance voices such as "Are you okay?" or "Please respond," as voice information for confirming whether the elderly person responds.

[0275] Additionally, the terminal control unit (110) inputs response information, including an audio signal generated through the sound receiver (107), into a pre-set VAD (Voice Activity Detection) model based on the sound received by the sound receiver (107) during a pre-set response waiting time after the transmission of the voice confirmation message. Then, through the VAD model, the response information is classified into one of the following response types: a normal response state in which the elderly person responds normally, a static state in which the voice of the elderly person is not detected, a groaning sound detection state, an impact sound detection state, and a general living noise detection state. After classifying the response information into one of the following response types, the response type information corresponding to the classified response type can be generated (S33).

[0276] At this time, the above-mentioned preset response waiting time can be set to various time lengths, such as 10 seconds, 20 seconds, 30 seconds, etc.

[0277] In addition, the VAD model may be a deep learning model that is pre-trained to analyze the audio signal included in the input response information and classify the response information into one of a plurality of response types, including the normal response state, the static state, the groaning sound detection state, the impact sound detection state, and the general living noise detection state.

[0278] For example, the above normal response state may mean a state in which a normal voice response of the elderly person is detected during the response waiting time.

[0279] Additionally, the static state may mean a state in which the voice or other sounds of the elderly person are not detected during the response waiting time, and the groaning sound detection state may mean a state in which the groaning sound of the elderly person is detected during the response waiting time.

[0280] Additionally, the impact sound detection state may mean a state in which an additional impact sound is detected during the response waiting time, and the general living noise detection state may mean a state in which general living noise other than the voice of the elderly person is detected during the response waiting time.

[0281] Additionally, the terminal control unit (110) can input speech history information and personal information stored in the terminal storage unit (102) of the elderly terminal (100) into the local LLM (109) when the occurrence of the abnormal sign event is detected, and calculate a baseline value based on a cognitive decline score that quantifies the degree of cognitive decline of the elderly person through the local LLM (109) and a baseline index for the elderly person's underlying disease (S34).

[0282] At this time, the terminal control unit (110) can input input prompt information for requesting the calculation of the baseline value, including speech history information and personal information stored in the terminal storage unit (102) of the elderly terminal (100), into the local LLM (109) when the occurrence of the abnormal sign event is detected.

[0283] Additionally, the above cognitive decline score may represent a quantitative value in which the local LLM (109) quantifies the degree of cognitive decline based on at least one of the changes in vocabulary diversity, changes in contextual consistency, and changes in the frequency of self-referential utterances over time of one or more utterance content information included in the utterance history information, for example.

[0284] Additionally, the above underlying disease index may represent, for example, a quantitative value in which the local LLM (109) quantifies the degree of underlying disease of the elderly person based on medical history information included in the above personal information.

[0285] Additionally, the baseline value may correspond to the sum, product, or weighted sum of the cognitive decline score and the underlying disease index.

[0286] For example, the baseline value may be a weighted sum calculated by adding a first value obtained by multiplying a pre-set first weight and the cognitive decline score by the local LLM (109), and a second value obtained by multiplying a pre-set second weight and the underlying disease index.

[0287] Additionally, the terminal control unit (110) can extract an audio signal corresponding to the sound received by the sound receiver (107) during a preset past time based on the time of occurrence of the abnormal sign event from the terminal storage unit (102), generate voice characteristic information based on the extracted audio signal (the one or more extracted audio signals), and input it into the local LLM (109) to calculate an anxiety index corresponding to the input voice characteristic information through the local LLM (109) (S35).

[0288] At this time, the operation of the terminal control unit (110) or the local LLM (109) generating voice characteristic information from the audio signal and the operation of the local LLM (109) calculating the anxiety index from the voice characteristic information are identical or similar to the operation described above with reference to FIG. 4, so a detailed description is omitted.

[0289] Additionally, the baseline value calculation operation (S34) and the anxiety index calculation operation (S35) may be performed in parallel by the terminal control unit (110) and the local LLM (109), or they may be performed sequentially.

[0290] In addition, the terminal control unit (110) can calculate a crisis score based on a pre-set crisis weight corresponding to a specific response type according to the response type information, the calculated anxiety index, and the baseline value (S36).

[0291] At this time, the crisis weight is a pre-set weight for each response type, and may be set as any one of, for example, a first crisis weight when the response type corresponds to the groaning sound detection state, a second crisis weight when the response type corresponds to the impact sound detection state, a third crisis weight when the response type corresponds to the static state, a fourth crisis weight when the response type corresponds to the general living noise detection state, and a fifth crisis weight when the response type corresponds to the normal response state.

[0292] Additionally, the first to fifth crisis weights may be set higher than the third and fourth crisis weights, for example, by reflecting that the groaning sound detection state or the impact sound detection state has a higher degree of urgency than the static state or the general living noise detection state, and the fifth crisis weight corresponding to the normal response state may be set to the lowest.

[0293] Additionally, the terminal control unit (110) may calculate an amplification factor by multiplying the anxiety index by the crisis weight corresponding to the specific response type and the anxiety index when the response type information corresponds to any one of the response types excluding the normal response state, or when the calculated anxiety index is greater than or equal to a preset anxiety threshold value, and calculate the crisis score by multiplying the baseline value by an exponential function value with the natural constant as the base and the amplification factor as the exponent.

[0294] At this time, the above natural constant may mean a mathematical constant corresponding to Euler's number, for example, having a value of about 2.71828.

[0295] In addition, the crisis score may be calculated as a larger value compared to the simple summation method due to the exponential amplification effect according to the amplification factor when the response type corresponds to any one of the preset response types excluding the normal response state, or when the calculated anxiety index is greater than or equal to the anxiety threshold value.

[0296] Additionally, the terminal control unit (110) may calculate the baseline value as the crisis score or terminate a series of operations without calculating the crisis score when the response type information corresponds to the normal response state and the calculated anxiety index is less than the anxiety threshold value.

[0297] In addition, the terminal control unit (110) can determine whether the calculated crisis score is greater than or equal to a preset threshold (crisis threshold) (S37).

[0298] Additionally, the terminal control unit (110) can generate abnormal situation occurrence information and transmit it to the control server (200) through the terminal communication unit (101) when the calculated crisis score is greater than or equal to the preset threshold (S37) (S38).

[0299] At this time, the above abnormal situation occurrence information may include, for example, elderly person identification information for identifying the elderly person, information on the time of occurrence of the above abnormal sign event, information on the location of occurrence of the above abnormal sign event, the above response type information, the above crisis score, the above baseline value, etc.

[0300] Additionally, the terminal control unit (110) may terminate a series of operations without generating abnormal situation information when the calculated crisis score is less than the threshold value (S37).

[0301] Meanwhile, FIG. 9 is a schematic diagram illustrating the operation of a control server (200) constituting a senior care system according to an embodiment of the present invention, which determines a dispatch priority by comparing wandering risk information received from a plurality of elderly terminals (100), generates dispatch request information including dispatch location information according to the determined dispatch priority, and transmits it to at least one of a guardian terminal and an administrator terminal.

[0302] As illustrated in FIG. 9, the control server (200) can receive the wandering risk information from each of one or more elderly terminals (100) corresponding to at least one of the elderly person and one or more other elderly people other than the elderly person.

[0303] At this time, each of the wandering risk information received from the one or more elderly terminals (100) may include the degree of wandering risk of the elderly calculated by each of the one or more elderly terminals (100) as described above.

[0304] For example, the above wandering risk information may further include identification information of an elderly person corresponding to the elderly terminal (100) that transmitted the wandering risk information, along with the degree of wandering risk.

[0305] Additionally, the control server (200) can determine the dispatch priority between one or more other elderly people and the elderly person by comparing the wandering risk information received from the elderly person terminal (100) with the wandering risk information received from other elderly person terminals (100).

[0306] For example, the control server (200) can determine the dispatch priority for an elderly person corresponding to each of the one or more elderly terminals (100) by comparing the degree of wandering risk included in each of the plurality of wandering risk information received from the one or more elderly terminals (100).

[0307] At this time, the control server (200) can compare the wandering risk information received from the elderly terminal (100) according to preset time intervals with the wandering risk information received from another elderly terminal (100) in the time interval corresponding to the wandering risk information, and if there is no wandering risk information received from another elderly terminal (100) in the time interval at which the wandering risk information was received from the elderly terminal (100), the highest priority for dispatching the elderly person corresponding to the elderly terminal (100) can be assigned.

[0308] Accordingly, the control server (200) can determine the dispatch priority such that the higher the degree of loitering risk, the higher the dispatch priority.

[0309] For example, as illustrated in FIG. 9, when the control server (200) receives wandering risk information including a wandering risk level of 60 from the first elderly terminal (100) (elderly terminal 1), wandering risk information including a wandering risk level of 95 from the second elderly terminal (100) (elderly terminal 2), and wandering risk information including a wandering risk level of 85 from the third elderly terminal (100) (elderly terminal 3), the control server (200) can determine the dispatch priority as follows: the elderly person corresponding to the second elderly terminal (100) with the highest wandering risk level as the first priority, the elderly person corresponding to the third elderly terminal (100) with the next highest wandering risk level as the second priority, and the elderly person corresponding to the first elderly terminal (100) with the lowest wandering risk level as the third priority.

[0310] Additionally, the control server (200) may generate dispatch location information (dispatch target location information) for each of the one or more elderly terminals (100) based on the determined dispatch priority, from the location information of the elderly most recently received from the elderly terminal (100), and generate dispatch request information (or dispatch request information) including the dispatch priority and the dispatch location information.

[0311] For example, the dispatch request information may include dispatch priority information regarding the determined dispatch priority and dispatch location information, and may further include elderly person identification information regarding the elderly person to be dispatched.

[0312] At this time, the elderly person identification information may be obtained from the wandering risk information, and the dispatch request information may include the elderly person's member identification information obtained from the member information of the member DB corresponding to the elderly person identification information.

[0313] Afterwards, the control server (200) can transmit the generated dispatch request information to a pre-set guardian terminal or a pre-set administrator terminal corresponding to the elderly person to be dispatched according to the dispatch priority.

[0314] At this time, the control server (200) can identify guardian information or administrator information for a guardian corresponding to the elderly person to be dispatched from the member information of the member DB corresponding to the elderly person identification information included in the wandering risk information, and can identify a guardian terminal or administrator terminal corresponding to the elderly person to be dispatched based on the identified guardian information or administrator information.

[0315] For example, as illustrated in FIG. 9, the control server (200) can transmit the dispatch request information to a pre-set guardian terminal and a pre-set administrator terminal in correspondence with the second elderly terminal (100) (elderly terminal 2) determined as the first priority.

[0316] At this time, when the dispatch priority is determined by comparing the wandering risk information between the elderly terminals (100) that transmitted the wandering risk information, the control server (200) can generate dispatch request information including dispatch priority information, dispatch location information, and elderly identification information for each of the elderly persons subject to dispatch who have been assigned a dispatch priority, and then transmit it to the administrator terminal or guardian terminal, thereby supporting the guardian or administrator to efficiently plan the dispatch route for the one or more elderly persons subject to dispatch by considering the dispatch priority and dispatch location, and then dispatch to the elderly persons subject to dispatch sequentially according to the dispatch route.

[0317] In addition, the guardian described in the present invention may refer to a person who protects the elderly person, and may be composed of various forms such as, for example, the elderly person's family (for example, spouse, children, siblings, etc.), a caregiver (or care manager) who cares for or protects the elderly person, a social worker, or dispatch personnel who go to the scene to assist the elderly person in the event of an emergency.

[0318] In addition, the manager described in the present invention may refer to a person who operates or manages the senior care system, and may include, for example, management personnel of a control center operating the control server (200), emergency dispatch control personnel, personnel belonging to the entity operating the senior care system, dispatch personnel, etc.

[0319] Through this, the control server (200) can support the priority dispatch of the elderly person with the highest degree of wandering risk even when the risk of wandering of multiple elderly people is detected at the same time or time, and can support guardians and managers in responding quickly to the elderly person with the highest degree of wandering risk.

[0320] Meanwhile, the control server (200) may generate dispatch request information based on abnormal situation occurrence information or movement abnormal information received from the elderly terminal (100), and transmit the generated dispatch request information to a pre-set guardian terminal or pre-set administrator terminal corresponding to the elderly person.

[0321] At this time, the above abnormal situation occurrence information may include a crisis score as described above, and the control server (200) may mutually compare the crisis scores of the abnormal situation occurrence information received from each of the one or more elderly terminals (100) at the same time, so that a higher crisis score is assigned a higher dispatch priority, and generate dispatch request information including the assigned dispatch priority and the most recent location information for the elderly person assigned a dispatch priority.

[0322] Additionally, the movement anomaly information may include information regarding the time of generation of the movement anomaly information, and the control server (200) may compare the time of generation of the movement anomaly information received from each of the one or more elderly terminals (100) at the same time, so that a higher dispatch priority is assigned to information with an earlier time of generation, and generate dispatch request information including the assigned dispatch priority and the most recent location information for the elderly person to whom the dispatch priority is assigned.

[0323] At this time, the control server (200) may assign a highest priority dispatch priority to an elderly person corresponding to the elderly terminal (100) if, at the time corresponding to the time when the abnormal situation occurrence information or the movement abnormal information is received from the elderly terminal (100), there is no information received from another elderly terminal (100) that can be compared with the abnormal situation occurrence information or movement abnormal information received from the elderly terminal (100).

[0324] Additionally, the control server (200) can generate the dispatch request information and then transmit it to the guardian terminal of a pre-set guardian or the pre-set administrator terminal for the elderly person corresponding to the dispatch request information.

[0325] In the above configuration, the operation of the control server (200) can be performed by a server control unit (230) configured in the control server (200), and the server control unit (230) can perform the above operation while communicating with the plurality of elderly terminals (100), guardian terminals, administrator terminals, etc. through the server communication unit (210).

[0326] Additionally, the server storage unit (220) may include a member DB in which the member information of the elderly person is stored, and the server control unit (230) may identify the member information of the elderly person in the member DB based on the elderly person identification information included in the wandering risk information, abnormal situation occurrence information, or movement abnormal information, and by referring to the identified member information of the elderly person, include the member information of the elderly person corresponding to the elderly person identification information included in the wandering risk information, abnormal situation occurrence information, or movement abnormal information in the dispatch request information.

[0327] In addition, the server control unit (230) can identify the guardian information of the guardian corresponding to the elderly person based on the member information, and can transmit the dispatch request information to the guardian terminal corresponding to the elderly person based on the guardian information.

[0328] As described above, the present invention enables the elderly terminal (100) to detect an outing trigger event for the occurrence of an outing of the elderly based on at least one of the elderly's location information, internal sensor information, and external sensor information, generate voice input information based on audio signals received during a pre-set past time based on the detection time of the outing trigger event, and input it into the local LLM (109) to compare and analyze a normal pattern including voice characteristic patterns and speech patterns generated according to the elderly's personal information and speech history information, and voice characteristic information and speech content information according to the voice input information, and calculate the degree of wandering risk of the elderly based on the deviation from the normal pattern according to the comparison analysis, thereby supporting the prediction of the elderly's wandering risk in advance through audio signals collected immediately before the elderly goes out, and thereby enabling guardians and managers to intervene preemptively before the elderly enter an actual wandering state and get lost or are exposed to safety accidents, thereby ensuring the safety of the elderly going out in advance and preventing missing person accidents caused by wandering.

[0329] In addition, the present invention can enhance personal information protection by minimizing the exposure of the elderly person's personal information and speech history by processing the personal information and speech history information of the elderly person that were previously stored in advance and the elderly person's speech history information in the local LLM (109) configured inside the elderly person terminal (100) and not transmitting them to an external server.

[0330] In addition, the present invention supports the continuous transmission of the location of the elderly person to the control server (200) by switching the tracking mode of the elderly person terminal (100) to the precision tracking mode when a risk of wandering is expected for an elderly person who has triggered an outing event based on voice analysis, and by activating all communication methods supported by the elderly person terminal (100) in the precision tracking mode, and by supporting the transmission of the tracking information to the control server (200) through one or more surrounding terminals discovered via BLE communication even when the communication connection with the control server (200) is disconnected or when the reception of an external positioning signal through the location measurement unit (106) is impossible, thereby enabling the location tracking of the elderly person to be performed without interruption even in communication-restricted environments such as shadow areas, thereby effectively protecting the elderly person.

[0331] In addition, the present invention supports the ability to make a precise judgment of an abnormal situation regarding an elderly person by considering the cognitive and physical characteristics of the elderly person analyzed based on the classified response type, personal information and speech history information of the elderly person, together with the current voice analysis status of the elderly person, by performing voice analysis on the elderly person's voice through conversation with the elderly person at the elderly person terminal (100) when an abnormal sign occurs in the elderly person's residence, and by notifying the control server (200) only when an actual abnormal situation occurs regarding the elderly person, thereby minimizing false alarms that frequently occurred in conventional methods relying on simple acceleration changes or impact sound detection, and allowing a guardian or manager to focus on rapid and reliable emergency response to a true emergency situation, thereby preventing waste of control resources and improving the effectiveness of emergency response.

[0332] In addition, the present invention has the effect of detecting personalized abnormal movement at an early stage during the elderly person's daily activity time, unlike conventional methods that judge abnormal movement based only on static geographical boundaries such as residential areas, by comparing normal movement pattern information by time period generated based on the elderly person's usual movement history with the elderly person's current movement path information and transmitting abnormal movement information to the control server (200) when deviation conditions are satisfied.

[0333] In addition, the present invention supports the priority dispatch of the elderly person with the highest level of urgency even when a crisis situation involving multiple elderly people is detected at the same time or time, thereby providing an intelligent senior care environment that enables the securing of the safety of the elderly person, a rapid response by the guardian, and efficient control by the manager.

[0335] The components described in the embodiments of the present invention may be implemented using one or more general-purpose computers or special-purpose computers, such as hardware including, for example, memory, processor, controller, ALU (arithmetic logic unit), digital signal processor, microcomputer, FPGA (Field Programmable Gate Array), PLU (programmable logic unit), microprocessor, logic gate, etc., software including an instruction set or a combination thereof, or any other device capable of executing and responding to instructions.

[0336] The various devices and components described in this specification may be implemented by hardware circuits (e.g., CMOS-based logic circuits), firmware, software, or a combination thereof. For example, they may be implemented using transistors, logic gates, and electronic circuits in the form of various electrical structures.

[0337] In addition, the deep learning algorithm or deep learning model described in the present invention may be composed of one or more neural network models, and the neural network model (or neural network) may be composed of an input layer, one or more hidden layers and an output layer, and various types of neural networks such as a Deep Neural Network (DNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN), Transformer, Attention mechanism, Graph Neural Network, etc. may be applied to the neural network model.

[0338] A person skilled in the art to which the present invention pertains will be able to make modifications and variations to the foregoing without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by these embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0339] 100: Elderly terminal 101: Terminal communication unit 102: Terminal storage unit 103: Terminal input unit 104: Terminal display unit 105: Sensor unit 106: Position measuring unit 107: Acoustic receiving unit 108: Audio Output Section 109: Local LLM 110: Terminal control unit 200: Control server 210: Server Communication Unit 220: Server Storage Unit 230: Server control unit 300: Anomaly detection device

Claims

Claim 1 In a voice analysis-based intelligent senior care system comprising an elderly terminal and a control server, the elderly terminal is configured to include a Local Large Language Model (LLM) that communicates with the control server via a communication network and operates by referencing the elderly person's personal information and speech history information stored in the elderly terminal; detects an outing trigger event corresponding to the elderly person's outing based on at least one of the elderly person's location information, internal sensor information, external sensor information, and wireless connection status information for a wireless access point pre-registered in the elderly terminal; generates voice input information based on an audio signal generated based on sound received by the elderly terminal during a pre-set past time based on the detection time of the outing trigger event, inputs it into the Local LLM, and compares and analyzes a normal pattern including the elderly person's voice characteristic pattern and speech pattern generated according to the elderly person's personal information and speech history information through the Local LLM with the voice characteristic information and speech content information according to the voice input information, and calculates the degree of wandering risk of the elderly person based on the deviation from the normal pattern according to the comparative analysis; A voice analysis-based intelligent senior care system characterized by generating wandering risk information regarding the degree of wandering risk, transmitting the wandering risk information to the control server when the wandering risk information satisfies preset risk conditions, and switching the tracking mode of the elderly terminal to a precision tracking mode. Claim 2 The voice analysis-based intelligent senior care system according to claim 1, wherein the elderly terminal detects that an outing trigger event has occurred when a state in which the elderly person leaves a preset residential area is detected based on location information generated by measuring the location of the elderly person, or when a door opening / closing signal, which is external sensor information, is received from a front door opening / closing sensor unit provided in the elderly person's residential space, or when a movement start pattern is detected in which an acceleration value, which is internal sensor information measured through an acceleration sensor provided in the elderly terminal, changes above a preset threshold from an acceleration value corresponding to a stationary state, or when a state of disconnection of wireless connection to a preset wireless connection point is detected based on wireless connection status information generated through a terminal communication unit provided in the elderly terminal for communication of the elderly terminal. Claim 3 The voice analysis-based intelligent senior care system according to claim 1, wherein the elderly terminal separates and generates a voice signal of the elderly person from the audio signal through a preset voice signal separation algorithm, measures at least one of pitch, jitter, and shimmer from the voice signal to generate voice characteristic information including at least one of pitch variation rate information, jitter information, and shimmer information, converts the voice signal into text information to generate speech content information, and inputs the voice input information including the voice characteristic information and the speech content information into the local LLM. Claim 4 In claim 3, the local LLM calculates an anxiety index by converting the degree of psychological anxiety of the elderly person into a quantitative value of a preset scale based on the voice characteristic information, calculates a speech confusion index by converting at least one of the frequency of occurrence of self-referential speech corresponding to monologue and the degree of contextual confusion from the speech content information into a quantitative value of a preset scale, calculates a normal anxiety index distribution and a normal speech confusion index distribution based on the voice characteristic pattern and speech pattern included in the normal pattern, calculates the deviation based on the degree to which the calculated anxiety index deviates from the normal anxiety index distribution and the degree to which the calculated speech confusion index deviates from the normal speech confusion index distribution, and calculates the degree of wandering risk based on the deviation. Claim 5 In claim 4, the elderly terminal transmits a voice confirmation message to the elderly when the occurrence of an abnormal sign event, including at least one of a fall event, an impact sound event, and an abnormal movement event, is detected based on at least one of the internal sensor information and the external sensor information; after transmitting the voice confirmation message, response information including an audio signal generated based on sound received by the elderly terminal during a preset response waiting time is input into a preset VAD (Voice Activity Detection) model, and classifies the response type into one of a normal response state, a static state, a groaning sound detection state, an impact sound detection state, and a general living noise detection state through the VAD model, and then generates response type information corresponding to the classified response type; upon detection of the occurrence of the abnormal sign event, speech history information and personal information stored in the elderly terminal are input into the local LLM, and a baseline value is calculated based on a cognitive decline score that quantifies the degree of cognitive decline of the elderly through the local LLM and an underlying disease index for the elderly's underlying disease, and the occurrence of the abnormal sign event A voice analysis-based intelligent senior care system characterized by generating voice characteristic information based on audio signals corresponding to sounds received by the elderly terminal during a preset past time based on a point in time, inputting this information into the local LLM to calculate an anxiety index through the local LLM, calculating a crisis score based on a preset crisis weight corresponding to a specific response type according to the response type information, the calculated anxiety index, and the baseline value, and generating abnormal situation occurrence information and transmitting it to the control server if the calculated crisis score is greater than or equal to a preset threshold. Claim 6 The voice analysis-based intelligent senior care system according to claim 5, wherein the elderly terminal calculates an amplification factor by multiplying the crisis weight corresponding to the specific response type and the anxiety index when the response type information corresponds to any one response type excluding a normal response state or when the anxiety index is greater than or equal to a preset anxiety threshold value, and calculates the crisis score by multiplying the baseline value by an exponential function value with a natural constant as the base and the amplification factor as the exponent. Claim 7 The voice analysis-based intelligent senior care system according to claim 1, wherein when the elderly terminal is switched to the precision tracking mode, it transmits the location information of the elderly to the control server according to a preset second cycle shorter than the first cycle applied in the general tracking mode among the tracking modes, activates BLE (Bluetooth Low Energy) communication, and if the state of communication connection with the control server or the state of receiving an external positioning signal through a location measurement unit configured in the elderly terminal satisfies a preset communication restriction condition, it changes at least one of the scanning cycle or advertising cycle of the BLE communication to a fourth cycle shorter than the third cycle applied in the general tracking mode, and then performs scanning or advertising of the BLE communication according to the fourth cycle, and transmits tracking information including the recent location information of the elderly and identification information of the elderly terminal to a nearby terminal discovered through the BLE communication, so that the tracking information is transmitted to the control server through the nearby terminal. Claim 8 A voice analysis-based intelligent senior care system according to claim 7, wherein the elderly terminal acquires location information of a surrounding terminal that communicates via BLE communication when reception of an external positioning signal through the location measuring unit is not possible in the precision tracking mode, generates location information of the elderly based on the acquired location information of the surrounding terminal, and generates tracking information including the generated location information of the elderly. Claim 9 In claim 7, the control server receives together surrounding terminal identification information for identifying the surrounding terminal from the surrounding terminal transmitting the tracking information and the tracking information for the elderly person transmitted from the elderly terminal to the surrounding terminal via BLE communication; if the elderly person identification information for identifying the elderly terminal included in the received tracking information and the surrounding terminal identification information are inconsistent, the control server determines the surrounding terminal corresponding to the received surrounding terminal identification information as a compensation target; calculates a relay contribution based on at least one of the number of transmissions of tracking information and the size of transmission data based on one or more tracking information received from the surrounding terminal that is the compensation target during a pre-set compensation period through a pre-set compensation calculation algorithm; generates compensation information for the surrounding terminal based on the relay contribution; and transmits the compensation information to a pre-set external compensation management server so that compensation for the surrounding terminal according to the compensation information is provided. Claim 10 The voice analysis-based intelligent senior care system according to claim 1, wherein the elderly terminal generates normal movement pattern information by time period based on movement history according to location information of the elderly accumulated and stored over a past period based on the current time point, compares the current movement path information of the elderly according to changes in the location information of the elderly with the normal movement pattern information by time period, and if the comparison result satisfies a pre-set deviation condition, determines that the elderly has deviated from the normal movement path, generates movement anomaly information, and transmits it to the control server. Claim 11 A voice analysis-based intelligent senior care system according to claim 1, wherein the control server determines a dispatch priority between one or more other elderly people and the elderly person by comparing the wandering risk information received from the elderly person terminal with the wandering risk information received from other elderly person terminals, generates dispatch request information including dispatch location information generated based on the dispatch priority for the elderly person and location information received from the elderly person terminal, and transmits the dispatch request information to a pre-set guardian terminal or a pre-set administrator terminal corresponding to the elderly person. Claim 12 The voice analysis-based intelligent senior care system according to claim 1, wherein the elderly terminal monitors resource status information of the elderly terminal, and when resource status information regarding the usage status or availability status of the operating resources constituting the elderly terminal satisfies a preset offloading condition, transmits voice analysis request information including the audio signal to a preset external voice analysis server via an encrypted communication network, receives analysis result information including the voice characteristic information and the utterance content information calculated by the external voice analysis server based on the audio signal from the external voice analysis server, and generates voice input information including the voice characteristic information and the utterance content information included in the analysis result information and inputs it into the local LLM.

Citation Information

Patent Citations

  • System

    JP2026024405A

  • Apparatus and method for selecting complete local field-specific language model

    JP2026053538A

  • Emergency situation sensing system and emergency situation sensing method

    KR1020230055572A

  • Method for controling caring system for a senior citizen

    KR102742759B1