System

Abstract

A system is provided.SOLUTION: The system of claim 1, wherein the server is further configured to: receive data from the user in real time; transmit the received data to the server; use AI techniques to analyze the received data at the server; detect anomalies based on the analyzing; notify the detected anomalies; and visualize the notified anomalies.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Monitoring the elderly is a social issue, becoming increasingly serious as the aging population continues. Current monitoring systems lack the means to immediately detect and notify abnormalities and dangers in the daily lives of elderly people, making it difficult to respond quickly. For this reason, there is a need to develop a new monitoring system that can effectively monitor the safety and health of elderly people and provide appropriate support. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including a means for acquiring data from users in real time, a means for transmitting the acquired data to a server, a means for using AI technology to analyze the data received by the server, a means for detecting anomalies based on the analysis results, a means for notifying the detected anomalies, and a means for visualizing the notified anomaly data. This enables effective analysis of data from the daily lives of elderly people, prompt notification when an anomaly is detected, and appropriate action to be taken.

[0006] A "user" is a person to be watched over, including elderly people, and is a person to be monitored by the system.

[0007] "Data" refers to all information obtained from the user's biometric and environmental information, such as heart rate, walking patterns, location information, and voice data.

[0008] "Real-time" means that data is collected and processed immediately, with little delay.

[0009] A "means" is a hardware or software component used to realize a particular function.

[0010] A "server" is a computer system that has the ability to receive, store, and analyze data over a network.

[0011] "Acquisition" refers to the act of collecting data using sensors and input devices.

[0012] "Transmitting" refers to the act of transferring data from one device to another.

[0013] "AI technologies" means artificial intelligence technologies, including machine learning and deep learning, for analyzing data and detecting patterns and anomalies.

[0014] An "anomaly" refers to an irregular pattern or dangerous condition that deviates from a user's normal behavior or biometric data.

[0015] "Notification" refers to the act of issuing a warning or alert to the user, family, caregiver, etc. when an abnormality is detected.

[0016] "Visualization" refers to displaying data in the form of charts, graphs, text, etc., making it easy to understand at a glance. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology, and provides a means to detect and notify abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization.

[0039] System program and its processing overview

[0040] 1. Data Collection

[0041] The device (smart glasses) collects data such as the user's heart rate, walking patterns, location information, surrounding sounds, etc. Specifically, the data is collected using the heart rate sensor, accelerometer, gyro sensor, GPS, microphone, etc. built into the smart glasses.

[0042] 2. Data Transmission

[0043] The device sends the collected data to the server in batches at regular intervals. The data is sent over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, etc.

[0044] 3. Data Reception

[0045] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0046] 4. Data Analysis

[0047] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (for example, a sudden increase in heart rate or irregular walking). This analysis is performed in real time, and any abnormalities detected are immediately processed.

[0048] 5. Anomaly detection and notification

[0049] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0050] 6. Data Visualization

[0051] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0052] Specific examples

[0053] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0054] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0055] The terminal packages this data in a batch format and sends it to the server every minute.

[0056] The server receives the data from the device and stores it in a database.

[0057] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0058] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. The device acquires this data and immediately sends it to the server.

[0059] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0060] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0061] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0062] The processing flow will be explained below.

[0063] Program processing steps

[0064] Step 1:

[0065] The device collects user data (heart rate, walking patterns, location information, and ambient sounds) in real time.

[0066] Data is captured by sensors built into the device (heart rate sensor, accelerometer, gyroscope, GPS, microphone, etc.).

[0067] The captured data is temporarily stored in the device's memory.

[0068] Step 2:

[0069] The terminal sends data to the server at regular intervals (for example, every minute).

[0070] The terminal packages the data in a batch format.

[0071] The packaged data is then transmitted over Wi-Fi or cellular networks.

[0072] A secure communication protocol (e.g., HTTPS) is used for transmission.

[0073] Step 3:

[0074] The server receives the data sent from the terminal.

[0075] A dedicated API endpoint receives the data.

[0076] The received raw data is stored in a real-time database.

[0077] Step 4:

[0078] The server analyzes the received data in real time.

[0079] AI models (e.g., machine learning or deep learning models) process the data in real time.

[0080] The received data is compared with existing normal data to detect abnormal patterns.

[0081] Use feedback loops to continuously improve your AI models.

[0082] Step 5:

[0083] If the server detects an abnormality, it will immediately issue a notification.

[0084] Create a notification that includes the type of anomaly, the user's current location, and detailed data (e.g., changes in heart rate, irregular walking patterns).

[0085] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[0086] A voice message is also sent to the device to notify the user of the abnormality.

[0087] Step 6:

[0088] The server provides a dashboard for visually displaying the collected data.

[0089] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[0090] Current real-time data, past abnormal data, and statistical information are displayed on the dashboard.

[0091] Visualize users' data trends and anomaly history to enable proactive action.

[0092] Through this process, the system can monitor the safety of the elderly in real time and respond quickly if necessary.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Monitoring the daily lives of the elderly requires the collection of biometric and environmental data in real time and the prompt detection and notification of abnormalities. However, existing technologies do not seamlessly collect, transmit, analyze, and notify data, which can lead to insufficient efforts to ensure the safety of the elderly. Furthermore, the visual display of data is insufficient to enable family members and caregivers to respond quickly to abnormalities. This leaves the elderly unable to be ensured safely and also creates a sense of security for family members and caregivers.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for acquiring biometric data and environmental data from the user in real time, means for transmitting the acquired data to the server at regular intervals, means for storing the data received by the server in a real-time database, means for detecting abnormalities based on the received data using AI technology, means for notifying the user and a third party of the detected abnormality, and means for visually displaying the notified abnormal data. This makes it possible to monitor the biometric data and environmental data of the elderly in real time and to respond quickly when an abnormality is detected.

[0098] "User" means a subject who uses the system to provide biometric and environmental data.

[0099] "Biometric data" refers to data that indicates the physiological state of the user, such as heart rate or walking patterns.

[0100] "Environmental data" is data that indicates the user's surrounding environment, such as location information and surrounding audio data.

[0101] "Real-time" means that data is processed from the moment it is collected with little delay.

[0102] A "terminal" is a device for collecting data from a user, specifically smart glasses.

[0103] "Server" means a central processing unit for receiving and analyzing collected data.

[0104] "Fixed interval" refers to the time setting for processing and transmitting data at regular time intervals.

[0105] "Batch format" refers to a method of sending data in batches at regular intervals.

[0106] "AI technology" is a technology that uses machine learning models to analyze data and detect anomalies.

[0107] "Abnormal" refers to data patterns that deviate from normal patterns, including, for example, a sudden increase in heart rate or irregular gait.

[0108] "Notification" refers to the act of issuing a warning to the user and third parties when an abnormality is detected.

[0109] "Visually displaying" refers to showing data in a visual manner, such as in a graph or chart.

[0110] This invention is a system for monitoring elderly people, which collects biometric and environmental data from users in real time, transmits it to a server, analyzes the data using AI technology, and detects and notifies users of abnormalities. This system is mainly composed of a terminal (smart glasses) and a server, and operates as follows:

[0111] Hardware and Software Configuration

[0112] The smart glasses used as terminals have the following built-in devices:

[0113] Heart rate sensor: Measures the user's heart rate.

[0114] Acceleration sensor: Recognizes the user's walking pattern.

[0115] Gyro sensor: Measures walking stability.

[0116] GPS: Obtains the user's location information.

[0117] Microphone: Collects surrounding audio data.

[0118] This data is collected in real time by a data processing unit in the smart glasses and packaged in batches at regular intervals, after which the collected data is securely transmitted to a server via Wi-Fi or cellular networks using the HTTPS protocol.

[0119] The server receives, stores, and analyzes the collected data. The server has the following functions:

[0120] API endpoint: Responsible for receiving data.

[0121] Real-time database: Stores incoming data without loss.

[0122] AI model: Detects anomalies based on incoming data. The AI ​​model uses machine learning algorithms to compare normal patterns with current data and detect abnormal patterns in real time.

[0123] Specific examples

[0124] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0125] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0126] The terminal packages this data in a batch format and sends it to the server every minute.

[0127] The server receives the data from the device and stores it in a database.

[0128] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0129] The device captures data on the sudden rise in heart rate to 150 bpm and the walking pattern becoming unstable, and immediately sends it to the server.

[0130] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0131] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0132] Prompt Sentence Examples

[0133] "We are developing a monitoring system for the elderly. This system uses smart glasses to collect data such as heart rate, walking patterns, location information, and surrounding sounds, and sends it to a server. The server uses an AI model to analyze the data in real time and notify users if an abnormality is detected. Please briefly explain the system's processing steps, taking into account the specific example below."

[0134] This invention allows for real-time monitoring of a user's biometric and environmental data, enabling rapid response when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to family members and caregivers.

[0135] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0136] Step 1:

[0137] Data collection

[0138] The terminal acquires the user's biometric data and environmental data in real time.

[0139] Input: User's heart rate, walking patterns, location, and ambient audio.

[0140] How it works: The smart glasses have a built-in heart rate sensor to measure your heart rate, an accelerometer and gyroscope to recognize your walking patterns, a GPS to acquire your location, and a microphone to collect surrounding audio data.

[0141] Output: A package of collected biometric and environmental data.

[0142] Step 2:

[0143] Data transmission

[0144] The terminal transmits the collected data to the server at regular intervals.

[0145] Input: A package of collected biometric and environmental data.

[0146] What it does: It packages collected data into batches and sends them to a server over Wi-Fi or a cellular network, using a secure communication protocol such as HTTPS.

[0147] Output: The data package sent to the server.

[0148] Step 3:

[0149] Data reception

[0150] The server receives the data sent from the terminal.

[0151] Input: Data package sent from the terminal.

[0152] Specific operation: Receive data at the API endpoint and store it in the real-time database.

[0153] Output: Data stored in the real-time database.

[0154] Step 4:

[0155] Data analysis

[0156] The server uses AI technology to detect anomalies based on the received data.

[0157] Input: Data stored in the real-time database.

[0158] What it does: Uses AI models to compare normal patterns with current data and detect abnormal patterns.

[0159] Output: Anomaly detection results.

[0160] Step 5:

[0161] Anomaly detection and notification

[0162] If the server detects an abnormality, it notifies the user and third parties.

[0163] Input: Anomaly detection results.

[0164] Specific operation: If an abnormality is detected, a notification will be issued to family members or caregivers via a dedicated app, SMS, email, phone call, etc. A voice message will also be sent to the device to notify the user of the abnormality.

[0165] Output: Notice to users and third parties.

[0166] Step 6:

[0167] Data Visualization

[0168] The server visually displays the notified abnormal data.

[0169] Input: Data stored in the real-time database and anomaly detection results.

[0170] What it does: Data is displayed visually on a dashboard, allowing family members and caregivers to view detailed data, abnormality history, and real-time raw data. Accessible via a web browser or dedicated app.

[0171] Output: Visually displayed data.

[0172] (Application example 1)

[0173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0174] Conventional technologies for monitoring elderly people have difficulty acquiring and analyzing data in real time, making it difficult to respond quickly when an abnormality occurs. Additionally, there is a lack of a means to comprehensively monitor the security status of elderly people and immediately notify family members or caregivers when an abnormality is detected.

[0175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0176] In this invention, the server includes means for acquiring user data in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting anomalies based on the analysis results, means for notifying the detected anomalies, means for visualizing the notified anomaly data, data analysis means for monitoring the security status of the user and identifying anomalies, and communication means for immediately notifying the user of a response when an anomaly is detected. This enables advanced real-time monitoring of the health and security status of elderly people, and enables prompt and appropriate response when an anomaly occurs.

[0177] "User" refers to the person wearing the smart glasses or other device and providing the data.

[0178] "Real-time acquisition means" refers to methods of collecting data instantly using sensors or devices.

[0179] "Means of sending to a server" refers to a method of transferring collected data to a remote server using Wi-Fi, cellular networks, etc.

[0180] "AI technology for analyzing data received by the server" refers to a method of processing data and extracting information using machine learning models and data analysis techniques.

[0181] "Means for detecting anomalies based on analysis results" refers to a method for identifying data that deviates from normal patterns and determining that the data is abnormal.

[0182] "Means for notifying detected abnormalities" refers to communication means for notifying family members or caregivers when an abnormality occurs.

[0183] "Means for visualizing notified abnormal data" refers to methods for displaying abnormal data in an easy-to-understand manner using dashboards and graphs.

[0184] "Data analysis means for monitoring users' security status and identifying anomalies" refers to technology for analyzing the status of elderly people and identifying security risks.

[0185] "Communication means for immediately notifying the user when an abnormality is detected" refers to a system or method for quickly notifying the user when an abnormality occurs.

[0186] This invention provides a system for watching over elderly people and monitoring security situations. The configuration of the system and how it is implemented will be specifically described below.

[0187] 1. System Configuration

[0188] Hardware

[0189] Smart glasses: Equipped with a heart rate sensor, accelerometer, gyroscope, GPS, and microphone, these sensors are used to collect data in real time.

[0190] Server: A cloud server (e.g. AWS) that provides an API endpoint for receiving data.

[0191] software

[0192] Smart Glasses App: Uses Python scripts to collect data from sensors and send it to the server.

[0193] Server API: API endpoints built using a web framework such as Flask.

[0194] Data analysis system: Uses AI technologies such as TensorFlow and PyTorch to analyze data in real time.

[0195] Notification system: Use APIs such as Twilio to send emergency notifications via SMS or email.

[0196] 2. Data collection and transmission

[0197] Smart glasses collect the user's heart rate, walking patterns, location information, and voice data in real time. The collected data is transmitted to a server at regular intervals via Wi-Fi or cellular networks. The transmitted data includes time information, sensor values, location information, etc.

[0198] 3. Receipt and analysis of data

[0199] The server receives the data sent from the smart glasses and stores it in a real-time database. The stored data is analyzed using AI technology (e.g., TensorFlow, PyTorch). If an abnormality is detected by comparing it with normal patterns, a notification is sent immediately to family members or caregivers.

[0200] 4. Notification of abnormalities

[0201] If an abnormality is detected, detailed information, including security risks, is sent to family members or caregivers via emergency notification. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., a sudden increase in heart rate or irregular gait). Notification methods include a dedicated app, SMS, email, and phone call.

[0202] 5. Data Visualization

[0203] The server provides a dashboard to visualize the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0204] Specific examples

[0205] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0206] The smart glasses capture heart rate (e.g., 70 bpm), walking patterns (stationary), location information (in your living room), and ambient sounds (audio from the TV).

[0207] The acquired data is sent to the server every minute.

[0208] The server receives the data and analyzes it in real time. Since the patient is in a relaxed state, it is determined to be normal.

[0209] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. This data is immediately sent to the server.

[0210] The server detects this abnormality, determines that "your heart rate is rising sharply and there is a high possibility of you falling," and immediately sends a notification to your family. It also sends a voice notification to the user through the smart glasses saying, "Your heart rate is rising. Please sit down and rest."

[0211] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0212] Example prompts for generative AI models

[0213] Create an anomaly detection scenario for an elderly security monitoring app. Data collected includes heart rate, walking patterns, and location. Detail how to respond and what notifications are sent when an anomaly is detected.

[0214] This will realize an effective monitoring system to ensure the safety and security of the elderly.

[0215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0216] Step 1:

[0217] The device collects the user's heart rate, walking patterns, location information, and voice data in real time. Specifically, it uses the device's built-in heart rate sensor, accelerometer, gyroscope, GPS, and microphone to collect this sensor data. The input is the user's biometric information and surrounding environmental sounds, and the output is digital sensor data.

[0218] Step 2:

[0219] The terminal packages the collected sensor data into a batch format at regular intervals (for example, every minute) and sends it to the server. The transmission is done over Wi-Fi or cellular networks using a secure communication protocol such as HTTPS. The input is the sensor data, and the output is an encrypted data packet.

[0220] Step 3:

[0221] The server receives data sent from the device using a dedicated API endpoint and stores the received data in a real-time database. The input is the encrypted data packet, and the output is the decoded sensor data.

[0222] Step 4:

[0223] The server analyzes the data stored in the real-time database using an AI model. Machine learning libraries such as TensorFlow and PyTorch are used for the analysis to detect abnormal patterns from the data. The input is sensor data, and the output is the anomaly detection results and detailed information.

[0224] Step 5:

[0225] If the server detects an abnormality based on the analysis results, it issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high possibility of falling). Notifications are sent via SMS, email, phone, etc. using external communication APIs such as Twilio. The input is the anomaly detection result and detailed information, and the output is a notification message.

[0226] Step 6:

[0227] If an abnormality is detected, the device will notify the user by voice. Specifically, it uses voice synthesis technology to issue a message such as "Your heart rate is increasing. Please sit down and rest." The input is the result of the anomaly detection and the text of the voice message, and the output is the voice message.

[0228] Step 7:

[0229] The server provides a dashboard for visualizing the collected data, which can be accessed by family members or caregivers via a web browser or a dedicated app. The dashboard displays real-time data, abnormality history, and detailed data. The input is sensor data and analysis results, and the output is a graphical dashboard.

[0230] This will enable real-time monitoring of the health and security status of elderly people and rapid response in the event of an abnormality.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] This invention is a system for monitoring elderly people. It collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization. Furthermore, this invention combines an emotion engine to recognize the user's emotional state, achieving more comprehensive monitoring.

[0233] System program and its processing overview

[0234] 1. Data Collection

[0235] The device (smart glasses) collects the user's heart rate, walking patterns, location information, surrounding voices, and facial expressions in real time. Specifically, data is collected using the smart glasses' built-in heart rate sensor, accelerometer, gyro sensor, GPS, microphone, camera, etc. The user's voice data and facial expression data are also captured.

[0236] 2. Data Transmission

[0237] The device sends the collected data to the server in batches at regular intervals over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, voice data, and facial expression data.

[0238] 3. Data Reception

[0239] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0240] 4. Data Analysis

[0241] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and if an abnormality is detected, immediate action is taken.

[0242] 5. Anomaly detection and notification

[0243] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, sadness emotional state). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0244] 6. Data Visualization

[0245] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app. The dashboard also visualizes the user's emotional state, showing emotional fluctuations and trends.

[0246] Specific examples

[0247] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0248] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), ambient sounds (audio from the television), and facial expression data (relaxed facial expression).

[0249] The terminal packages this data in a batch format and sends it to the server every minute.

[0250] The server receives the data from the device and stores it in a database.

[0251] The server uses an AI model and emotion engine to analyze the received data in real time, determining that the state is normal because the person is relaxed.

[0252] The device then detects a sudden increase in heart rate to 150 bpm and an unstable walking pattern, and immediately sends this data to the server. The emotion engine then detects "fear."

[0253] The server detects this as an abnormality, determines that "your heart rate has suddenly increased, you are at high risk of falling, and you are in a state of fear," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is increasing. Please sit down and rest."

[0254] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0255] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when abnormalities, including emotional states, are detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0256] The processing flow will be explained below.

[0257] Program processing steps

[0258] Step 1:

[0259] The device (smart glasses) collects user data (heart rate, walking patterns, location information, ambient sounds, facial expressions) in real time.

[0260] A heart rate sensor measures the user's heart rate and detects abnormal heart rate variations.

[0261] Acceleration and gyro sensors analyze the user's walking pattern and detect falls and unsteady walking.

[0262] GPS identifies the user's current location.

[0263] A microphone captures the sounds around the user and collects data to analyze tone and emotion.

[0264] A camera captures the user's facial expressions and collects facial expression data.

[0265] Step 2:

[0266] The terminal periodically (for example, every minute) sends the collected data to the server.

[0267] The terminal packages the data in a batch format.

[0268] Data is sent to a server using Wi-Fi or cellular networks.

[0269] Ensure data security by using secure communication protocols (e.g., HTTPS).

[0270] Step 3:

[0271] The server receives the data sent from the terminal.

[0272] Receive data using a dedicated API endpoint.

[0273] The received raw data is stored in a real-time database.

[0274] Step 4:

[0275] The server analyzes the received data in real time.

[0276] It uses AI models to detect abnormalities in heart rate and walking patterns.

[0277] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., joy, anger, sadness, fear).

[0278] Comparison with existing databases is performed to identify normal data and outliers.

[0279] Step 5:

[0280] If the server detects an abnormality based on the data analysis results, it will immediately issue a notification.

[0281] The server creates a notification that includes the type of anomaly, the user's current location, and detailed data (heart rate, walking pattern, emotional state, etc.).

[0282] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[0283] A voice message is also sent to the device to notify the user of the abnormality.

[0284] Step 6:

[0285] The server provides a dashboard to visually display the collected data.

[0286] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[0287] The dashboard displays current real-time data, historical anomaly data, statistical information, and emotional state.

[0288] Allow users to view data trends and anomaly history through a dashboard.

[0289] As a concrete example, the following occurs while a user is relaxing at home:

[0290] The device captures heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (TV audio), and facial expression data (relaxed facial expression).

[0291] The device sends this data to the server every minute.

[0292] The server receives the data and stores it in a database.

[0293] The server analyzes the data in real time using an AI model and emotion engine to determine whether the person is in a normal relaxed state.

[0294] Suddenly, the heart rate rises to 150 bpm, and data is sent showing an unstable walking pattern, along with data that the emotion engine detects as "fear."

[0295] The server determines this to be an abnormality and sends a notification to the family stating, "Heart rate is rising rapidly, there is a high possibility of falling, and the patient is in a state of fear."

[0296] The device will notify the user by voice, "Your heart rate is increasing. Please sit down and rest."

[0297] The server displays detailed data on abnormalities on a dashboard, allowing family members and caregivers to check the data in real time.

[0298] In this way, the present invention can monitor the daily life of the elderly in real time and respond quickly, including their emotional state, if necessary.

[0299] Example 2

[0300] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0301] Conventional systems for monitoring elderly people rely on limited biometric information acquisition and anomaly detection, and lack comprehensive monitoring that takes into account the user's emotional state. This makes it difficult to respond quickly and accurately when an abnormality is detected, posing a major challenge to ensuring the safety of elderly people and the peace of mind of family members and caregivers.

[0302] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0303] In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using artificial intelligence technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormalities, means for visualizing the notified abnormal data, means for collecting heart rate, walking pattern, location information, voice data, and facial expression data, means for analyzing the voice data and facial expression data using an emotion engine to recognize the emotional state, and means for issuing notifications to the user and third parties based on the abnormality and emotional state. This makes it possible to monitor not only the user's biometric information but also their emotional state in real time, enabling quick and accurate response when an abnormality occurs.

[0304] "User" refers to individuals, especially elderly people, who use the system.

[0305] "Data" refers to information collected from the user, including heart rate, walking patterns, location information, voice data, and facial expression data.

[0306] "Real-time" refers to the immediacy in time that data is acquired and processed immediately.

[0307] A "terminal" is a device for acquiring and transmitting data, specifically referring to smart glasses.

[0308] "Server" refers to a computer system that receives, stores, analyzes data sent from a terminal, and issues notifications as needed.

[0309] "Artificial intelligence technology" refers to technology used to analyze data on servers, including machine learning and deep learning.

[0310] "Emotion engine" refers to a system that analyzes voice data and facial expression data to recognize a user's emotional state.

[0311] "Abnormal" refers to a state that deviates from normal data patterns or a potential problem related to user safety.

[0312] "Notification" refers to warnings and information sent to users and third parties when the server detects an abnormality.

[0313] "Visualization" refers to presenting data as graphs or tables to make it easier to read.

[0314] "Third parties" refer to significant parties with an interest in the user's condition, such as the user's family or caregivers.

[0315] MODE FOR CARRYING OUT THE INVENTION

[0316] The present invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. Specific embodiments of the present invention will be described below.

[0317] Data collection

[0318] The user wears the smart glasses and goes about their daily life. The smart glasses collect real-time data on their heart rate, walking patterns, location information, surrounding sounds, and facial expressions. The hardware used for this includes a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. For example, if a user is in their living room at home, their heart rate is recorded as 70 bpm, they are standing still, and they can hear the TV.

[0319] Data transmission

[0320] The device (smart glasses) packages the collected data into batches at regular intervals (e.g., every minute) and sends them to a server. This transmission is done via Wi-Fi or cellular networks, and the data is sent using a secure communication protocol (e.g., HTTPS). Examples of data include heart rate, walking patterns, location information, voice data, and facial expression data.

[0321] Data reception

[0322] The server receives data sent from the device via a dedicated API endpoint and stores it in a real-time database, ensuring continuous data storage and availability for later analysis.

[0323] Data analysis

[0324] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and any abnormalities detected are immediately addressed.

[0325] Anomaly detection and notification

[0326] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"), etc. Notifications are sent via a dedicated app, SMS, email, phone call, etc. A voice message is also sent to the device to notify the user of the abnormality.

[0327] Data Visualization

[0328] The server provides a dashboard to visually display the collected data. Family members and caregivers can use this dashboard to check the user's detailed data, abnormality history, and real-time status. The dashboard can be accessed from a web browser or a dedicated app, and heart rate fluctuations, movement paths, emotional trends, and other data are visualized in graphs and tables.

[0329] Specific examples

[0330] For example, if a user is relaxing at home, the smart glasses will collect their heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent to the server every minute and analyzed in real time. If the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine will detect "fear." Based on this, the server will determine that "the heart rate has suddenly risen, there is a high risk of falling, and the user is in a state of fear," and will notify family members and the user via voice notification.

[0331] Example prompts for generative AI models

[0332] "Please explain the specific data flow for an elderly care system that collects and analyzes heart rate, walking patterns, location information, voice data, and facial expression data in real time, and notifies users when an abnormality is detected."

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] Step 1:

[0335] The device (smart glasses) collects the user's heart rate, walking pattern, location information, voice data, and facial expression data in real time. The device collects data using a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. The input is raw data from the sensors, and the output is a collected composite data package. Specifically, the data collected is the user's heart rate of 70 bpm, walking pattern of standing still, location information of the living room, ambient sound of the TV, and facial expression of a relaxed state.

[0336] Step 2:

[0337] The terminal packages the collected data in a batch format at regular intervals (for example, every minute) and sends it to the server. Data is sent over Wi-Fi or cellular networks, using the secure HTTPS communication protocol. The input is the collected composite data package, and the output is an encrypted data packet. Specifically, the data from each sensor is combined into a single packet and sent to the server via the Internet.

[0338] Step 3:

[0339] The server receives data sent from the device via a dedicated API endpoint. The received data is stored in a real-time database to prevent data loss. The input is an encrypted data packet, and the output is structured data stored in the database. Specifically, the server decrypts the received data packet and records the user's heart rate, location information, etc. in the real-time database.

[0340] Step 4:

[0341] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data to detect abnormal patterns. The emotion engine analyzes the voice data and facial expression data to recognize the emotional state. The input is raw data obtained from the real-time database, and the output is events recognized as abnormal and the detected emotional state. Specifically, the system inputs heart rate and walking patterns into the AI ​​model to determine whether there are any abnormalities. Additionally, the voice data and facial expression data are input into the emotion engine to analyze whether the user is in a state of fear.

[0342] Step 5:

[0343] When an abnormality is detected, the server issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"). The device also sends a voice message to the user. The input is the analysis results of the AI ​​model and emotion engine, and the output is the notification message. Specifically, the notification is sent via a dedicated app, SMS, email, phone call, etc., and the device issues a voice notification saying, "Your heart rate is rising. Please sit down and rest."

[0344] Step 6:

[0345] The server provides a dashboard for visualizing data. Family members and caregivers can check the user's detailed data, abnormality history, and real-time status through the dashboard. The input is data stored in the real-time database, and the output is the visualized dashboard. Specifically, it displays heart rate fluctuations, movement routes, emotional trends, etc. in graphs and tables.

[0346] (Application example 2)

[0347] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0348] In elderly care systems, simply collecting physical data such as heart rate and walking patterns is difficult to accurately grasp the individual's condition. Furthermore, there is insufficient means for quickly and accurately notifying family members or caregivers when an abnormality occurs. Furthermore, there is a need for a system that can monitor emotional fluctuations and respond accordingly.

[0349] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormality, means for visualizing the notified abnormal data, means for acquiring surrounding voice data and facial expression data using a smart device, means for analyzing the emotional state based on AI technology, and means for quickly issuing a notification based on the abnormality and the emotional state. This makes it possible to monitor not only the physical state of the elderly person but also their emotional state in real time and to quickly and accurately notify the elderly person of any abnormalities.

[0350] "User" refers to an individual who provides data and is monitored by the system.

[0351] "Means for acquiring data in real time" refers to technology and devices for instantly collecting heart rate, walking patterns, location information, voice data, facial expression data, and the like from a user.

[0352] "Means for transmitting data to the server" refers to the communication technologies and protocols for securely and quickly transmitting acquired data to the server.

[0353] "AI technology for analyzing data received by the server" refers to artificial intelligence technology for analyzing data sent to the server and detecting anomalies and patterns.

[0354] "Means for detecting anomalies" refers to the techniques and processes that use AI technology to detect patterns or symptoms that deviate from normal conditions based on received data.

[0355] "Means for notifying abnormalities" refers to notification functions and means for notifying the user, family, or caregivers of detected abnormalities.

[0356] "Means for visualizing notified abnormal data" refers to technology and devices for visually displaying abnormal data in an easy-to-understand manner.

[0357] "Smart device" refers to an electronic device worn or carried by a user for collecting and transmitting data.

[0358] "Means for acquiring voice data and facial expression data" refers to technologies and sensors for capturing and recording the user's voice and facial expressions.

[0359] "Means for analyzing emotional state" refers to techniques and processes that analyze acquired voice data and facial expression data to determine the user's psychological and emotional state.

[0360] "Means for issuing prompt notifications" refers to technologies and functions for sending necessary warnings and information to users without delay based on detected abnormalities and emotional states.

[0361] A system for implementing this invention utilizes a smart device (e.g., smart glasses) worn by a user to acquire user data in real time and transmit it to a server. The data includes heart rate, walking patterns, location information, voice data, facial expression data, etc., and AI technology and an emotion engine analyze the data for abnormalities and emotional states.

[0362] Smart devices are equipped with heart rate sensors, accelerometers, gyroscopes, GPS, microphones, cameras, etc., and use these sensors to collect user data in real time. This data is sent to a server in batch format at regular intervals. The data is sent via Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS).

[0363] The server receives data sent from the device and stores it in a real-time database. The received data is analyzed in real time using an AI model (e.g., TensorFlow) and an emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., sudden increases in heart rate, unstable walking patterns, and emotional fluctuations). The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear).

[0364] If an abnormality is detected, the server immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, emotional state sadness, etc.). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0365] The server also provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard, accessible via a web browser or a dedicated app, also visualizes the user's emotional state, displaying emotional fluctuations and trends.

[0366] As a concrete example, consider a situation where a user is relaxing at home. Wearing smart glasses, the device collects the user's heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent in batch format to a server, where it is analyzed in real time using AI technology and an emotion engine. Because the user is in a relaxed state, this is deemed normal. However, if the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine detects "fear." The server determines this situation as abnormal and issues a notification to family members stating, "The heart rate has risen sharply, there is a risk of falling, and the emotional state is fear." A voice notification can also be sent to the user via the device, enabling a prompt response.

[0367] An example of a prompt sentence might be:

[0368] "User has heart rate of 150 bpm, potential fall, and high-pitched voice tone indicating fear. Detailed location is outside of home. Please take appropriate action."

[0369] This will enable real-time monitoring of not only the physical condition of the elderly but also their emotional state, enabling prompt and accurate responses.

[0370] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0371] Step 1:

[0372] The device collects user data in real time. Inputs include data from the heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. Using these sensors, the device obtains the user's heart rate, walking patterns, location information, voice data, and facial expression data. The output is raw data that is temporarily stored on the device and later sent to the server.

[0373] Step 2:

[0374] The device sends the acquired data to the server. The input includes the raw data collected in step 1. This data is packaged in batches at regular time intervals and sent to the server over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). As an output, a status indicating that the transmission is complete is returned to the device.

[0375] Step 3:

[0376] The server receives the data sent from the devices. The input includes the raw data package sent from step 2. The server stores this in a real-time database. The output is a dataset stored in the database. This ensures that the data is preserved without loss and can be used for further analysis.

[0377] Step 4:

[0378] The server analyzes the data received using an AI model and emotion engine. The input includes a dataset obtained from a real-time database. This dataset is fed into an AI model (e.g., TensorFlow) that compares the current data with previously learned normal patterns. The emotion engine analyzes voice and facial expression data to recognize the user's emotional state. The output is an analysis result that includes abnormal patterns and the user's emotional state.

[0379] Step 5:

[0380] The server detects anomalies based on the analysis results and issues a notification if necessary. The input includes the analysis results obtained in step 4. If an anomaly is detected based on this, the server immediately issues a notification to family members or caregivers. Notification methods include a dedicated app, SMS, email, and phone. As an output, an anomaly notification is sent to family members or caregivers. A voice message is also sent to the device to notify the user of the anomaly.

[0381] Step 6:

[0382] The server provides a dashboard to visually display the collected data. Inputs include real-time and historical data. The server visualizes this data in an easy-to-understand manner and makes it accessible to family members and caregivers via a web browser or dedicated app. Outputs include detailed data, abnormality history, and real-time raw data, all visualized on the dashboard. The user's emotional state is also visualized, showing emotional fluctuations and trends.

[0383] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0384] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0385] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0386] [Second embodiment]

[0387] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0388] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0389] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0390] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0391] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0392] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0393] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0394] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0395] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0396] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0397] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0398] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0399] This invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology, and provides a means to detect and notify abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization.

[0400] System program and its processing overview

[0401] 1. Data Collection

[0402] The device (smart glasses) collects data such as the user's heart rate, walking patterns, location information, surrounding sounds, etc. Specifically, the data is collected using the heart rate sensor, accelerometer, gyro sensor, GPS, microphone, etc. built into the smart glasses.

[0403] 2. Data Transmission

[0404] The device sends the collected data to the server in batches at regular intervals. The data is sent over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, etc.

[0405] 3. Data Reception

[0406] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0407] 4. Data Analysis

[0408] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (for example, a sudden increase in heart rate or irregular walking). This analysis is performed in real time, and any abnormalities detected are immediately processed.

[0409] 5. Anomaly detection and notification

[0410] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0411] 6. Data Visualization

[0412] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0413] Specific examples

[0414] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0415] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0416] The terminal packages this data in a batch format and sends it to the server every minute.

[0417] The server receives the data from the device and stores it in a database.

[0418] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0419] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. The device acquires this data and immediately sends it to the server.

[0420] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0421] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0422] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0423] The processing flow will be explained below.

[0424] Program processing steps

[0425] Step 1:

[0426] The device collects user data (heart rate, walking patterns, location information, and ambient sounds) in real time.

[0427] Data is captured by sensors built into the device (heart rate sensor, accelerometer, gyroscope, GPS, microphone, etc.).

[0428] The captured data is temporarily stored in the device's memory.

[0429] Step 2:

[0430] The terminal sends data to the server at regular intervals (for example, every minute).

[0431] The terminal packages the data in a batch format.

[0432] The packaged data is then transmitted over Wi-Fi or cellular networks.

[0433] A secure communication protocol (e.g., HTTPS) is used for transmission.

[0434] Step 3:

[0435] The server receives the data sent from the terminal.

[0436] A dedicated API endpoint receives the data.

[0437] The received raw data is stored in a real-time database.

[0438] Step 4:

[0439] The server analyzes the received data in real time.

[0440] AI models (e.g., machine learning or deep learning models) process the data in real time.

[0441] The received data is compared with existing normal data to detect abnormal patterns.

[0442] Use feedback loops to continuously improve your AI models.

[0443] Step 5:

[0444] If the server detects an abnormality, it will immediately issue a notification.

[0445] Create a notification that includes the type of anomaly, the user's current location, and detailed data (e.g., changes in heart rate, irregular walking patterns).

[0446] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[0447] A voice message is also sent to the device to notify the user of the abnormality.

[0448] Step 6:

[0449] The server provides a dashboard for visually displaying the collected data.

[0450] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[0451] Current real-time data, past abnormal data, and statistical information are displayed on the dashboard.

[0452] Visualize users' data trends and anomaly history to enable proactive action.

[0453] Through this process, the system can monitor the safety of the elderly in real time and respond quickly if necessary.

[0454] Example 1

[0455] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0456] Monitoring the daily lives of the elderly requires the collection of biometric and environmental data in real time and the prompt detection and notification of abnormalities. However, existing technologies do not seamlessly collect, transmit, analyze, and notify data, which can lead to insufficient efforts to ensure the safety of the elderly. Furthermore, the visual display of data is insufficient to enable family members and caregivers to respond quickly to abnormalities. This leaves the elderly unable to be ensured safely and also creates a sense of security for family members and caregivers.

[0457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0458] In this invention, the server includes means for acquiring biometric data and environmental data from the user in real time, means for transmitting the acquired data to the server at regular intervals, means for storing the data received by the server in a real-time database, means for detecting abnormalities based on the received data using AI technology, means for notifying the user and a third party of the detected abnormality, and means for visually displaying the notified abnormal data. This makes it possible to monitor the biometric data and environmental data of the elderly in real time and to respond quickly when an abnormality is detected.

[0459] "User" means a subject who uses the system to provide biometric and environmental data.

[0460] "Biometric data" refers to data that indicates the physiological state of the user, such as heart rate or walking patterns.

[0461] "Environmental data" is data that indicates the user's surrounding environment, such as location information and surrounding audio data.

[0462] "Real-time" means that data is processed from the moment it is collected with little delay.

[0463] A "terminal" is a device for collecting data from a user, specifically smart glasses.

[0464] "Server" means a central processing unit for receiving and analyzing collected data.

[0465] "Fixed interval" refers to the time setting for processing and transmitting data at regular time intervals.

[0466] "Batch format" refers to a method of sending data in batches at regular intervals.

[0467] "AI technology" is a technology that uses machine learning models to analyze data and detect anomalies.

[0468] "Abnormal" refers to data patterns that deviate from normal patterns, including, for example, a sudden increase in heart rate or irregular gait.

[0469] "Notification" refers to the act of issuing a warning to the user and third parties when an abnormality is detected.

[0470] "Visually displaying" refers to showing data in a visual manner, such as in a graph or chart.

[0471] This invention is a system for monitoring elderly people, which collects biometric and environmental data from users in real time, transmits it to a server, analyzes the data using AI technology, and detects and notifies users of abnormalities. This system is mainly composed of a terminal (smart glasses) and a server, and operates as follows:

[0472] Hardware and Software Configuration

[0473] The smart glasses used as terminals have the following built-in devices:

[0474] Heart rate sensor: Measures the user's heart rate.

[0475] Acceleration sensor: Recognizes the user's walking pattern.

[0476] Gyro sensor: Measures walking stability.

[0477] GPS: Obtains the user's location information.

[0478] Microphone: Collects surrounding audio data.

[0479] This data is collected in real time by a data processing unit in the smart glasses and packaged in batches at regular intervals, after which the collected data is securely transmitted to a server via Wi-Fi or cellular networks using the HTTPS protocol.

[0480] The server receives, stores, and analyzes the collected data. The server has the following functions:

[0481] API endpoint: Responsible for receiving data.

[0482] Real-time database: Stores incoming data without loss.

[0483] AI model: Detects anomalies based on incoming data. The AI ​​model uses machine learning algorithms to compare normal patterns with current data and detect abnormal patterns in real time.

[0484] Specific examples

[0485] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0486] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0487] The terminal packages this data in a batch format and sends it to the server every minute.

[0488] The server receives the data from the device and stores it in a database.

[0489] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0490] The device captures data on the sudden rise in heart rate to 150 bpm and the walking pattern becoming unstable, and immediately sends it to the server.

[0491] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0492] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0493] Prompt Sentence Examples

[0494] "We are developing a monitoring system for the elderly. This system uses smart glasses to collect data such as heart rate, walking patterns, location information, and surrounding sounds, and sends it to a server. The server uses an AI model to analyze the data in real time and notify users if an abnormality is detected. Please briefly explain the system's processing steps, taking into account the specific example below."

[0495] This invention allows for real-time monitoring of a user's biometric and environmental data, enabling rapid response when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to family members and caregivers.

[0496] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0497] Step 1:

[0498] Data collection

[0499] The terminal acquires the user's biometric data and environmental data in real time.

[0500] Input: User's heart rate, walking patterns, location, and ambient audio.

[0501] How it works: The smart glasses have a built-in heart rate sensor to measure your heart rate, an accelerometer and gyroscope to recognize your walking patterns, a GPS to acquire your location, and a microphone to collect surrounding audio data.

[0502] Output: A package of collected biometric and environmental data.

[0503] Step 2:

[0504] Data transmission

[0505] The terminal transmits the collected data to the server at regular intervals.

[0506] Input: A package of collected biometric and environmental data.

[0507] What it does: It packages collected data into batches and sends them to a server over Wi-Fi or a cellular network, using a secure communication protocol such as HTTPS.

[0508] Output: The data package sent to the server.

[0509] Step 3:

[0510] Data reception

[0511] The server receives the data sent from the terminal.

[0512] Input: Data package sent from the terminal.

[0513] Specific operation: Receive data at the API endpoint and store it in the real-time database.

[0514] Output: Data stored in the real-time database.

[0515] Step 4:

[0516] Data analysis

[0517] The server uses AI technology to detect anomalies based on the received data.

[0518] Input: Data stored in the real-time database.

[0519] What it does: Uses AI models to compare normal patterns with current data and detect abnormal patterns.

[0520] Output: Anomaly detection results.

[0521] Step 5:

[0522] Anomaly detection and notification

[0523] If the server detects an abnormality, it notifies the user and third parties.

[0524] Input: Anomaly detection results.

[0525] Specific operation: If an abnormality is detected, a notification will be issued to family members or caregivers via a dedicated app, SMS, email, phone call, etc. A voice message will also be sent to the device to notify the user of the abnormality.

[0526] Output: Notice to users and third parties.

[0527] Step 6:

[0528] Data Visualization

[0529] The server visually displays the notified abnormal data.

[0530] Input: Data stored in the real-time database and anomaly detection results.

[0531] What it does: Data is displayed visually on a dashboard, allowing family members and caregivers to view detailed data, abnormality history, and real-time raw data. Accessible via a web browser or dedicated app.

[0532] Output: Visually displayed data.

[0533] (Application example 1)

[0534] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0535] Conventional technologies for monitoring elderly people have difficulty acquiring and analyzing data in real time, making it difficult to respond quickly when an abnormality occurs. Additionally, there is a lack of a means to comprehensively monitor the security status of elderly people and immediately notify family members or caregivers when an abnormality is detected.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0537] In this invention, the server includes means for acquiring user data in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting anomalies based on the analysis results, means for notifying the detected anomalies, means for visualizing the notified anomaly data, data analysis means for monitoring the security status of the user and identifying anomalies, and communication means for immediately notifying the user of a response when an anomaly is detected. This enables advanced real-time monitoring of the health and security status of elderly people, and enables prompt and appropriate response when an anomaly occurs.

[0538] "User" refers to the person wearing the smart glasses or other device and providing the data.

[0539] "Real-time acquisition means" refers to methods of collecting data instantly using sensors or devices.

[0540] "Means of sending to a server" refers to a method of transferring collected data to a remote server using Wi-Fi, cellular networks, etc.

[0541] "AI technology for analyzing data received by the server" refers to a method of processing data and extracting information using machine learning models and data analysis techniques.

[0542] "Means for detecting anomalies based on analysis results" refers to a method for identifying data that deviates from normal patterns and determining that the data is abnormal.

[0543] "Means for notifying detected abnormalities" refers to communication means for notifying family members or caregivers when an abnormality occurs.

[0544] "Means for visualizing notified abnormal data" refers to methods for displaying abnormal data in an easy-to-understand manner using dashboards and graphs.

[0545] "Data analysis means for monitoring users' security status and identifying anomalies" refers to technology for analyzing the status of elderly people and identifying security risks.

[0546] "Communication means for immediately notifying the user when an abnormality is detected" refers to a system or method for quickly notifying the user when an abnormality occurs.

[0547] This invention provides a system for watching over elderly people and monitoring security situations. The configuration of the system and how it is implemented will be specifically described below.

[0548] 1. System Configuration

[0549] Hardware

[0550] Smart glasses: Equipped with a heart rate sensor, accelerometer, gyroscope, GPS, and microphone, these sensors are used to collect data in real time.

[0551] Server: A cloud server (e.g. AWS) that provides an API endpoint for receiving data.

[0552] software

[0553] Smart Glasses App: Uses Python scripts to collect data from sensors and send it to the server.

[0554] Server API: API endpoints built using a web framework such as Flask.

[0555] Data analysis system: Uses AI technologies such as TensorFlow and PyTorch to analyze data in real time.

[0556] Notification system: Use APIs such as Twilio to send emergency notifications via SMS or email.

[0557] 2. Data collection and transmission

[0558] Smart glasses collect the user's heart rate, walking patterns, location information, and voice data in real time. The collected data is transmitted to a server at regular intervals via Wi-Fi or cellular networks. The transmitted data includes time information, sensor values, location information, etc.

[0559] 3. Receipt and analysis of data

[0560] The server receives the data sent from the smart glasses and stores it in a real-time database. The stored data is analyzed using AI technology (e.g., TensorFlow, PyTorch). If an abnormality is detected by comparing it with normal patterns, a notification is sent immediately to family members or caregivers.

[0561] 4. Notification of abnormalities

[0562] If an abnormality is detected, detailed information, including security risks, is sent to family members or caregivers via emergency notification. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., a sudden increase in heart rate or irregular gait). Notification methods include a dedicated app, SMS, email, and phone call.

[0563] 5. Data Visualization

[0564] The server provides a dashboard to visualize the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0565] Specific examples

[0566] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0567] The smart glasses capture heart rate (e.g., 70 bpm), walking patterns (stationary), location information (in your living room), and ambient sounds (audio from the TV).

[0568] The acquired data is sent to the server every minute.

[0569] The server receives the data and analyzes it in real time. Since the patient is in a relaxed state, it is determined to be normal.

[0570] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. This data is immediately sent to the server.

[0571] The server detects this abnormality, determines that "your heart rate is rising sharply and there is a high possibility of you falling," and immediately sends a notification to your family. It also sends a voice notification to the user through the smart glasses saying, "Your heart rate is rising. Please sit down and rest."

[0572] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0573] Example prompts for generative AI models

[0574] Create an anomaly detection scenario for an elderly security monitoring app. Data collected includes heart rate, walking patterns, and location. Detail how to respond and what notifications are sent when an anomaly is detected.

[0575] This will realize an effective monitoring system to ensure the safety and security of the elderly.

[0576] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0577] Step 1:

[0578] The device collects the user's heart rate, walking patterns, location information, and voice data in real time. Specifically, it uses the device's built-in heart rate sensor, accelerometer, gyroscope, GPS, and microphone to collect this sensor data. The input is the user's biometric information and surrounding environmental sounds, and the output is digital sensor data.

[0579] Step 2:

[0580] The terminal packages the collected sensor data into a batch format at regular intervals (for example, every minute) and sends it to the server. The transmission is done over Wi-Fi or cellular networks using a secure communication protocol such as HTTPS. The input is the sensor data, and the output is an encrypted data packet.

[0581] Step 3:

[0582] The server receives data sent from the device using a dedicated API endpoint and stores the received data in a real-time database. The input is the encrypted data packet, and the output is the decoded sensor data.

[0583] Step 4:

[0584] The server analyzes the data stored in the real-time database using an AI model. Machine learning libraries such as TensorFlow and PyTorch are used for the analysis to detect abnormal patterns from the data. The input is sensor data, and the output is the anomaly detection results and detailed information.

[0585] Step 5:

[0586] If the server detects an abnormality based on the analysis results, it issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high possibility of falling). Notifications are sent via SMS, email, phone, etc. using external communication APIs such as Twilio. The input is the anomaly detection result and detailed information, and the output is a notification message.

[0587] Step 6:

[0588] If an abnormality is detected, the device will notify the user by voice. Specifically, it uses voice synthesis technology to issue a message such as "Your heart rate is increasing. Please sit down and rest." The input is the result of the anomaly detection and the text of the voice message, and the output is the voice message.

[0589] Step 7:

[0590] The server provides a dashboard for visualizing the collected data, which can be accessed by family members or caregivers via a web browser or a dedicated app. The dashboard displays real-time data, abnormality history, and detailed data. The input is sensor data and analysis results, and the output is a graphical dashboard.

[0591] This will enable real-time monitoring of the health and security status of elderly people and rapid response in the event of an abnormality.

[0592] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0593] This invention is a system for monitoring elderly people. It collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization. Furthermore, this invention combines an emotion engine to recognize the user's emotional state, achieving more comprehensive monitoring.

[0594] System program and its processing overview

[0595] 1. Data Collection

[0596] The device (smart glasses) collects the user's heart rate, walking patterns, location information, surrounding voices, and facial expressions in real time. Specifically, data is collected using the smart glasses' built-in heart rate sensor, accelerometer, gyro sensor, GPS, microphone, camera, etc. The user's voice data and facial expression data are also captured.

[0597] 2. Data Transmission

[0598] The device sends the collected data to the server in batches at regular intervals over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, voice data, and facial expression data.

[0599] 3. Data Reception

[0600] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0601] 4. Data Analysis

[0602] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and if an abnormality is detected, immediate action is taken.

[0603] 5. Anomaly detection and notification

[0604] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, sadness emotional state). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0605] 6. Data Visualization

[0606] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app. The dashboard also visualizes the user's emotional state, showing emotional fluctuations and trends.

[0607] Specific examples

[0608] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0609] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), ambient sounds (audio from the television), and facial expression data (relaxed facial expression).

[0610] The terminal packages this data in a batch format and sends it to the server every minute.

[0611] The server receives the data from the device and stores it in a database.

[0612] The server uses an AI model and emotion engine to analyze the received data in real time, determining that the state is normal because the person is relaxed.

[0613] The device then detects a sudden increase in heart rate to 150 bpm and an unstable walking pattern, and immediately sends this data to the server. The emotion engine then detects "fear."

[0614] The server detects this as an abnormality, determines that "your heart rate has suddenly increased, you are at high risk of falling, and you are in a state of fear," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is increasing. Please sit down and rest."

[0615] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0616] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when abnormalities, including emotional states, are detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0617] The processing flow will be explained below.

[0618] Program processing steps

[0619] Step 1:

[0620] The device (smart glasses) collects user data (heart rate, walking patterns, location information, ambient sounds, facial expressions) in real time.

[0621] A heart rate sensor measures the user's heart rate and detects abnormal heart rate variations.

[0622] Acceleration and gyro sensors analyze the user's walking pattern and detect falls and unsteady walking.

[0623] GPS identifies the user's current location.

[0624] A microphone captures the sounds around the user and collects data to analyze tone and emotion.

[0625] A camera captures the user's facial expressions and collects facial expression data.

[0626] Step 2:

[0627] The terminal periodically (for example, every minute) sends the collected data to the server.

[0628] The terminal packages the data in a batch format.

[0629] Data is sent to a server using Wi-Fi or cellular networks.

[0630] Ensure data security by using secure communication protocols (e.g., HTTPS).

[0631] Step 3:

[0632] The server receives the data sent from the terminal.

[0633] Receive data using a dedicated API endpoint.

[0634] The received raw data is stored in a real-time database.

[0635] Step 4:

[0636] The server analyzes the received data in real time.

[0637] It uses AI models to detect abnormalities in heart rate and walking patterns.

[0638] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., joy, anger, sadness, fear).

[0639] Comparison with existing databases is performed to identify normal data and outliers.

[0640] Step 5:

[0641] If the server detects an abnormality based on the data analysis results, it will immediately issue a notification.

[0642] The server creates a notification that includes the type of anomaly, the user's current location, and detailed data (heart rate, walking pattern, emotional state, etc.).

[0643] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[0644] A voice message is also sent to the device to notify the user of the abnormality.

[0645] Step 6:

[0646] The server provides a dashboard to visually display the collected data.

[0647] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[0648] The dashboard displays current real-time data, historical anomaly data, statistical information, and emotional state.

[0649] Allow users to view data trends and anomaly history through a dashboard.

[0650] As a concrete example, the following occurs while a user is relaxing at home:

[0651] The device captures heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (TV audio), and facial expression data (relaxed facial expression).

[0652] The device sends this data to the server every minute.

[0653] The server receives the data and stores it in a database.

[0654] The server analyzes the data in real time using an AI model and emotion engine to determine whether the person is in a normal relaxed state.

[0655] Suddenly, the heart rate rises to 150 bpm, and data is sent showing an unstable walking pattern, along with data that the emotion engine detects as "fear."

[0656] The server determines this to be an abnormality and sends a notification to the family stating, "Heart rate is rising rapidly, there is a high possibility of falling, and the patient is in a state of fear."

[0657] The device will notify the user by voice, "Your heart rate is increasing. Please sit down and rest."

[0658] The server displays detailed data on abnormalities on a dashboard, allowing family members and caregivers to check the data in real time.

[0659] In this way, the present invention can monitor the daily life of the elderly in real time and respond quickly, including their emotional state, if necessary.

[0660] Example 2

[0661] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0662] Conventional systems for monitoring elderly people rely on limited biometric information acquisition and anomaly detection, and lack comprehensive monitoring that takes into account the user's emotional state. This makes it difficult to respond quickly and accurately when an abnormality is detected, posing a major challenge to ensuring the safety of elderly people and the peace of mind of family members and caregivers.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0664] In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using artificial intelligence technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormalities, means for visualizing the notified abnormal data, means for collecting heart rate, walking pattern, location information, voice data, and facial expression data, means for analyzing the voice data and facial expression data using an emotion engine to recognize the emotional state, and means for issuing notifications to the user and third parties based on the abnormality and emotional state. This makes it possible to monitor not only the user's biometric information but also their emotional state in real time, enabling quick and accurate response when an abnormality occurs.

[0665] "User" refers to individuals, especially elderly people, who use the system.

[0666] "Data" refers to information collected from the user, including heart rate, walking patterns, location information, voice data, and facial expression data.

[0667] "Real-time" refers to the immediacy in time that data is acquired and processed immediately.

[0668] A "terminal" is a device for acquiring and transmitting data, specifically referring to smart glasses.

[0669] "Server" refers to a computer system that receives, stores, analyzes data sent from a terminal, and issues notifications as needed.

[0670] "Artificial intelligence technology" refers to technology used to analyze data on servers, including machine learning and deep learning.

[0671] "Emotion engine" refers to a system that analyzes voice data and facial expression data to recognize a user's emotional state.

[0672] "Abnormal" refers to a state that deviates from normal data patterns or a potential problem related to user safety.

[0673] "Notification" refers to warnings and information sent to users and third parties when the server detects an abnormality.

[0674] "Visualization" refers to presenting data as graphs or tables to make it easier to read.

[0675] "Third parties" refer to significant parties with an interest in the user's condition, such as the user's family or caregivers.

[0676] MODE FOR CARRYING OUT THE INVENTION

[0677] The present invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. Specific embodiments of the present invention will be described below.

[0678] Data collection

[0679] The user wears the smart glasses and goes about their daily life. The smart glasses collect real-time data on their heart rate, walking patterns, location information, surrounding sounds, and facial expressions. The hardware used for this includes a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. For example, if a user is in their living room at home, their heart rate is recorded as 70 bpm, they are standing still, and they can hear the TV.

[0680] Data transmission

[0681] The device (smart glasses) packages the collected data into batches at regular intervals (e.g., every minute) and sends them to a server. This transmission is done via Wi-Fi or cellular networks, and the data is sent using a secure communication protocol (e.g., HTTPS). Examples of data include heart rate, walking patterns, location information, voice data, and facial expression data.

[0682] Data reception

[0683] The server receives data sent from the device via a dedicated API endpoint and stores it in a real-time database, ensuring continuous data storage and availability for later analysis.

[0684] Data analysis

[0685] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and any abnormalities detected are immediately addressed.

[0686] Anomaly detection and notification

[0687] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"), etc. Notifications are sent via a dedicated app, SMS, email, phone call, etc. A voice message is also sent to the device to notify the user of the abnormality.

[0688] Data Visualization

[0689] The server provides a dashboard to visually display the collected data. Family members and caregivers can use this dashboard to check the user's detailed data, abnormality history, and real-time status. The dashboard can be accessed from a web browser or a dedicated app, and heart rate fluctuations, movement paths, emotional trends, and other data are visualized in graphs and tables.

[0690] Specific examples

[0691] For example, if a user is relaxing at home, the smart glasses will collect their heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent to the server every minute and analyzed in real time. If the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine will detect "fear." Based on this, the server will determine that "the heart rate has suddenly risen, there is a high risk of falling, and the user is in a state of fear," and will notify family members and the user via voice notification.

[0692] Example prompts for generative AI models

[0693] "Please explain the specific data flow for an elderly care system that collects and analyzes heart rate, walking patterns, location information, voice data, and facial expression data in real time, and notifies users when an abnormality is detected."

[0694] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0695] Step 1:

[0696] The device (smart glasses) collects the user's heart rate, walking pattern, location information, voice data, and facial expression data in real time. The device collects data using a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. The input is raw data from the sensors, and the output is a collected composite data package. Specifically, the data collected is the user's heart rate of 70 bpm, walking pattern of standing still, location information of the living room, ambient sound of the TV, and facial expression of a relaxed state.

[0697] Step 2:

[0698] The terminal packages the collected data in a batch format at regular intervals (for example, every minute) and sends it to the server. Data is sent over Wi-Fi or cellular networks, using the secure HTTPS communication protocol. The input is the collected composite data package, and the output is an encrypted data packet. Specifically, the data from each sensor is combined into a single packet and sent to the server via the Internet.

[0699] Step 3:

[0700] The server receives data sent from the device via a dedicated API endpoint. The received data is stored in a real-time database to prevent data loss. The input is an encrypted data packet, and the output is structured data stored in the database. Specifically, the server decrypts the received data packet and records the user's heart rate, location information, etc. in the real-time database.

[0701] Step 4:

[0702] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data to detect abnormal patterns. The emotion engine analyzes the voice data and facial expression data to recognize the emotional state. The input is raw data obtained from the real-time database, and the output is events recognized as abnormal and the detected emotional state. Specifically, the system inputs heart rate and walking patterns into the AI ​​model to determine whether there are any abnormalities. Additionally, the voice data and facial expression data are input into the emotion engine to analyze whether the user is in a state of fear.

[0703] Step 5:

[0704] When an abnormality is detected, the server issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"). The device also sends a voice message to the user. The input is the analysis results of the AI ​​model and emotion engine, and the output is the notification message. Specifically, the notification is sent via a dedicated app, SMS, email, phone call, etc., and the device issues a voice notification saying, "Your heart rate is rising. Please sit down and rest."

[0705] Step 6:

[0706] The server provides a dashboard for visualizing data. Family members and caregivers can check the user's detailed data, abnormality history, and real-time status through the dashboard. The input is data stored in the real-time database, and the output is the visualized dashboard. Specifically, it displays heart rate fluctuations, movement routes, emotional trends, etc. in graphs and tables.

[0707] (Application example 2)

[0708] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] In elderly care systems, simply collecting physical data such as heart rate and walking patterns is difficult to accurately grasp the individual's condition. Furthermore, there is insufficient means for quickly and accurately notifying family members or caregivers when an abnormality occurs. Furthermore, there is a need for a system that can monitor emotional fluctuations and respond accordingly.

[0710] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormality, means for visualizing the notified abnormal data, means for acquiring surrounding voice data and facial expression data using a smart device, means for analyzing the emotional state based on AI technology, and means for quickly issuing a notification based on the abnormality and the emotional state. This makes it possible to monitor not only the physical state of the elderly person but also their emotional state in real time and to quickly and accurately notify the elderly person of any abnormalities.

[0711] "User" refers to an individual who provides data and is monitored by the system.

[0712] "Means for acquiring data in real time" refers to technology and devices for instantly collecting heart rate, walking patterns, location information, voice data, facial expression data, and the like from a user.

[0713] "Means for transmitting data to the server" refers to the communication technologies and protocols for securely and quickly transmitting acquired data to the server.

[0714] "AI technology for analyzing data received by the server" refers to artificial intelligence technology for analyzing data sent to the server and detecting anomalies and patterns.

[0715] "Means for detecting anomalies" refers to the techniques and processes that use AI technology to detect patterns or symptoms that deviate from normal conditions based on received data.

[0716] "Means for notifying abnormalities" refers to notification functions and means for notifying the user, family, or caregivers of detected abnormalities.

[0717] "Means for visualizing notified abnormal data" refers to technology and devices for visually displaying abnormal data in an easy-to-understand manner.

[0718] "Smart device" refers to an electronic device worn or carried by a user for collecting and transmitting data.

[0719] "Means for acquiring voice data and facial expression data" refers to technologies and sensors for capturing and recording the user's voice and facial expressions.

[0720] "Means for analyzing emotional state" refers to techniques and processes that analyze acquired voice data and facial expression data to determine the user's psychological and emotional state.

[0721] "Means for issuing prompt notifications" refers to technologies and functions for sending necessary warnings and information to users without delay based on detected abnormalities and emotional states.

[0722] A system for implementing this invention utilizes a smart device (e.g., smart glasses) worn by a user to acquire user data in real time and transmit it to a server. The data includes heart rate, walking patterns, location information, voice data, facial expression data, etc., and AI technology and an emotion engine analyze the data for abnormalities and emotional states.

[0723] Smart devices are equipped with heart rate sensors, accelerometers, gyroscopes, GPS, microphones, cameras, etc., and use these sensors to collect user data in real time. This data is sent to a server in batch format at regular intervals. The data is sent via Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS).

[0724] The server receives data sent from the device and stores it in a real-time database. The received data is analyzed in real time using an AI model (e.g., TensorFlow) and an emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., sudden increases in heart rate, unstable walking patterns, and emotional fluctuations). The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear).

[0725] If an abnormality is detected, the server immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, emotional state sadness, etc.). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0726] The server also provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard, accessible via a web browser or a dedicated app, also visualizes the user's emotional state, displaying emotional fluctuations and trends.

[0727] As a concrete example, consider a situation where a user is relaxing at home. Wearing smart glasses, the device collects the user's heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent in batch format to a server, where it is analyzed in real time using AI technology and an emotion engine. Because the user is in a relaxed state, this is deemed normal. However, if the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine detects "fear." The server determines this situation as abnormal and issues a notification to family members stating, "The heart rate has risen sharply, there is a risk of falling, and the emotional state is fear." A voice notification can also be sent to the user via the device, enabling a prompt response.

[0728] An example of a prompt sentence might be:

[0729] "User has heart rate of 150 bpm, potential fall, and high-pitched voice tone indicating fear. Detailed location is outside of home. Please take appropriate action."

[0730] This will enable real-time monitoring of not only the physical condition of the elderly but also their emotional state, enabling prompt and accurate responses.

[0731] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0732] Step 1:

[0733] The device collects user data in real time. Inputs include data from the heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. Using these sensors, the device obtains the user's heart rate, walking patterns, location information, voice data, and facial expression data. The output is raw data that is temporarily stored on the device and later sent to the server.

[0734] Step 2:

[0735] The device sends the acquired data to the server. The input includes the raw data collected in step 1. This data is packaged in batches at regular time intervals and sent to the server over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). As an output, a status indicating that the transmission is complete is returned to the device.

[0736] Step 3:

[0737] The server receives the data sent from the devices. The input includes the raw data package sent from step 2. The server stores this in a real-time database. The output is a dataset stored in the database. This ensures that the data is preserved without loss and can be used for further analysis.

[0738] Step 4:

[0739] The server analyzes the data received using an AI model and emotion engine. The input includes a dataset obtained from a real-time database. This dataset is fed into an AI model (e.g., TensorFlow) that compares the current data with previously learned normal patterns. The emotion engine analyzes voice and facial expression data to recognize the user's emotional state. The output is an analysis result that includes abnormal patterns and the user's emotional state.

[0740] Step 5:

[0741] The server detects anomalies based on the analysis results and issues a notification if necessary. The input includes the analysis results obtained in step 4. If an anomaly is detected based on this, the server immediately issues a notification to family members or caregivers. Notification methods include a dedicated app, SMS, email, and phone. As an output, an anomaly notification is sent to family members or caregivers. A voice message is also sent to the device to notify the user of the anomaly.

[0742] Step 6:

[0743] The server provides a dashboard to visually display the collected data. Inputs include real-time and historical data. The server visualizes this data in an easy-to-understand manner and makes it accessible to family members and caregivers via a web browser or dedicated app. Outputs include detailed data, abnormality history, and real-time raw data, all visualized on the dashboard. The user's emotional state is also visualized, showing emotional fluctuations and trends.

[0744] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0745] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0746] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0747] [Third embodiment]

[0748] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0749] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0750] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0751] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0752] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0753] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0754] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0755] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0756] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0757] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0758] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0759] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0760] This invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology, and provides a means to detect and notify abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization.

[0761] System program and its processing overview

[0762] 1. Data Collection

[0763] The device (smart glasses) collects data such as the user's heart rate, walking patterns, location information, surrounding sounds, etc. Specifically, the data is collected using the heart rate sensor, accelerometer, gyro sensor, GPS, microphone, etc. built into the smart glasses.

[0764] 2. Data Transmission

[0765] The device sends the collected data to the server in batches at regular intervals. The data is sent over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, etc.

[0766] 3. Data Reception

[0767] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0768] 4. Data Analysis

[0769] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (for example, a sudden increase in heart rate or irregular walking). This analysis is performed in real time, and any abnormalities detected are immediately processed.

[0770] 5. Anomaly detection and notification

[0771] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0772] 6. Data Visualization

[0773] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0774] Specific examples

[0775] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0776] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0777] The terminal packages this data in a batch format and sends it to the server every minute.

[0778] The server receives the data from the device and stores it in a database.

[0779] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0780] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. The device acquires this data and immediately sends it to the server.

[0781] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0782] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0783] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0784] The processing flow will be explained below.

[0785] Program processing steps

[0786] Step 1:

[0787] The device collects user data (heart rate, walking patterns, location information, and ambient sounds) in real time.

[0788] Data is captured by sensors built into the device (heart rate sensor, accelerometer, gyroscope, GPS, microphone, etc.).

[0789] The captured data is temporarily stored in the device's memory.

[0790] Step 2:

[0791] The terminal sends data to the server at regular intervals (for example, every minute).

[0792] The terminal packages the data in a batch format.

[0793] The packaged data is then transmitted over Wi-Fi or cellular networks.

[0794] A secure communication protocol (e.g., HTTPS) is used for transmission.

[0795] Step 3:

[0796] The server receives the data sent from the terminal.

[0797] A dedicated API endpoint receives the data.

[0798] The received raw data is stored in a real-time database.

[0799] Step 4:

[0800] The server analyzes the received data in real time.

[0801] AI models (e.g., machine learning or deep learning models) process the data in real time.

[0802] The received data is compared with existing normal data to detect abnormal patterns.

[0803] Use feedback loops to continuously improve your AI models.

[0804] Step 5:

[0805] If the server detects an abnormality, it will immediately issue a notification.

[0806] Create a notification that includes the type of anomaly, the user's current location, and detailed data (e.g., changes in heart rate, irregular walking patterns).

[0807] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[0808] A voice message is also sent to the device to notify the user of the abnormality.

[0809] Step 6:

[0810] The server provides a dashboard for visually displaying the collected data.

[0811] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[0812] Current real-time data, past abnormal data, and statistical information are displayed on the dashboard.

[0813] Visualize users' data trends and anomaly history to enable proactive action.

[0814] Through this process, the system can monitor the safety of the elderly in real time and respond quickly if necessary.

[0815] Example 1

[0816] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0817] Monitoring the daily lives of the elderly requires the collection of biometric and environmental data in real time and the prompt detection and notification of abnormalities. However, existing technologies do not seamlessly collect, transmit, analyze, and notify data, which can lead to insufficient efforts to ensure the safety of the elderly. Furthermore, the visual display of data is insufficient to enable family members and caregivers to respond quickly to abnormalities. This leaves the elderly unable to be ensured safely and also creates a sense of security for family members and caregivers.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0819] In this invention, the server includes means for acquiring biometric data and environmental data from the user in real time, means for transmitting the acquired data to the server at regular intervals, means for storing the data received by the server in a real-time database, means for detecting abnormalities based on the received data using AI technology, means for notifying the user and a third party of the detected abnormality, and means for visually displaying the notified abnormal data. This makes it possible to monitor the biometric data and environmental data of the elderly in real time and to respond quickly when an abnormality is detected.

[0820] "User" means a subject who uses the system to provide biometric and environmental data.

[0821] "Biometric data" refers to data that indicates the physiological state of the user, such as heart rate or walking patterns.

[0822] "Environmental data" is data that indicates the user's surrounding environment, such as location information and surrounding audio data.

[0823] "Real-time" means that data is processed from the moment it is collected with little delay.

[0824] A "terminal" is a device for collecting data from a user, specifically smart glasses.

[0825] "Server" means a central processing unit for receiving and analyzing collected data.

[0826] "Fixed interval" refers to the time setting for processing and transmitting data at regular time intervals.

[0827] "Batch format" refers to a method of sending data in batches at regular intervals.

[0828] "AI technology" is a technology that uses machine learning models to analyze data and detect anomalies.

[0829] "Abnormal" refers to data patterns that deviate from normal patterns, including, for example, a sudden increase in heart rate or irregular gait.

[0830] "Notification" refers to the act of issuing a warning to the user and third parties when an abnormality is detected.

[0831] "Visually displaying" refers to showing data in a visual manner, such as in a graph or chart.

[0832] This invention is a system for monitoring elderly people, which collects biometric and environmental data from users in real time, transmits it to a server, analyzes the data using AI technology, and detects and notifies users of abnormalities. This system is mainly composed of a terminal (smart glasses) and a server, and operates as follows:

[0833] Hardware and Software Configuration

[0834] The smart glasses used as terminals have the following built-in devices:

[0835] Heart rate sensor: Measures the user's heart rate.

[0836] Acceleration sensor: Recognizes the user's walking pattern.

[0837] Gyro sensor: Measures walking stability.

[0838] GPS: Obtains the user's location information.

[0839] Microphone: Collects surrounding audio data.

[0840] This data is collected in real time by a data processing unit in the smart glasses and packaged in batches at regular intervals, after which the collected data is securely transmitted to a server via Wi-Fi or cellular networks using the HTTPS protocol.

[0841] The server receives, stores, and analyzes the collected data. The server has the following functions:

[0842] API endpoint: Responsible for receiving data.

[0843] Real-time database: Stores incoming data without loss.

[0844] AI model: Detects anomalies based on incoming data. The AI ​​model uses machine learning algorithms to compare normal patterns with current data and detect abnormal patterns in real time.

[0845] Specific examples

[0846] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0847] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[0848] The terminal packages this data in a batch format and sends it to the server every minute.

[0849] The server receives the data from the device and stores it in a database.

[0850] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[0851] The device captures data on the sudden rise in heart rate to 150 bpm and the walking pattern becoming unstable, and immediately sends it to the server.

[0852] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[0853] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0854] Prompt Sentence Examples

[0855] "We are developing a monitoring system for the elderly. This system uses smart glasses to collect data such as heart rate, walking patterns, location information, and surrounding sounds, and sends it to a server. The server uses an AI model to analyze the data in real time and notify users if an abnormality is detected. Please briefly explain the system's processing steps, taking into account the specific example below."

[0856] This invention allows for real-time monitoring of a user's biometric and environmental data, enabling rapid response when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to family members and caregivers.

[0857] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0858] Step 1:

[0859] Data collection

[0860] The terminal acquires the user's biometric data and environmental data in real time.

[0861] Input: User's heart rate, walking patterns, location, and ambient audio.

[0862] How it works: The smart glasses have a built-in heart rate sensor to measure your heart rate, an accelerometer and gyroscope to recognize your walking patterns, a GPS to acquire your location, and a microphone to collect surrounding audio data.

[0863] Output: A package of collected biometric and environmental data.

[0864] Step 2:

[0865] Data transmission

[0866] The terminal transmits the collected data to the server at regular intervals.

[0867] Input: A package of collected biometric and environmental data.

[0868] What it does: It packages collected data into batches and sends them to a server over Wi-Fi or a cellular network, using a secure communication protocol such as HTTPS.

[0869] Output: The data package sent to the server.

[0870] Step 3:

[0871] Data reception

[0872] The server receives the data sent from the terminal.

[0873] Input: Data package sent from the terminal.

[0874] Specific operation: Receive data at the API endpoint and store it in the real-time database.

[0875] Output: Data stored in the real-time database.

[0876] Step 4:

[0877] Data analysis

[0878] The server uses AI technology to detect anomalies based on the received data.

[0879] Input: Data stored in the real-time database.

[0880] What it does: Uses AI models to compare normal patterns with current data and detect abnormal patterns.

[0881] Output: Anomaly detection results.

[0882] Step 5:

[0883] Anomaly detection and notification

[0884] If the server detects an abnormality, it notifies the user and third parties.

[0885] Input: Anomaly detection results.

[0886] Specific operation: If an abnormality is detected, a notification will be issued to family members or caregivers via a dedicated app, SMS, email, phone call, etc. A voice message will also be sent to the device to notify the user of the abnormality.

[0887] Output: Notice to users and third parties.

[0888] Step 6:

[0889] Data Visualization

[0890] The server visually displays the notified abnormal data.

[0891] Input: Data stored in the real-time database and anomaly detection results.

[0892] What it does: Data is displayed visually on a dashboard, allowing family members and caregivers to view detailed data, abnormality history, and real-time raw data. Accessible via a web browser or dedicated app.

[0893] Output: Visually displayed data.

[0894] (Application example 1)

[0895] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0896] Conventional technologies for monitoring elderly people have difficulty acquiring and analyzing data in real time, making it difficult to respond quickly when an abnormality occurs. Additionally, there is a lack of a means to comprehensively monitor the security status of elderly people and immediately notify family members or caregivers when an abnormality is detected.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0898] In this invention, the server includes means for acquiring user data in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting anomalies based on the analysis results, means for notifying the detected anomalies, means for visualizing the notified anomaly data, data analysis means for monitoring the security status of the user and identifying anomalies, and communication means for immediately notifying the user of a response when an anomaly is detected. This enables advanced real-time monitoring of the health and security status of elderly people, and enables prompt and appropriate response when an anomaly occurs.

[0899] "User" refers to the person wearing the smart glasses or other device and providing the data.

[0900] "Real-time acquisition means" refers to methods of collecting data instantly using sensors or devices.

[0901] "Means of sending to a server" refers to a method of transferring collected data to a remote server using Wi-Fi, cellular networks, etc.

[0902] "AI technology for analyzing data received by the server" refers to a method of processing data and extracting information using machine learning models and data analysis techniques.

[0903] "Means for detecting anomalies based on analysis results" refers to a method for identifying data that deviates from normal patterns and determining that the data is abnormal.

[0904] "Means for notifying detected abnormalities" refers to communication means for notifying family members or caregivers when an abnormality occurs.

[0905] "Means for visualizing notified abnormal data" refers to methods for displaying abnormal data in an easy-to-understand manner using dashboards and graphs.

[0906] "Data analysis means for monitoring users' security status and identifying anomalies" refers to technology for analyzing the status of elderly people and identifying security risks.

[0907] "Communication means for immediately notifying the user when an abnormality is detected" refers to a system or method for quickly notifying the user when an abnormality occurs.

[0908] This invention provides a system for watching over elderly people and monitoring security situations. The configuration of the system and how it is implemented will be specifically described below.

[0909] 1. System Configuration

[0910] Hardware

[0911] Smart glasses: Equipped with a heart rate sensor, accelerometer, gyroscope, GPS, and microphone, these sensors are used to collect data in real time.

[0912] Server: A cloud server (e.g. AWS) that provides an API endpoint for receiving data.

[0913] software

[0914] Smart Glasses App: Uses Python scripts to collect data from sensors and send it to the server.

[0915] Server API: API endpoints built using a web framework such as Flask.

[0916] Data analysis system: Uses AI technologies such as TensorFlow and PyTorch to analyze data in real time.

[0917] Notification system: Use APIs such as Twilio to send emergency notifications via SMS or email.

[0918] 2. Data collection and transmission

[0919] Smart glasses collect the user's heart rate, walking patterns, location information, and voice data in real time. The collected data is transmitted to a server at regular intervals via Wi-Fi or cellular networks. The transmitted data includes time information, sensor values, location information, etc.

[0920] 3. Receipt and analysis of data

[0921] The server receives the data sent from the smart glasses and stores it in a real-time database. The stored data is analyzed using AI technology (e.g., TensorFlow, PyTorch). If an abnormality is detected by comparing it with normal patterns, a notification is sent immediately to family members or caregivers.

[0922] 4. Notification of abnormalities

[0923] If an abnormality is detected, detailed information, including security risks, is sent to family members or caregivers via emergency notification. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., a sudden increase in heart rate or irregular gait). Notification methods include a dedicated app, SMS, email, and phone call.

[0924] 5. Data Visualization

[0925] The server provides a dashboard to visualize the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[0926] Specific examples

[0927] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0928] The smart glasses capture heart rate (e.g., 70 bpm), walking patterns (stationary), location information (in your living room), and ambient sounds (audio from the TV).

[0929] The acquired data is sent to the server every minute.

[0930] The server receives the data and analyzes it in real time. Since the patient is in a relaxed state, it is determined to be normal.

[0931] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. This data is immediately sent to the server.

[0932] The server detects this abnormality, determines that "your heart rate is rising sharply and there is a high possibility of you falling," and immediately sends a notification to your family. It also sends a voice notification to the user through the smart glasses saying, "Your heart rate is rising. Please sit down and rest."

[0933] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0934] Example prompts for generative AI models

[0935] Create an anomaly detection scenario for an elderly security monitoring app. Data collected includes heart rate, walking patterns, and location. Detail how to respond and what notifications are sent when an anomaly is detected.

[0936] This will realize an effective monitoring system to ensure the safety and security of the elderly.

[0937] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0938] Step 1:

[0939] The device collects the user's heart rate, walking patterns, location information, and voice data in real time. Specifically, it uses the device's built-in heart rate sensor, accelerometer, gyroscope, GPS, and microphone to collect this sensor data. The input is the user's biometric information and surrounding environmental sounds, and the output is digital sensor data.

[0940] Step 2:

[0941] The terminal packages the collected sensor data into a batch format at regular intervals (for example, every minute) and sends it to the server. The transmission is done over Wi-Fi or cellular networks using a secure communication protocol such as HTTPS. The input is the sensor data, and the output is an encrypted data packet.

[0942] Step 3:

[0943] The server receives data sent from the device using a dedicated API endpoint and stores the received data in a real-time database. The input is the encrypted data packet, and the output is the decoded sensor data.

[0944] Step 4:

[0945] The server analyzes the data stored in the real-time database using an AI model. Machine learning libraries such as TensorFlow and PyTorch are used for the analysis to detect abnormal patterns from the data. The input is sensor data, and the output is the anomaly detection results and detailed information.

[0946] Step 5:

[0947] If the server detects an abnormality based on the analysis results, it issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high possibility of falling). Notifications are sent via SMS, email, phone, etc. using external communication APIs such as Twilio. The input is the anomaly detection result and detailed information, and the output is a notification message.

[0948] Step 6:

[0949] If an abnormality is detected, the device will notify the user by voice. Specifically, it uses voice synthesis technology to issue a message such as "Your heart rate is increasing. Please sit down and rest." The input is the result of the anomaly detection and the text of the voice message, and the output is the voice message.

[0950] Step 7:

[0951] The server provides a dashboard for visualizing the collected data, which can be accessed by family members or caregivers via a web browser or a dedicated app. The dashboard displays real-time data, abnormality history, and detailed data. The input is sensor data and analysis results, and the output is a graphical dashboard.

[0952] This will enable real-time monitoring of the health and security status of elderly people and rapid response in the event of an abnormality.

[0953] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0954] This invention is a system for monitoring elderly people. It collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization. Furthermore, this invention combines an emotion engine to recognize the user's emotional state, achieving more comprehensive monitoring.

[0955] System program and its processing overview

[0956] 1. Data Collection

[0957] The device (smart glasses) collects the user's heart rate, walking patterns, location information, surrounding voices, and facial expressions in real time. Specifically, data is collected using the smart glasses' built-in heart rate sensor, accelerometer, gyro sensor, GPS, microphone, camera, etc. The user's voice data and facial expression data are also captured.

[0958] 2. Data Transmission

[0959] The device sends the collected data to the server in batches at regular intervals over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, voice data, and facial expression data.

[0960] 3. Data Reception

[0961] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[0962] 4. Data Analysis

[0963] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and if an abnormality is detected, immediate action is taken.

[0964] 5. Anomaly detection and notification

[0965] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, sadness emotional state). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[0966] 6. Data Visualization

[0967] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app. The dashboard also visualizes the user's emotional state, showing emotional fluctuations and trends.

[0968] Specific examples

[0969] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[0970] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), ambient sounds (audio from the television), and facial expression data (relaxed facial expression).

[0971] The terminal packages this data in a batch format and sends it to the server every minute.

[0972] The server receives the data from the device and stores it in a database.

[0973] The server uses an AI model and emotion engine to analyze the received data in real time, determining that the state is normal because the person is relaxed.

[0974] The device then detects a sudden increase in heart rate to 150 bpm and an unstable walking pattern, and immediately sends this data to the server. The emotion engine then detects "fear."

[0975] The server detects this as an abnormality, determines that "your heart rate has suddenly increased, you are at high risk of falling, and you are in a state of fear," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is increasing. Please sit down and rest."

[0976] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[0977] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when abnormalities, including emotional states, are detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[0978] The processing flow will be explained below.

[0979] Program processing steps

[0980] Step 1:

[0981] The device (smart glasses) collects user data (heart rate, walking patterns, location information, ambient sounds, facial expressions) in real time.

[0982] A heart rate sensor measures the user's heart rate and detects abnormal heart rate variations.

[0983] Acceleration and gyro sensors analyze the user's walking pattern and detect falls and unsteady walking.

[0984] GPS identifies the user's current location.

[0985] A microphone captures the sounds around the user and collects data to analyze tone and emotion.

[0986] A camera captures the user's facial expressions and collects facial expression data.

[0987] Step 2:

[0988] The terminal periodically (for example, every minute) sends the collected data to the server.

[0989] The terminal packages the data in a batch format.

[0990] Data is sent to a server using Wi-Fi or cellular networks.

[0991] Ensure data security by using secure communication protocols (e.g., HTTPS).

[0992] Step 3:

[0993] The server receives the data sent from the terminal.

[0994] Receive data using a dedicated API endpoint.

[0995] The received raw data is stored in a real-time database.

[0996] Step 4:

[0997] The server analyzes the received data in real time.

[0998] It uses AI models to detect abnormalities in heart rate and walking patterns.

[0999] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., joy, anger, sadness, fear).

[1000] Comparison with existing databases is performed to identify normal data and outliers.

[1001] Step 5:

[1002] If the server detects an abnormality based on the data analysis results, it will immediately issue a notification.

[1003] The server creates a notification that includes the type of anomaly, the user's current location, and detailed data (heart rate, walking pattern, emotional state, etc.).

[1004] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[1005] A voice message is also sent to the device to notify the user of the abnormality.

[1006] Step 6:

[1007] The server provides a dashboard to visually display the collected data.

[1008] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[1009] The dashboard displays current real-time data, historical anomaly data, statistical information, and emotional state.

[1010] Allow users to view data trends and anomaly history through a dashboard.

[1011] As a concrete example, the following occurs while a user is relaxing at home:

[1012] The device captures heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (TV audio), and facial expression data (relaxed facial expression).

[1013] The device sends this data to the server every minute.

[1014] The server receives the data and stores it in a database.

[1015] The server analyzes the data in real time using an AI model and emotion engine to determine whether the person is in a normal relaxed state.

[1016] Suddenly, the heart rate rises to 150 bpm, and data is sent showing an unstable walking pattern, along with data that the emotion engine detects as "fear."

[1017] The server determines this to be an abnormality and sends a notification to the family stating, "Heart rate is rising rapidly, there is a high possibility of falling, and the patient is in a state of fear."

[1018] The device will notify the user by voice, "Your heart rate is increasing. Please sit down and rest."

[1019] The server displays detailed data on abnormalities on a dashboard, allowing family members and caregivers to check the data in real time.

[1020] In this way, the present invention can monitor the daily life of the elderly in real time and respond quickly, including their emotional state, if necessary.

[1021] Example 2

[1022] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1023] Conventional systems for monitoring elderly people rely on limited biometric information acquisition and anomaly detection, and lack comprehensive monitoring that takes into account the user's emotional state. This makes it difficult to respond quickly and accurately when an abnormality is detected, posing a major challenge to ensuring the safety of elderly people and the peace of mind of family members and caregivers.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1025] In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using artificial intelligence technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormalities, means for visualizing the notified abnormal data, means for collecting heart rate, walking pattern, location information, voice data, and facial expression data, means for analyzing the voice data and facial expression data using an emotion engine to recognize the emotional state, and means for issuing notifications to the user and third parties based on the abnormality and emotional state. This makes it possible to monitor not only the user's biometric information but also their emotional state in real time, enabling quick and accurate response when an abnormality occurs.

[1026] "User" refers to individuals, especially elderly people, who use the system.

[1027] "Data" refers to information collected from the user, including heart rate, walking patterns, location information, voice data, and facial expression data.

[1028] "Real-time" refers to the immediacy in time that data is acquired and processed immediately.

[1029] A "terminal" is a device for acquiring and transmitting data, specifically referring to smart glasses.

[1030] "Server" refers to a computer system that receives, stores, analyzes data sent from a terminal, and issues notifications as needed.

[1031] "Artificial intelligence technology" refers to technology used to analyze data on servers, including machine learning and deep learning.

[1032] "Emotion engine" refers to a system that analyzes voice data and facial expression data to recognize a user's emotional state.

[1033] "Abnormal" refers to a state that deviates from normal data patterns or a potential problem related to user safety.

[1034] "Notification" refers to warnings and information sent to users and third parties when the server detects an abnormality.

[1035] "Visualization" refers to presenting data as graphs or tables to make it easier to read.

[1036] "Third parties" refer to significant parties with an interest in the user's condition, such as the user's family or caregivers.

[1037] MODE FOR CARRYING OUT THE INVENTION

[1038] The present invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. Specific embodiments of the present invention will be described below.

[1039] Data collection

[1040] The user wears the smart glasses and goes about their daily life. The smart glasses collect real-time data on their heart rate, walking patterns, location information, surrounding sounds, and facial expressions. The hardware used for this includes a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. For example, if a user is in their living room at home, their heart rate is recorded as 70 bpm, they are standing still, and they can hear the TV.

[1041] Data transmission

[1042] The device (smart glasses) packages the collected data into batches at regular intervals (e.g., every minute) and sends them to a server. This transmission is done via Wi-Fi or cellular networks, and the data is sent using a secure communication protocol (e.g., HTTPS). Examples of data include heart rate, walking patterns, location information, voice data, and facial expression data.

[1043] Data reception

[1044] The server receives data sent from the device via a dedicated API endpoint and stores it in a real-time database, ensuring continuous data storage and availability for later analysis.

[1045] Data analysis

[1046] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and any abnormalities detected are immediately addressed.

[1047] Anomaly detection and notification

[1048] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"), etc. Notifications are sent via a dedicated app, SMS, email, phone call, etc. A voice message is also sent to the device to notify the user of the abnormality.

[1049] Data Visualization

[1050] The server provides a dashboard to visually display the collected data. Family members and caregivers can use this dashboard to check the user's detailed data, abnormality history, and real-time status. The dashboard can be accessed from a web browser or a dedicated app, and heart rate fluctuations, movement paths, emotional trends, and other data are visualized in graphs and tables.

[1051] Specific examples

[1052] For example, if a user is relaxing at home, the smart glasses will collect their heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent to the server every minute and analyzed in real time. If the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine will detect "fear." Based on this, the server will determine that "the heart rate has suddenly risen, there is a high risk of falling, and the user is in a state of fear," and will notify family members and the user via voice notification.

[1053] Example prompts for generative AI models

[1054] "Please explain the specific data flow for an elderly care system that collects and analyzes heart rate, walking patterns, location information, voice data, and facial expression data in real time, and notifies users when an abnormality is detected."

[1055] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1056] Step 1:

[1057] The device (smart glasses) collects the user's heart rate, walking pattern, location information, voice data, and facial expression data in real time. The device collects data using a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. The input is raw data from the sensors, and the output is a collected composite data package. Specifically, the data collected is the user's heart rate of 70 bpm, walking pattern of standing still, location information of the living room, ambient sound of the TV, and facial expression of a relaxed state.

[1058] Step 2:

[1059] The terminal packages the collected data in a batch format at regular intervals (for example, every minute) and sends it to the server. Data is sent over Wi-Fi or cellular networks, using the secure HTTPS communication protocol. The input is the collected composite data package, and the output is an encrypted data packet. Specifically, the data from each sensor is combined into a single packet and sent to the server via the Internet.

[1060] Step 3:

[1061] The server receives data sent from the device via a dedicated API endpoint. The received data is stored in a real-time database to prevent data loss. The input is an encrypted data packet, and the output is structured data stored in the database. Specifically, the server decrypts the received data packet and records the user's heart rate, location information, etc. in the real-time database.

[1062] Step 4:

[1063] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data to detect abnormal patterns. The emotion engine analyzes the voice data and facial expression data to recognize the emotional state. The input is raw data obtained from the real-time database, and the output is events recognized as abnormal and the detected emotional state. Specifically, the system inputs heart rate and walking patterns into the AI ​​model to determine whether there are any abnormalities. Additionally, the voice data and facial expression data are input into the emotion engine to analyze whether the user is in a state of fear.

[1064] Step 5:

[1065] When an abnormality is detected, the server issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"). The device also sends a voice message to the user. The input is the analysis results of the AI ​​model and emotion engine, and the output is the notification message. Specifically, the notification is sent via a dedicated app, SMS, email, phone call, etc., and the device issues a voice notification saying, "Your heart rate is rising. Please sit down and rest."

[1066] Step 6:

[1067] The server provides a dashboard for visualizing data. Family members and caregivers can check the user's detailed data, abnormality history, and real-time status through the dashboard. The input is data stored in the real-time database, and the output is the visualized dashboard. Specifically, it displays heart rate fluctuations, movement routes, emotional trends, etc. in graphs and tables.

[1068] (Application example 2)

[1069] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1070] In elderly care systems, simply collecting physical data such as heart rate and walking patterns is difficult to accurately grasp the individual's condition. Furthermore, there is insufficient means for quickly and accurately notifying family members or caregivers when an abnormality occurs. Furthermore, there is a need for a system that can monitor emotional fluctuations and respond accordingly.

[1071] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormality, means for visualizing the notified abnormal data, means for acquiring surrounding voice data and facial expression data using a smart device, means for analyzing the emotional state based on AI technology, and means for quickly issuing a notification based on the abnormality and the emotional state. This makes it possible to monitor not only the physical state of the elderly person but also their emotional state in real time and to quickly and accurately notify the elderly person of any abnormalities.

[1072] "User" refers to an individual who provides data and is monitored by the system.

[1073] "Means for acquiring data in real time" refers to technology and devices for instantly collecting heart rate, walking patterns, location information, voice data, facial expression data, and the like from a user.

[1074] "Means for transmitting data to the server" refers to the communication technologies and protocols for securely and quickly transmitting acquired data to the server.

[1075] "AI technology for analyzing data received by the server" refers to artificial intelligence technology for analyzing data sent to the server and detecting anomalies and patterns.

[1076] "Means for detecting anomalies" refers to the techniques and processes that use AI technology to detect patterns or symptoms that deviate from normal conditions based on received data.

[1077] "Means for notifying abnormalities" refers to notification functions and means for notifying the user, family, or caregivers of detected abnormalities.

[1078] "Means for visualizing notified abnormal data" refers to technology and devices for visually displaying abnormal data in an easy-to-understand manner.

[1079] "Smart device" refers to an electronic device worn or carried by a user for collecting and transmitting data.

[1080] "Means for acquiring voice data and facial expression data" refers to technologies and sensors for capturing and recording the user's voice and facial expressions.

[1081] "Means for analyzing emotional state" refers to techniques and processes that analyze acquired voice data and facial expression data to determine the user's psychological and emotional state.

[1082] "Means for issuing prompt notifications" refers to technologies and functions for sending necessary warnings and information to users without delay based on detected abnormalities and emotional states.

[1083] A system for implementing this invention utilizes a smart device (e.g., smart glasses) worn by a user to acquire user data in real time and transmit it to a server. The data includes heart rate, walking patterns, location information, voice data, facial expression data, etc., and AI technology and an emotion engine analyze the data for abnormalities and emotional states.

[1084] Smart devices are equipped with heart rate sensors, accelerometers, gyroscopes, GPS, microphones, cameras, etc., and use these sensors to collect user data in real time. This data is sent to a server in batch format at regular intervals. The data is sent via Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS).

[1085] The server receives data sent from the device and stores it in a real-time database. The received data is analyzed in real time using an AI model (e.g., TensorFlow) and an emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., sudden increases in heart rate, unstable walking patterns, and emotional fluctuations). The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear).

[1086] If an abnormality is detected, the server immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, emotional state sadness, etc.). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[1087] The server also provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard, accessible via a web browser or a dedicated app, also visualizes the user's emotional state, displaying emotional fluctuations and trends.

[1088] As a concrete example, consider a situation where a user is relaxing at home. Wearing smart glasses, the device collects the user's heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent in batch format to a server, where it is analyzed in real time using AI technology and an emotion engine. Because the user is in a relaxed state, this is deemed normal. However, if the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine detects "fear." The server determines this situation as abnormal and issues a notification to family members stating, "The heart rate has risen sharply, there is a risk of falling, and the emotional state is fear." A voice notification can also be sent to the user via the device, enabling a prompt response.

[1089] An example of a prompt sentence might be:

[1090] "User has heart rate of 150 bpm, potential fall, and high-pitched voice tone indicating fear. Detailed location is outside of home. Please take appropriate action."

[1091] This will enable real-time monitoring of not only the physical condition of the elderly but also their emotional state, enabling prompt and accurate responses.

[1092] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1093] Step 1:

[1094] The device collects user data in real time. Inputs include data from the heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. Using these sensors, the device obtains the user's heart rate, walking patterns, location information, voice data, and facial expression data. The output is raw data that is temporarily stored on the device and later sent to the server.

[1095] Step 2:

[1096] The device sends the acquired data to the server. The input includes the raw data collected in step 1. This data is packaged in batches at regular time intervals and sent to the server over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). As an output, a status indicating that the transmission is complete is returned to the device.

[1097] Step 3:

[1098] The server receives the data sent from the devices. The input includes the raw data package sent from step 2. The server stores this in a real-time database. The output is a dataset stored in the database. This ensures that the data is preserved without loss and can be used for further analysis.

[1099] Step 4:

[1100] The server analyzes the data received using an AI model and emotion engine. The input includes a dataset obtained from a real-time database. This dataset is fed into an AI model (e.g., TensorFlow) that compares the current data with previously learned normal patterns. The emotion engine analyzes voice and facial expression data to recognize the user's emotional state. The output is an analysis result that includes abnormal patterns and the user's emotional state.

[1101] Step 5:

[1102] The server detects anomalies based on the analysis results and issues a notification if necessary. The input includes the analysis results obtained in step 4. If an anomaly is detected based on this, the server immediately issues a notification to family members or caregivers. Notification methods include a dedicated app, SMS, email, and phone. As an output, an anomaly notification is sent to family members or caregivers. A voice message is also sent to the device to notify the user of the anomaly.

[1103] Step 6:

[1104] The server provides a dashboard to visually display the collected data. Inputs include real-time and historical data. The server visualizes this data in an easy-to-understand manner and makes it accessible to family members and caregivers via a web browser or dedicated app. Outputs include detailed data, abnormality history, and real-time raw data, all visualized on the dashboard. The user's emotional state is also visualized, showing emotional fluctuations and trends.

[1105] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1106] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1107] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1108] [Fourth embodiment]

[1109] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1110] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1111] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1112] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1113] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1114] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1115] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1116] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1117] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1118] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1119] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1120] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1121] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1122] This invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology, and provides a means to detect and notify abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization.

[1123] System program and its processing overview

[1124] 1. Data Collection

[1125] The device (smart glasses) collects data such as the user's heart rate, walking patterns, location information, surrounding sounds, etc. Specifically, the data is collected using the heart rate sensor, accelerometer, gyro sensor, GPS, microphone, etc. built into the smart glasses.

[1126] 2. Data Transmission

[1127] The device sends the collected data to the server in batches at regular intervals. The data is sent over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, etc.

[1128] 3. Data Reception

[1129] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[1130] 4. Data Analysis

[1131] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (for example, a sudden increase in heart rate or irregular walking). This analysis is performed in real time, and any abnormalities detected are immediately processed.

[1132] 5. Anomaly detection and notification

[1133] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[1134] 6. Data Visualization

[1135] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[1136] Specific examples

[1137] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[1138] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[1139] The terminal packages this data in a batch format and sends it to the server every minute.

[1140] The server receives the data from the device and stores it in a database.

[1141] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[1142] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. The device acquires this data and immediately sends it to the server.

[1143] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[1144] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[1145] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[1146] The processing flow will be explained below.

[1147] Program processing steps

[1148] Step 1:

[1149] The device collects user data (heart rate, walking patterns, location information, and ambient sounds) in real time.

[1150] Data is captured by sensors built into the device (heart rate sensor, accelerometer, gyroscope, GPS, microphone, etc.).

[1151] The captured data is temporarily stored in the device's memory.

[1152] Step 2:

[1153] The terminal sends data to the server at regular intervals (for example, every minute).

[1154] The terminal packages the data in a batch format.

[1155] The packaged data is then transmitted over Wi-Fi or cellular networks.

[1156] A secure communication protocol (e.g., HTTPS) is used for transmission.

[1157] Step 3:

[1158] The server receives the data sent from the terminal.

[1159] A dedicated API endpoint receives the data.

[1160] The received raw data is stored in a real-time database.

[1161] Step 4:

[1162] The server analyzes the received data in real time.

[1163] AI models (e.g., machine learning or deep learning models) process the data in real time.

[1164] The received data is compared with existing normal data to detect abnormal patterns.

[1165] Use feedback loops to continuously improve your AI models.

[1166] Step 5:

[1167] If the server detects an abnormality, it will immediately issue a notification.

[1168] Create a notification that includes the type of anomaly, the user's current location, and detailed data (e.g., changes in heart rate, irregular walking patterns).

[1169] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[1170] A voice message is also sent to the device to notify the user of the abnormality.

[1171] Step 6:

[1172] The server provides a dashboard for visually displaying the collected data.

[1173] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[1174] Current real-time data, past abnormal data, and statistical information are displayed on the dashboard.

[1175] Visualize users' data trends and anomaly history to enable proactive action.

[1176] Through this process, the system can monitor the safety of the elderly in real time and respond quickly if necessary.

[1177] Example 1

[1178] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1179] Monitoring the daily lives of the elderly requires the collection of biometric and environmental data in real time and the prompt detection and notification of abnormalities. However, existing technologies do not seamlessly collect, transmit, analyze, and notify data, which can lead to insufficient efforts to ensure the safety of the elderly. Furthermore, the visual display of data is insufficient to enable family members and caregivers to respond quickly to abnormalities. This leaves the elderly unable to be ensured safely and also creates a sense of security for family members and caregivers.

[1180] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1181] In this invention, the server includes means for acquiring biometric data and environmental data from the user in real time, means for transmitting the acquired data to the server at regular intervals, means for storing the data received by the server in a real-time database, means for detecting abnormalities based on the received data using AI technology, means for notifying the user and a third party of the detected abnormality, and means for visually displaying the notified abnormal data. This makes it possible to monitor the biometric data and environmental data of the elderly in real time and to respond quickly when an abnormality is detected.

[1182] "User" means a subject who uses the system to provide biometric and environmental data.

[1183] "Biometric data" refers to data that indicates the physiological state of the user, such as heart rate or walking patterns.

[1184] "Environmental data" is data that indicates the user's surrounding environment, such as location information and surrounding audio data.

[1185] "Real-time" means that data is processed from the moment it is collected with little delay.

[1186] A "terminal" is a device for collecting data from a user, specifically smart glasses.

[1187] "Server" means a central processing unit for receiving and analyzing collected data.

[1188] "Fixed interval" refers to the time setting for processing and transmitting data at regular time intervals.

[1189] "Batch format" refers to a method of sending data in batches at regular intervals.

[1190] "AI technology" is a technology that uses machine learning models to analyze data and detect anomalies.

[1191] "Abnormal" refers to data patterns that deviate from normal patterns, including, for example, a sudden increase in heart rate or irregular gait.

[1192] "Notification" refers to the act of issuing a warning to the user and third parties when an abnormality is detected.

[1193] "Visually displaying" refers to showing data in a visual manner, such as in a graph or chart.

[1194] This invention is a system for monitoring elderly people, which collects biometric and environmental data from users in real time, transmits it to a server, analyzes the data using AI technology, and detects and notifies users of abnormalities. This system is mainly composed of a terminal (smart glasses) and a server, and operates as follows:

[1195] Hardware and Software Configuration

[1196] The smart glasses used as terminals have the following built-in devices:

[1197] Heart rate sensor: Measures the user's heart rate.

[1198] Acceleration sensor: Recognizes the user's walking pattern.

[1199] Gyro sensor: Measures walking stability.

[1200] GPS: Obtains the user's location information.

[1201] Microphone: Collects surrounding audio data.

[1202] This data is collected in real time by a data processing unit in the smart glasses and packaged in batches at regular intervals, after which the collected data is securely transmitted to a server via Wi-Fi or cellular networks using the HTTPS protocol.

[1203] The server receives, stores, and analyzes the collected data. The server has the following functions:

[1204] API endpoint: Responsible for receiving data.

[1205] Real-time database: Stores incoming data without loss.

[1206] AI model: Detects anomalies based on incoming data. The AI ​​model uses machine learning algorithms to compare normal patterns with current data and detect abnormal patterns in real time.

[1207] Specific examples

[1208] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[1209] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), and ambient sounds (audio from the TV).

[1210] The terminal packages this data in a batch format and sends it to the server every minute.

[1211] The server receives the data from the device and stores it in a database.

[1212] The server uses an AI model to analyze the received data in real time and determines that the state is normal because the person is in a relaxed state.

[1213] The device captures data on the sudden rise in heart rate to 150 bpm and the walking pattern becoming unstable, and immediately sends it to the server.

[1214] The server detects this as an abnormality, determines that "your heart rate is rising sharply and you are likely to fall," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is rising. Please sit down and rest."

[1215] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[1216] Prompt Sentence Examples

[1217] "We are developing a monitoring system for the elderly. This system uses smart glasses to collect data such as heart rate, walking patterns, location information, and surrounding sounds, and sends it to a server. The server uses an AI model to analyze the data in real time and notify users if an abnormality is detected. Please briefly explain the system's processing steps, taking into account the specific example below."

[1218] This invention allows for real-time monitoring of a user's biometric and environmental data, enabling rapid response when an abnormality is detected, thereby ensuring the safety of the elderly and providing peace of mind to family members and caregivers.

[1219] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1220] Step 1:

[1221] Data collection

[1222] The terminal acquires the user's biometric data and environmental data in real time.

[1223] Input: User's heart rate, walking patterns, location, and ambient audio.

[1224] How it works: The smart glasses have a built-in heart rate sensor to measure your heart rate, an accelerometer and gyroscope to recognize your walking patterns, a GPS to acquire your location, and a microphone to collect surrounding audio data.

[1225] Output: A package of collected biometric and environmental data.

[1226] Step 2:

[1227] Data transmission

[1228] The terminal transmits the collected data to the server at regular intervals.

[1229] Input: A package of collected biometric and environmental data.

[1230] What it does: It packages collected data into batches and sends them to a server over Wi-Fi or a cellular network, using a secure communication protocol such as HTTPS.

[1231] Output: The data package sent to the server.

[1232] Step 3:

[1233] Data reception

[1234] The server receives the data sent from the terminal.

[1235] Input: Data package sent from the terminal.

[1236] Specific operation: Receive data at the API endpoint and store it in the real-time database.

[1237] Output: Data stored in the real-time database.

[1238] Step 4:

[1239] Data analysis

[1240] The server uses AI technology to detect anomalies based on the received data.

[1241] Input: Data stored in the real-time database.

[1242] What it does: Uses AI models to compare normal patterns with current data and detect abnormal patterns.

[1243] Output: Anomaly detection results.

[1244] Step 5:

[1245] Anomaly detection and notification

[1246] If the server detects an abnormality, it notifies the user and third parties.

[1247] Input: Anomaly detection results.

[1248] Specific operation: If an abnormality is detected, a notification will be issued to family members or caregivers via a dedicated app, SMS, email, phone call, etc. A voice message will also be sent to the device to notify the user of the abnormality.

[1249] Output: Notice to users and third parties.

[1250] Step 6:

[1251] Data Visualization

[1252] The server visually displays the notified abnormal data.

[1253] Input: Data stored in the real-time database and anomaly detection results.

[1254] What it does: Data is displayed visually on a dashboard, allowing family members and caregivers to view detailed data, abnormality history, and real-time raw data. Accessible via a web browser or dedicated app.

[1255] Output: Visually displayed data.

[1256] (Application example 1)

[1257] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1258] Conventional technologies for monitoring elderly people have difficulty acquiring and analyzing data in real time, making it difficult to respond quickly when an abnormality occurs. Additionally, there is a lack of a means to comprehensively monitor the security status of elderly people and immediately notify family members or caregivers when an abnormality is detected.

[1259] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1260] In this invention, the server includes means for acquiring user data in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting anomalies based on the analysis results, means for notifying the detected anomalies, means for visualizing the notified anomaly data, data analysis means for monitoring the security status of the user and identifying anomalies, and communication means for immediately notifying the user of a response when an anomaly is detected. This enables advanced real-time monitoring of the health and security status of elderly people, and enables prompt and appropriate response when an anomaly occurs.

[1261] "User" refers to the person wearing the smart glasses or other device and providing the data.

[1262] "Real-time acquisition means" refers to methods of collecting data instantly using sensors or devices.

[1263] "Means of sending to a server" refers to a method of transferring collected data to a remote server using Wi-Fi, cellular networks, etc.

[1264] "AI technology for analyzing data received by the server" refers to a method of processing data and extracting information using machine learning models and data analysis techniques.

[1265] "Means for detecting anomalies based on analysis results" refers to a method for identifying data that deviates from normal patterns and determining that the data is abnormal.

[1266] "Means for notifying detected abnormalities" refers to communication means for notifying family members or caregivers when an abnormality occurs.

[1267] "Means for visualizing notified abnormal data" refers to methods for displaying abnormal data in an easy-to-understand manner using dashboards and graphs.

[1268] "Data analysis means for monitoring users' security status and identifying anomalies" refers to technology for analyzing the status of elderly people and identifying security risks.

[1269] "Communication means for immediately notifying the user when an abnormality is detected" refers to a system or method for quickly notifying the user when an abnormality occurs.

[1270] This invention provides a system for watching over elderly people and monitoring security situations. The configuration of the system and how it is implemented will be specifically described below.

[1271] 1. System Configuration

[1272] Hardware

[1273] Smart glasses: Equipped with a heart rate sensor, accelerometer, gyroscope, GPS, and microphone, these sensors are used to collect data in real time.

[1274] Server: A cloud server (e.g. AWS) that provides an API endpoint for receiving data.

[1275] software

[1276] Smart Glasses App: Uses Python scripts to collect data from sensors and send it to the server.

[1277] Server API: API endpoints built using a web framework such as Flask.

[1278] Data analysis system: Uses AI technologies such as TensorFlow and PyTorch to analyze data in real time.

[1279] Notification system: Use APIs such as Twilio to send emergency notifications via SMS or email.

[1280] 2. Data collection and transmission

[1281] Smart glasses collect the user's heart rate, walking patterns, location information, and voice data in real time. The collected data is transmitted to a server at regular intervals via Wi-Fi or cellular networks. The transmitted data includes time information, sensor values, location information, etc.

[1282] 3. Receipt and analysis of data

[1283] The server receives the data sent from the smart glasses and stores it in a real-time database. The stored data is analyzed using AI technology (e.g., TensorFlow, PyTorch). If an abnormality is detected by comparing it with normal patterns, a notification is sent immediately to family members or caregivers.

[1284] 4. Notification of abnormalities

[1285] If an abnormality is detected, detailed information, including security risks, is sent to family members or caregivers via emergency notification. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., a sudden increase in heart rate or irregular gait). Notification methods include a dedicated app, SMS, email, and phone call.

[1286] 5. Data Visualization

[1287] The server provides a dashboard to visualize the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app.

[1288] Specific examples

[1289] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[1290] The smart glasses capture heart rate (e.g., 70 bpm), walking patterns (stationary), location information (in your living room), and ambient sounds (audio from the TV).

[1291] The acquired data is sent to the server every minute.

[1292] The server receives the data and analyzes it in real time. Since the patient is in a relaxed state, it is determined to be normal.

[1293] After that, the heart rate suddenly rises to 150 bpm, and the walking pattern becomes unstable. This data is immediately sent to the server.

[1294] The server detects this abnormality, determines that "your heart rate is rising sharply and there is a high possibility of you falling," and immediately sends a notification to your family. It also sends a voice notification to the user through the smart glasses saying, "Your heart rate is rising. Please sit down and rest."

[1295] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[1296] Example prompts for generative AI models

[1297] Create an anomaly detection scenario for an elderly security monitoring app. Data collected includes heart rate, walking patterns, and location. Detail how to respond and what notifications are sent when an anomaly is detected.

[1298] This will realize an effective monitoring system to ensure the safety and security of the elderly.

[1299] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1300] Step 1:

[1301] The device collects the user's heart rate, walking patterns, location information, and voice data in real time. Specifically, it uses the device's built-in heart rate sensor, accelerometer, gyroscope, GPS, and microphone to collect this sensor data. The input is the user's biometric information and surrounding environmental sounds, and the output is digital sensor data.

[1302] Step 2:

[1303] The terminal packages the collected sensor data into a batch format at regular intervals (for example, every minute) and sends it to the server. The transmission is done over Wi-Fi or cellular networks using a secure communication protocol such as HTTPS. The input is the sensor data, and the output is an encrypted data packet.

[1304] Step 3:

[1305] The server receives data sent from the device using a dedicated API endpoint and stores the received data in a real-time database. The input is the encrypted data packet, and the output is the decoded sensor data.

[1306] Step 4:

[1307] The server analyzes the data stored in the real-time database using an AI model. Machine learning libraries such as TensorFlow and PyTorch are used for the analysis to detect abnormal patterns from the data. The input is sensor data, and the output is the anomaly detection results and detailed information.

[1308] Step 5:

[1309] If the server detects an abnormality based on the analysis results, it issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high possibility of falling). Notifications are sent via SMS, email, phone, etc. using external communication APIs such as Twilio. The input is the anomaly detection result and detailed information, and the output is a notification message.

[1310] Step 6:

[1311] If an abnormality is detected, the device will notify the user by voice. Specifically, it uses voice synthesis technology to issue a message such as "Your heart rate is increasing. Please sit down and rest." The input is the result of the anomaly detection and the text of the voice message, and the output is the voice message.

[1312] Step 7:

[1313] The server provides a dashboard for visualizing the collected data, which can be accessed by family members or caregivers via a web browser or a dedicated app. The dashboard displays real-time data, abnormality history, and detailed data. The input is sensor data and analysis results, and the output is a graphical dashboard.

[1314] This will enable real-time monitoring of the health and security status of elderly people and rapid response in the event of an abnormality.

[1315] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1316] This invention is a system for monitoring elderly people. It collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. In this embodiment, smart glasses are used as a terminal to seamlessly perform data collection, transmission, analysis, notification, and visualization. Furthermore, this invention combines an emotion engine to recognize the user's emotional state, achieving more comprehensive monitoring.

[1317] System program and its processing overview

[1318] 1. Data Collection

[1319] The device (smart glasses) collects the user's heart rate, walking patterns, location information, surrounding voices, and facial expressions in real time. Specifically, data is collected using the smart glasses' built-in heart rate sensor, accelerometer, gyro sensor, GPS, microphone, camera, etc. The user's voice data and facial expression data are also captured.

[1320] 2. Data Transmission

[1321] The device sends the collected data to the server in batches at regular intervals over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). The data includes time information, sensor values, location information, voice data, and facial expression data.

[1322] 3. Data Reception

[1323] The server receives the data sent from the device. The received data is imported into the server via a dedicated API endpoint and stored in a real-time database. This ensures that the data is stored without loss and can be used for later analysis.

[1324] 4. Data Analysis

[1325] The server analyzes the received data using an AI model. The AI ​​model compares the current data with previously learned normal patterns and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and if an abnormality is detected, immediate action is taken.

[1326] 5. Anomaly detection and notification

[1327] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, sadness emotional state). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[1328] 6. Data Visualization

[1329] The server provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard can be accessed from a web browser or a dedicated app. The dashboard also visualizes the user's emotional state, showing emotional fluctuations and trends.

[1330] Specific examples

[1331] Let's say you're relaxing at home. Because you're wearing smart glasses, the following happens:

[1332] The device acquires heart rate (e.g., 70 bpm), walking pattern (still in a relaxed state), location information (living room at home), ambient sounds (audio from the television), and facial expression data (relaxed facial expression).

[1333] The terminal packages this data in a batch format and sends it to the server every minute.

[1334] The server receives the data from the device and stores it in a database.

[1335] The server uses an AI model and emotion engine to analyze the received data in real time, determining that the state is normal because the person is relaxed.

[1336] The device then detects a sudden increase in heart rate to 150 bpm and an unstable walking pattern, and immediately sends this data to the server. The emotion engine then detects "fear."

[1337] The server detects this as an abnormality, determines that "your heart rate has suddenly increased, you are at high risk of falling, and you are in a state of fear," and immediately sends a notification to your family. It also sends a voice notification to the user via their device saying, "Your heart rate is increasing. Please sit down and rest."

[1338] The server displays detailed data on a dashboard when an abnormality occurs, allowing family members and caregivers to check the data in real time.

[1339] In this way, the present invention allows for real-time monitoring of the elderly's daily life and enables prompt action when abnormalities, including emotional states, are detected, thereby ensuring the safety of the elderly and providing peace of mind to their families and caregivers.

[1340] The processing flow will be explained below.

[1341] Program processing steps

[1342] Step 1:

[1343] The device (smart glasses) collects user data (heart rate, walking patterns, location information, ambient sounds, facial expressions) in real time.

[1344] A heart rate sensor measures the user's heart rate and detects abnormal heart rate variations.

[1345] Acceleration and gyro sensors analyze the user's walking pattern and detect falls and unsteady walking.

[1346] GPS identifies the user's current location.

[1347] A microphone captures the sounds around the user and collects data to analyze tone and emotion.

[1348] A camera captures the user's facial expressions and collects facial expression data.

[1349] Step 2:

[1350] The terminal periodically (for example, every minute) sends the collected data to the server.

[1351] The terminal packages the data in a batch format.

[1352] Data is sent to a server using Wi-Fi or cellular networks.

[1353] Ensure data security by using secure communication protocols (e.g., HTTPS).

[1354] Step 3:

[1355] The server receives the data sent from the terminal.

[1356] Receive data using a dedicated API endpoint.

[1357] The received raw data is stored in a real-time database.

[1358] Step 4:

[1359] The server analyzes the received data in real time.

[1360] It uses AI models to detect abnormalities in heart rate and walking patterns.

[1361] The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., joy, anger, sadness, fear).

[1362] Comparison with existing databases is performed to identify normal data and outliers.

[1363] Step 5:

[1364] If the server detects an abnormality based on the data analysis results, it will immediately issue a notification.

[1365] The server creates a notification that includes the type of anomaly, the user's current location, and detailed data (heart rate, walking pattern, emotional state, etc.).

[1366] Notifications will be sent to family members and caregivers via apps, SMS, email, phone calls, etc.

[1367] A voice message is also sent to the device to notify the user of the abnormality.

[1368] Step 6:

[1369] The server provides a dashboard to visually display the collected data.

[1370] Build a web dashboard or dedicated app that can be accessed by family members or caregivers.

[1371] The dashboard displays current real-time data, historical anomaly data, statistical information, and emotional state.

[1372] Allow users to view data trends and anomaly history through a dashboard.

[1373] As a concrete example, the following occurs while a user is relaxing at home:

[1374] The device captures heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (TV audio), and facial expression data (relaxed facial expression).

[1375] The device sends this data to the server every minute.

[1376] The server receives the data and stores it in a database.

[1377] The server analyzes the data in real time using an AI model and emotion engine to determine whether the person is in a normal relaxed state.

[1378] Suddenly, the heart rate rises to 150 bpm, and data is sent showing an unstable walking pattern, along with data that the emotion engine detects as "fear."

[1379] The server determines this to be an abnormality and sends a notification to the family stating, "Heart rate is rising rapidly, there is a high possibility of falling, and the patient is in a state of fear."

[1380] The device will notify the user by voice, "Your heart rate is increasing. Please sit down and rest."

[1381] The server displays detailed data on abnormalities on a dashboard, allowing family members and caregivers to check the data in real time.

[1382] In this way, the present invention can monitor the daily life of the elderly in real time and respond quickly, including their emotional state, if necessary.

[1383] Example 2

[1384] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1385] Conventional systems for monitoring elderly people rely on limited biometric information acquisition and anomaly detection, and lack comprehensive monitoring that takes into account the user's emotional state. This makes it difficult to respond quickly and accurately when an abnormality is detected, posing a major challenge to ensuring the safety of elderly people and the peace of mind of family members and caregivers.

[1386] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1387] In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using artificial intelligence technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormalities, means for visualizing the notified abnormal data, means for collecting heart rate, walking pattern, location information, voice data, and facial expression data, means for analyzing the voice data and facial expression data using an emotion engine to recognize the emotional state, and means for issuing notifications to the user and third parties based on the abnormality and emotional state. This makes it possible to monitor not only the user's biometric information but also their emotional state in real time, enabling quick and accurate response when an abnormality occurs.

[1388] "User" refers to individuals, especially elderly people, who use the system.

[1389] "Data" refers to information collected from the user, including heart rate, walking patterns, location information, voice data, and facial expression data.

[1390] "Real-time" refers to the immediacy in time that data is acquired and processed immediately.

[1391] A "terminal" is a device for acquiring and transmitting data, specifically referring to smart glasses.

[1392] "Server" refers to a computer system that receives, stores, analyzes data sent from a terminal, and issues notifications as needed.

[1393] "Artificial intelligence technology" refers to technology used to analyze data on servers, including machine learning and deep learning.

[1394] "Emotion engine" refers to a system that analyzes voice data and facial expression data to recognize a user's emotional state.

[1395] "Abnormal" refers to a state that deviates from normal data patterns or a potential problem related to user safety.

[1396] "Notification" refers to warnings and information sent to users and third parties when the server detects an abnormality.

[1397] "Visualization" refers to presenting data as graphs or tables to make it easier to read.

[1398] "Third parties" refer to significant parties with an interest in the user's condition, such as the user's family or caregivers.

[1399] MODE FOR CARRYING OUT THE INVENTION

[1400] The present invention is a system for monitoring elderly people, which collects user data in real time, transmits it to a server, analyzes the data using AI technology and an emotion engine, and provides a means for detecting and notifying abnormalities. Specific embodiments of the present invention will be described below.

[1401] Data collection

[1402] The user wears the smart glasses and goes about their daily life. The smart glasses collect real-time data on their heart rate, walking patterns, location information, surrounding sounds, and facial expressions. The hardware used for this includes a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. For example, if a user is in their living room at home, their heart rate is recorded as 70 bpm, they are standing still, and they can hear the TV.

[1403] Data transmission

[1404] The device (smart glasses) packages the collected data into batches at regular intervals (e.g., every minute) and sends them to a server. This transmission is done via Wi-Fi or cellular networks, and the data is sent using a secure communication protocol (e.g., HTTPS). Examples of data include heart rate, walking patterns, location information, voice data, and facial expression data.

[1405] Data reception

[1406] The server receives data sent from the device via a dedicated API endpoint and stores it in a real-time database, ensuring continuous data storage and availability for later analysis.

[1407] Data analysis

[1408] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., a sudden increase in heart rate, irregular walking, or emotional fluctuations). The emotion engine analyzes the voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear). This analysis is performed in real time, and any abnormalities detected are immediately addressed.

[1409] Anomaly detection and notification

[1410] When the server detects an abnormality, it immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"), etc. Notifications are sent via a dedicated app, SMS, email, phone call, etc. A voice message is also sent to the device to notify the user of the abnormality.

[1411] Data Visualization

[1412] The server provides a dashboard to visually display the collected data. Family members and caregivers can use this dashboard to check the user's detailed data, abnormality history, and real-time status. The dashboard can be accessed from a web browser or a dedicated app, and heart rate fluctuations, movement paths, emotional trends, and other data are visualized in graphs and tables.

[1413] Specific examples

[1414] For example, if a user is relaxing at home, the smart glasses will collect their heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent to the server every minute and analyzed in real time. If the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine will detect "fear." Based on this, the server will determine that "the heart rate has suddenly risen, there is a high risk of falling, and the user is in a state of fear," and will notify family members and the user via voice notification.

[1415] Example prompts for generative AI models

[1416] "Please explain the specific data flow for an elderly care system that collects and analyzes heart rate, walking patterns, location information, voice data, and facial expression data in real time, and notifies users when an abnormality is detected."

[1417] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1418] Step 1:

[1419] The device (smart glasses) collects the user's heart rate, walking pattern, location information, voice data, and facial expression data in real time. The device collects data using a heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. The input is raw data from the sensors, and the output is a collected composite data package. Specifically, the data collected is the user's heart rate of 70 bpm, walking pattern of standing still, location information of the living room, ambient sound of the TV, and facial expression of a relaxed state.

[1420] Step 2:

[1421] The terminal packages the collected data in a batch format at regular intervals (for example, every minute) and sends it to the server. Data is sent over Wi-Fi or cellular networks, using the secure HTTPS communication protocol. The input is the collected composite data package, and the output is an encrypted data packet. Specifically, the data from each sensor is combined into a single packet and sent to the server via the Internet.

[1422] Step 3:

[1423] The server receives data sent from the device via a dedicated API endpoint. The received data is stored in a real-time database to prevent data loss. The input is an encrypted data packet, and the output is structured data stored in the database. Specifically, the server decrypts the received data packet and records the user's heart rate, location information, etc. in the real-time database.

[1424] Step 4:

[1425] The server analyzes the received data using an AI model and emotion engine. The AI ​​model compares normal patterns with the current data to detect abnormal patterns. The emotion engine analyzes the voice data and facial expression data to recognize the emotional state. The input is raw data obtained from the real-time database, and the output is events recognized as abnormal and the detected emotional state. Specifically, the system inputs heart rate and walking patterns into the AI ​​model to determine whether there are any abnormalities. Additionally, the voice data and facial expression data are input into the emotion engine to analyze whether the user is in a state of fear.

[1426] Step 5:

[1427] When an abnormality is detected, the server issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, possibility of falling, emotional state "fear"). The device also sends a voice message to the user. The input is the analysis results of the AI ​​model and emotion engine, and the output is the notification message. Specifically, the notification is sent via a dedicated app, SMS, email, phone call, etc., and the device issues a voice notification saying, "Your heart rate is rising. Please sit down and rest."

[1428] Step 6:

[1429] The server provides a dashboard for visualizing data. Family members and caregivers can check the user's detailed data, abnormality history, and real-time status through the dashboard. The input is data stored in the real-time database, and the output is the visualized dashboard. Specifically, it displays heart rate fluctuations, movement routes, emotional trends, etc. in graphs and tables.

[1430] (Application example 2)

[1431] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1432] In elderly care systems, simply collecting physical data such as heart rate and walking patterns is difficult to accurately grasp the individual's condition. Furthermore, there is insufficient means for quickly and accurately notifying family members or caregivers when an abnormality occurs. Furthermore, there is a need for a system that can monitor emotional fluctuations and respond accordingly.

[1433] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring data from the user in real time, means for transmitting the acquired data to the server, means for using AI technology to analyze the data received by the server, means for detecting abnormalities based on the analysis results, means for notifying the detected abnormality, means for visualizing the notified abnormal data, means for acquiring surrounding voice data and facial expression data using a smart device, means for analyzing the emotional state based on AI technology, and means for quickly issuing a notification based on the abnormality and the emotional state. This makes it possible to monitor not only the physical state of the elderly person but also their emotional state in real time and to quickly and accurately notify the elderly person of any abnormalities.

[1434] "User" refers to an individual who provides data and is monitored by the system.

[1435] "Means for acquiring data in real time" refers to technology and devices for instantly collecting heart rate, walking patterns, location information, voice data, facial expression data, and the like from a user.

[1436] "Means for transmitting data to the server" refers to the communication technologies and protocols for securely and quickly transmitting acquired data to the server.

[1437] "AI technology for analyzing data received by the server" refers to artificial intelligence technology for analyzing data sent to the server and detecting anomalies and patterns.

[1438] "Means for detecting anomalies" refers to the techniques and processes that use AI technology to detect patterns or symptoms that deviate from normal conditions based on received data.

[1439] "Means for notifying abnormalities" refers to notification functions and means for notifying the user, family, or caregivers of detected abnormalities.

[1440] "Means for visualizing notified abnormal data" refers to technology and devices for visually displaying abnormal data in an easy-to-understand manner.

[1441] "Smart device" refers to an electronic device worn or carried by a user for collecting and transmitting data.

[1442] "Means for acquiring voice data and facial expression data" refers to technologies and sensors for capturing and recording the user's voice and facial expressions.

[1443] "Means for analyzing emotional state" refers to techniques and processes that analyze acquired voice data and facial expression data to determine the user's psychological and emotional state.

[1444] "Means for issuing prompt notifications" refers to technologies and functions for sending necessary warnings and information to users without delay based on detected abnormalities and emotional states.

[1445] A system for implementing this invention utilizes a smart device (e.g., smart glasses) worn by a user to acquire user data in real time and transmit it to a server. The data includes heart rate, walking patterns, location information, voice data, facial expression data, etc., and AI technology and an emotion engine analyze the data for abnormalities and emotional states.

[1446] Smart devices are equipped with heart rate sensors, accelerometers, gyroscopes, GPS, microphones, cameras, etc., and use these sensors to collect user data in real time. This data is sent to a server in batch format at regular intervals. The data is sent via Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS).

[1447] The server receives data sent from the device and stores it in a real-time database. The received data is analyzed in real time using an AI model (e.g., TensorFlow) and an emotion engine. The AI ​​model compares normal patterns with the current data and detects abnormal patterns (e.g., sudden increases in heart rate, unstable walking patterns, and emotional fluctuations). The emotion engine analyzes voice and facial expression data to recognize the user's emotional state (e.g., anger, sadness, joy, fear).

[1448] If an abnormality is detected, the server immediately issues a notification to family members or caregivers. The notification includes the user's current location, the type of abnormality, and detailed information (e.g., heart rate 150 bpm, high risk of falling, emotional state sadness, etc.). The notification is sent via a dedicated app, SMS, email, phone, or other means. A voice message is also sent to the device to notify the user of the abnormality.

[1449] The server also provides a dashboard to visually display the collected data. Family members and caregivers can view detailed user data, abnormality history, and real-time raw data through this dashboard. The dashboard, accessible via a web browser or a dedicated app, also visualizes the user's emotional state, displaying emotional fluctuations and trends.

[1450] As a concrete example, consider a situation where a user is relaxing at home. Wearing smart glasses, the device collects the user's heart rate (70 bpm), walking pattern (stationary), location information (living room), ambient sounds (audio from the TV), and facial expression data (relaxed facial expression). This data is sent in batch format to a server, where it is analyzed in real time using AI technology and an emotion engine. Because the user is in a relaxed state, this is deemed normal. However, if the heart rate suddenly rises to 150 bpm and the walking pattern becomes unstable, the emotion engine detects "fear." The server determines this situation as abnormal and issues a notification to family members stating, "The heart rate has risen sharply, there is a risk of falling, and the emotional state is fear." A voice notification can also be sent to the user via the device, enabling a prompt response.

[1451] An example of a prompt sentence might be:

[1452] "User has heart rate of 150 bpm, potential fall, and high-pitched voice tone indicating fear. Detailed location is outside of home. Please take appropriate action."

[1453] This will enable real-time monitoring of not only the physical condition of the elderly but also their emotional state, enabling prompt and accurate responses.

[1454] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1455] Step 1:

[1456] The device collects user data in real time. Inputs include data from the heart rate sensor, accelerometer, gyroscope, GPS, microphone, and camera. Using these sensors, the device obtains the user's heart rate, walking patterns, location information, voice data, and facial expression data. The output is raw data that is temporarily stored on the device and later sent to the server.

[1457] Step 2:

[1458] The device sends the acquired data to the server. The input includes the raw data collected in step 1. This data is packaged in batches at regular time intervals and sent to the server over Wi-Fi or cellular networks using a secure communication protocol (e.g., HTTPS). As an output, a status indicating that the transmission is complete is returned to the device.

[1459] Step 3:

[1460] The server receives the data sent from the devices. The input includes the raw data package sent from step 2. The server stores this in a real-time database. The output is a dataset stored in the database. This ensures that the data is preserved without loss and can be used for further analysis.

[1461] Step 4:

[1462] The server analyzes the data received using an AI model and emotion engine. The input includes a dataset obtained from a real-time database. This dataset is fed into an AI model (e.g., TensorFlow) that compares the current data with previously learned normal patterns. The emotion engine analyzes voice and facial expression data to recognize the user's emotional state. The output is an analysis result that includes abnormal patterns and the user's emotional state.

[1463] Step 5:

[1464] The server detects anomalies based on the analysis results and issues a notification if necessary. The input includes the analysis results obtained in step 4. If an anomaly is detected based on this, the server immediately issues a notification to family members or caregivers. Notification methods include a dedicated app, SMS, email, and phone. As an output, an anomaly notification is sent to family members or caregivers. A voice message is also sent to the device to notify the user of the anomaly.

[1465] Step 6:

[1466] The server provides a dashboard to visually display the collected data. Inputs include real-time and historical data. The server visualizes this data in an easy-to-understand manner and makes it accessible to family members and caregivers via a web browser or dedicated app. Outputs include detailed data, abnormality history, and real-time raw data, all visualized on the dashboard. The user's emotional state is also visualized, showing emotional fluctuations and trends.

[1467] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1468] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1469] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1470] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1471] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1472] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1473] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1474] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1475] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1476] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1477] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1478] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1479] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1480] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1481] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1482] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1483] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1484] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1485] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1486] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1487] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1488] The following is further disclosed regarding the above embodiment.

[1489] (Claim 1)

[1490] a means for obtaining data from a user in real time;

[1491] means for transmitting the acquired data to a server;

[1492] a means for using AI technology to analyze the data received at the server;

[1493] means for detecting anomalies based on the analysis results;

[1494] means for notifying the detected anomaly;

[1495] A means for visualizing the notified abnormal data;

[1496] A system including:

[1497] (Claim 2)

[1498] 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, and voice data.

[1499] (Claim 3)

[1500] 2. The system according to claim 1, wherein when an abnormality is detected, a voice notification or a message notification is given to the user and a third party.

[1501] "Example 1"

[1502] (Claim 1)

[1503] means for acquiring biometric and environmental data from a user in real time;

[1504] A means for transmitting the acquired data to a server at regular intervals;

[1505] A means for storing the data received by the server in a real-time database;

[1506] A means of detecting anomalies using AI technology based on the received data;

[1507] means for notifying the user and a third party of the detected abnormality;

[1508] a means for visually displaying the notified abnormal data;

[1509] A system including:

[1510] (Claim 2)

[1511] 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, and voice data.

[1512] (Claim 3)

[1513] 2. The system according to claim 1, wherein when an abnormality is detected, a voice notification or a message notification is given to the user and a third party.

[1514] "Application Example 1"

[1515] (Claim 1)

[1516] a means for obtaining data from a user in real time;

[1517] means for transmitting the acquired data to a server;

[1518] a means for using AI technology to analyze the data received at the server;

[1519] means for detecting anomalies based on the analysis results;

[1520] means for notifying the detected anomaly;

[1521] A means for visualizing the notified abnormal data;

[1522] a data analysis means for monitoring the security status of users and identifying anomalies;

[1523] A communication means for immediately notifying the user of an abnormality when it is detected;

[1524] A system including:

[1525] (Claim 2)

[1526] 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, audio data, and security status information.

[1527] (Claim 3)

[1528] 2. The system according to claim 1, wherein when an abnormality is detected, a voice or message notification is given to the user and a third party to provide the security status.

[1529] "Example 2: Combining Emotion Engines"

[1530] (Claim 1)

[1531] a means for obtaining data from a user in real time;

[1532] means for transmitting the acquired data to a server;

[1533] means for using artificial intelligence techniques to analyze the data received at the server;

[1534] means for detecting anomalies based on the analysis results;

[1535] means for notifying the detected anomaly;

[1536] A means for visualizing the notified abnormal data;

[1537] A means for collecting heart rate, walking patterns, location information, voice data, and facial expression data;

[1538] means for analyzing voice data and facial expression data using an emotion engine to recognize an emotional state;

[1539] means for issuing notifications to the user and third parties based on the anomaly and emotional state;

[1540] A system including:

[1541] (Claim 2)

[1542] 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, voice data, and facial expression data.

[1543] (Claim 3)

[1544] 2. The system according to claim 1, wherein when an abnormality is detected, a voice notification or a message notification is given to the user and a third party.

[1545] "Application example 2 when combining emotion engines"

[1546] (Claim 1)

[1547] a means for obtaining data from a user in real time;

[1548] means for transmitting the acquired data to a server;

[1549] a means for using AI technology to analyze the data received at the server;

[1550] means for detecting anomalies based on the analysis results;

[1551] means for notifying the detected anomaly;

[1552] A means for visualizing the notified abnormal data;

[1553] A means for acquiring surrounding voice data and facial expression data using a smart device;

[1554] A means for analyzing emotional states based on AI technology;

[1555] a means for issuing prompt notifications based on abnormalities and emotional states;

[1556] A system including:

[1557] (Claim 2)

[1558] 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, voice data, and facial expression data.

[1559] (Claim 3)

[1560] 2. The system according to claim 1, wherein when an abnormality is detected, a voice notification or a message notification is given to the user and a third party. [Explanation of symbols]

[1561] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining data from a user in real time; means for transmitting the acquired data to a server; a means for using AI technology to analyze the data received at the server; means for detecting anomalies based on the analysis results; means for notifying the detected anomaly; A means for visualizing the notified abnormal data; A system including:

2. 10. The system of claim 1, wherein the data includes at least heart rate, walking patterns, location information, and audio data.

3. The system according to claim 1, wherein when an abnormality is detected, a voice notification or a message notification is given to the user and a third party.

Images