system

Abstract

Provide a system. A means for receiving and storing user initial setting information; A means for actively calling out to users on a regular basis and engaging in conversation; A means of monitoring surrounding sounds and movements and detecting abnormalities; A means of notifying emergency contacts if an abnormality is detected; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Social isolation for elderly people living alone, the inability to respond quickly in emergencies, and the difficulty for elderly people to move into rental housing are some of the issues that can be resolved by a system that goes beyond simple monitoring to provide support that allows elderly people to enjoy their daily lives and live with peace of mind. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. Specifically, a system including a means for receiving and saving a user's initial setting information, a means for periodically actively calling and conversing with the user, a means for monitoring surrounding sounds and movements and detecting abnormalities, and a means for notifying an emergency contact if an abnormality is detected, watches over the user's life and ensures safety. Furthermore, the system includes a means for recording the user's living situation data and generating and providing reports, and a means for accepting the user's personality settings and generating conversation content based on the personality, thereby providing a secure and fulfilling daily life for the elderly.

[0006] "User initial setting information" refers to basic information such as name, personality, language, and emergency contact information that is entered or set by the user when using the system for the first time.

[0007] "Means for saving" refers to storage or memory functions for storing user initial setting information and other data and making it accessible as needed.

[0008] "Means for actively calling out and engaging in conversation" refers to functions including speech recognition technology and natural language processing technology that enable the system to speak to the user voluntarily and realize two-way conversation.

[0009] "Means for monitoring surrounding sounds and movements and detecting abnormalities" refers to a function that uses microphones and sensors to analyze environmental sounds and user movements in real time and identify abnormal situations.

[0010] "Means for notifying emergency contacts" refers to the communication means or notification system for notifying registered contacts when an abnormality is detected.

[0011] "Life situation data" refers to data such as voice and activity patterns in the user's daily life, and is used to understand the user's health condition and lifestyle rhythm.

[0012] The "means for generating a report" is a function that automatically creates a report showing the safety and stability of the user's life based on the collected living situation data.

[0013] "Means for providing reports to third parties" refers to communication means or sharing systems for sending or providing generated reports to third parties such as family members, rental owners, facilities, etc. as needed.

[0014] "Personality settings" are information such as individual characters, personalities, and names that are set by the user for the system, allowing the system to respond in a way that is tailored to the user.

[0015] "Means for generating conversation content" refers to a function that includes natural language generation technology that enables the system to generate appropriate responses or next questions based on information input or spoken by the user. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and responding quickly to emergencies. Specific embodiments of the present invention will be described in detail below.

[0038] 1. Initial Setup

[0039] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[0040] Language settings

[0041] name

[0042] Personality settings (e.g., character, personality)

[0043] Emergency Contact Information

[0044] The device sends this initial setup information to the server, which stores it in a database for later access.

[0045] 2. Daily communication

[0046] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[0047] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0048] User: "Good morning, I'm going shopping today."

[0049] Terminal: "That's good. Be careful."

[0050] In this way, the terminal generates conversation content based on the user's personality settings, providing a more friendly conversation.

[0051] 3. Monitoring and Anomaly Detection

[0052] The device has built-in sensors and microphones to monitor surrounding sounds and movements. These sensors analyze and record daily sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[0053] Terminal: "Suzuki-san, are you okay?"

[0054] If there is no response, try again after a few minutes.

[0055] If there is still no reaction, it is determined to be abnormal.

[0056] 4. Emergency call

[0057] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[0058] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0059] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[0060] 5. Solving elderly housing problems

[0061] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[0062] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0063] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[0064] Specific scenario example

[0065] Example 1: Morning conversation scenario

[0066] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0067] User: "Good morning, I slept well last night."

[0068] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0069] Example 2: Anomaly detection scenario

[0070] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0071] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0072] Device: Further no response, so report abnormality to server

[0073] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0074] As described above, the present invention provides a multi-functional system that watches over elderly people to enable them to live their daily lives safely and with peace of mind, and responds quickly in emergencies.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] The user turns on the device and starts the application, which displays a guide for initial setup.

[0078] Step 2:

[0079] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[0080] Step 3:

[0081] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[0082] Step 4:

[0083] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[0084] Step 5:

[0085] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0086] Step 6:

[0087] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[0088] Step 7:

[0089] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[0090] Step 8:

[0091] If an abnormality is detected, such as a loud noise or a long period of silence, the device will ask the user, "Are you OK?"

[0092] Step 9:

[0093] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[0094] Step 10:

[0095] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[0096] Step 11:

[0097] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0098] Step 12:

[0099] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[0100] Step 13:

[0101] The device continuously records daily living situation data and periodically transmits it to the server.

[0102] Step 14:

[0103] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[0104] Step 15:

[0105] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[0106] The above are the specific processing steps of this system.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Current elderly care systems often lack sufficient functionality to properly monitor the surrounding environment and respond quickly in emergencies. Furthermore, due to a lack of daily communication with users to build trust, users may find it difficult to use the system and feel uneasy. Another issue is the lack of a means to report users' living conditions to a third party and prove the safety of the elderly.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes a means for receiving the user's initial setting information and storing it in data storage, a means for actively calling and conversing with the user based on timing, and a means for monitoring surrounding sounds and movements with sensors to detect abnormalities. This allows for appropriate monitoring of the user's living environment and enables rapid response in emergencies. Furthermore, daily communication through active calling based on timing builds a relationship of trust with the user, allowing them to use the system with peace of mind. The system also provides a means for verifying the safety of the elderly by recording the user's living situation data, analyzing it, generating reports, and providing them to third parties.

[0112] "Users" refer to people who use the system, especially elderly people.

[0113] "Initial setting information" refers to information that a user enters when using the system for the first time, and includes language settings, name, personality settings, emergency contact information, etc.

[0114] "Data storage" refers to a storage device for saving a user's initial setting information and living situation data, and refers to a hard disk built into a server or cloud storage.

[0115] "Based on timing" means acting according to pre-set times or conditions.

[0116] "Actively calling out" refers to the system proactively asking messages or questions to the user.

[0117] "Conversing" refers to engaging in voice or text-based communication between a user and a system.

[0118] "Sensor" refers to a device for detecting surrounding sounds and movements, and includes sound sensors and motion sensors.

[0119] "Abnormal" refers to conditions or situations that the system determines to be deviations from normal life patterns, such as loud, unusual noises or long periods of silence.

[0120] "Emergency Contact" refers to a person or organization registered to receive notifications when an abnormality is detected in the user's behavior.

[0121] "Life situation data" is data that records the user's daily life situation, and includes conversation history and sensor data.

[0122] "Analysis" refers to analyzing collected data to extract useful information and patterns.

[0123] A "report" is a written or electronic document summarizing the analyzed living situation data and provided to a third party to confirm the user's safety.

[0124] The present invention is a monitoring system for elderly people that supports their daily lives and provides technology for rapid response in emergencies. This system is used by users using devices such as smartphones and AI speakers. Specific embodiments for carrying out the invention are described below.

[0125] 1. Initial Setup

[0126] The user installs and launches a dedicated application on their smartphone or AI speaker. This application displays a guide for the user to input initial setup information. The initial setup information includes:

[0127] Language settings

[0128] name

[0129] Personality settings (character, personality, etc.)

[0130] Emergency Contact Information

[0131] After inputting this initial setting information, the terminal sends it to a server via the Internet and stores the information in data storage. Specific database management systems used by the server include MySQL (registered trademark) and PostgreSQL.

[0132] 2. Daily communication

[0133] The device actively calls the user periodically and engages in daily communication. For example, the following conversation takes place every morning at 8:00:

[0134] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0135] User: "Good morning. I'm going shopping today."

[0136] Terminal: "That's good. Be careful."

[0137] This communication is done using voice recognition software (e.g., Google® Speech-to-Text) and generative AI models, which generate more friendly conversational content based on the user's personality settings.

[0138] 3. Monitoring and Anomaly Detection

[0139] The device monitors the user's surroundings using built-in sensors (e.g., sound sensors, motion sensors) and microphones. This allows it to record and analyze everyday sounds and movement patterns in real time. When certain abnormal conditions occur (e.g., loud noises or long periods of silence), the device detects the abnormality as follows:

[0140] Terminal: "Suzuki-san, are you okay?"

[0141] Call again after a few minutes, and if there is still no response, assume that something is wrong.

[0142] 4. Emergency call

[0143] If an abnormality is detected, the device immediately sends this information to the server, which then sends a notification to the registered emergency contacts via SMS or email (e.g., Twilio, SendGrid). The notification content is as follows:

[0144] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0145] 5. Solving elderly housing problems

[0146] The device records the user's daily life situation and sends it to the server. The server analyzes the received data and generates periodic reports. These reports are generated using data analysis software (e.g., Python's Pandas library). The report contents are provided as follows:

[0147] Server: "Mr. Suzuki has been living his daily life stably for the past week, with no notable abnormalities."

[0148] This allows family members and other people involved to confirm the safety of the user.

[0149] Prompt Sentence Examples

[0150] Here are some example prompts to use when inputting a scenario into a generative AI model:

[0151] "We want to create a system that helps elderly people live their daily lives with peace of mind. Please generate scenarios that involve daily conversations, anomaly detection, and emergency calls. Please include specific examples of morning conversations and anomaly detection."

[0152] The above is a detailed description of the mode for carrying out the invention, and by using this system, elderly people can live their daily lives safely and with peace of mind.

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] System program processing flow

[0155] Initial Setup

[0156] Step 1:

[0157] Users install and launch a dedicated application on their smartphone or AI speaker.

[0158] Input: Install and launch the application

[0159] Output: The screen where the application starts

[0160] Step 2:

[0161] The terminal displays an initial setting screen for the application and prompts the user to input initial setting information.

[0162] What happens: A user enters language preferences, name, personality preferences, and emergency contact information into an application.

[0163] Input: User-entered initial configuration information

[0164] Output: The entered information is displayed on the terminal.

[0165] Step 3:

[0166] The terminal transmits the input initial setting information to a server via the Internet.

[0167] Specific operation: Sends information using the device's network communication function

[0168] Input: Initial setting information entered into the terminal

[0169] Output: Initial configuration information sent to the server

[0170] Step 4:

[0171] The server stores the received initial setting information in a data storage.

[0172] Specific operation: Use a database management system (MySQL or PostgreSQL)

[0173] Input: Received initial setting information

[0174] Output: Initial settings information stored in the database

[0175] Daily communication

[0176] Step 1:

[0177] The device uses an internal clock and timer function to call the user at the set time.

[0178] Specific behavior: Send a message to the user every morning at 8am

[0179] Input: Set time information

[0180] Output: A message to the user

[0181] Step 2:

[0182] The device uses voice recognition software (Google Speech-to-Text) to analyze the user's responses and generate appropriate replies.

[0183] What it does: Converts user speech to text and generates a response using a generative AI model.

[0184] Input: User's voice response

[0185] Output: Terminal-generated reply message

[0186] Specific working example:

[0187] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0188] User: "Good morning. I'm going shopping today."

[0189] Terminal: "That's good. Be careful."

[0190] Monitoring and Anomaly Detection

[0191] Step 1:

[0192] The device uses built-in sensors (sound sensor, motion sensor) and microphone to monitor the user's surroundings.

[0193] Specific operation: Sensors monitor the living environment in real time

[0194] Input: Sound and movement data from the surrounding area

[0195] Output: Monitoring data

[0196] Step 2:

[0197] The device analyzes the collected sensor data and detects abnormalities when certain abnormal conditions occur (loud noises, long periods of silence).

[0198] Specific operation: Real-time data analysis

[0199] Input: Sensor data

[0200] Output: Anomaly detection results

[0201] Specific working example:

[0202] Terminal: "Suzuki-san, are you okay?"

[0203] If there is no response, try again in a few minutes.

[0204] If there is still no reaction, it is considered abnormal.

[0205] emergency call

[0206] Step 1:

[0207] If an abnormality is detected, the terminal immediately transmits the information to the server.

[0208] Specific operation: Sends anomaly information via internet connection

[0209] Input: Anomaly detection information

[0210] Output: Anomaly information sent to the server

[0211] Step 2:

[0212] The server sends a notification to the registered emergency contact based on the received abnormality information.

[0213] Specific operation: Uses SMS and email sending systems (Twilio, SendGrid)

[0214] Input: Anomaly information

[0215] Output: Notification message to emergency contacts

[0216] Specific working example:

[0217] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[0218] Solving elderly housing problems

[0219] Step 1:

[0220] The terminal records the user's daily life situation and periodically transmits it to the server.

[0221] Specific operation: Uses the device's local storage and internet connection

[0222] Input: Daily life situation data

[0223] Output: Status data sent to the server

[0224] Step 2:

[0225] The server analyzes the received data and generates periodic reports.

[0226] Specific operation: Data analysis software (Python's Pandas library)

[0227] Input: Received living situation data

[0228] Output: Generated report

[0229] Specific working example:

[0230] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and provide it to his family and related parties.

[0231] The above is the specific flow of the system program processing and details of each step.

[0232] (Application example 1)

[0233] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0234] There is a need to ensure rapid response in emergencies while monitoring the safety of elderly people's daily lives. However, many current systems have low accuracy in detecting abnormalities, and visual and auditory monitoring in particular is insufficient. Furthermore, they are limited to simple warning sounds and notifications and lack functionality for daily communication with users, which can lead to a decline in the quality of daily life. Another issue is the lack of detailed recording and reporting of abnormal situations and reliable notification to emergency contacts.

[0235] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0236] In this invention, the server includes means for receiving and saving the user's initial setting information, means for periodically actively calling and conversing with the user, means for monitoring surrounding sounds and movements and detecting abnormalities, means for notifying an emergency contact when an abnormality is detected, means for monitoring audio, video, and motion data using sensors and cameras, and means for checking multiple times if there is no response and determining that there is an abnormality if there is still no response. This makes it possible to comprehensively monitor the lives of the elderly and respond quickly and effectively in emergencies.

[0237] The "means for receiving and saving user initial setting information" is a function for collecting information that a user inputs when using the system for the first time and storing it in a database.

[0238] "Means for actively calling out to and conversing with the user on a regular basis" refers to a function in which the system speaks to and communicates with the user based on a pre-set schedule.

[0239] "Means for monitoring surrounding sounds and movements and detecting abnormalities" refers to a system that uses sound sensors and motion sensors to constantly monitor environmental information and detect abnormalities.

[0240] "Means of notifying emergency contacts when an abnormality is detected" is a function that sends warnings and notifications to pre-registered emergency contacts when the system recognizes an abnormality.

[0241] "Means for monitoring audio, video, and motion data using sensors and cameras" refers to a function that utilizes sensors and cameras to acquire and monitor data such as audio, video, and motion.

[0242] "Means for making multiple confirmations if there is no response, and determining that an abnormality exists if there is still no response" is a function that makes multiple confirmation calls if the user does not respond within a certain period of time, and determines that an abnormality exists if no response is still received.

[0243] The present invention is a system for comprehensively monitoring the lives of elderly people and responding quickly and effectively in emergencies. The system includes a means for receiving and storing user initial setup information, including name, emergency contact information, and personality settings. This allows the device to respond appropriately based on the user's unique information.

[0244] The system has a means to actively call out to the user periodically and engage in conversation. For example, every morning at 8:00, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?" This conversation is generated based on the user's personality settings, making it more friendly.

[0245] The device is also equipped with a means of monitoring surrounding sounds and movements to detect abnormalities. Audio, video, and motion data are monitored using sensors and cameras. If an abnormality is detected, the device immediately reports this information to the server. The server then sends a notification to the registered emergency contacts. In addition, if there is no response, the device will check multiple times and, if there is still no response, will determine that an abnormality has occurred. This ensures a prompt response when an elderly person is in a dangerous situation.

[0246] The user's living situation data is constantly recorded and generated as a report. This report is automatically generated using a generative AI model and provided to family members or medical institutions as needed. This allows those involved to constantly keep track of the user's health and living situation.

[0247] The following are specific examples of the present invention.

[0248] Example 1:

[0249] 1. The user puts on the smart glasses and launches the application.

[0250] 2. The system speaks to the user, "Good morning, Suzuki-san. What are your plans for today?"

[0251] 3. The user replies, "Good morning, I'm going shopping today."

[0252] 4. The system responds, "That's good. Be careful."

[0253] Example 2:

[0254] 1. The system monitors surrounding sounds and activity.

[0255] 2. Suddenly, a loud noise occurs and the system detects an abnormality.

[0256] 3. The system calls out, "Mr. Suzuki, are you okay?" but there is no response.

[0257] 4. He then calls out a second time, "Mr. Suzuki, please respond," but there is still no response.

[0258] 5. The system reports the abnormality to the server, and the server sends a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0259] An example of a prompt sentence to input to the generative AI model is as follows:

[0260] "Apply the invention of the elderly monitoring system to create an application scenario for a security service using smart glasses."

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] The device receives and stores the user's initial setting information. The initial setting information (name, emergency contact, personality settings) is received from the user, sent to the server, and stored in a database. This allows the device to respond appropriately based on the user's unique information.

[0264] Step 2:

[0265] It periodically actively calls out to the user and starts a conversation. The user's name and personality settings are obtained from the server as input, and the generative AI model generates the conversation content as output, which the device then relays to the user via voice. For example, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0266] Step 3:

[0267] Sensors and cameras are used to monitor audio, video, and motion data. The device monitors surrounding sounds and motion in real time, taking environmental data as input. It analyzes the data to detect abnormalities, and if no abnormalities are found, records the data as normal, and if an abnormality is detected, moves on to the next step.

[0268] Step 4:

[0269] If an abnormality is detected, a response is checked. The terminal calls out, "Mr. Suzuki, are you OK?", but if there is no response from the user as input, the check is made multiple times (for example, again after 5 minutes). The output confirms that there is no response, and once this step is complete, the process proceeds to the next step.

[0270] Step 5:

[0271] The abnormality is reported to the server and an emergency contact is notified. As input, the device sends "no response" data to the server, and as output, the server sends a notification to the registered emergency contact. The content of the notification is "An abnormality has been detected in Suzuki's situation. Please check immediately."

[0272] Step 6:

[0273] It records the user's living situation data and generates a report. Data from sensors and cameras is sent as input to a server, and a living situation report is automatically generated using a generative AI model. The report generated as output is provided to family members and medical institutions. This allows those involved to constantly keep track of the user's health condition and living situation.

[0274] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0275] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and quickly responding to emergencies. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, thereby providing a more detailed and human-like response. Specific embodiments of the present invention are described in detail below.

[0276] 1. Initial Setup

[0277] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[0278] Language settings

[0279] name

[0280] Personality settings (character, personality)

[0281] Emergency Contact Information

[0282] The device sends this initial setup information to the server, which stores it in a database for later access.

[0283] 2. Daily communication

[0284] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[0285] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0286] User: "Good morning, I'm going shopping today."

[0287] Terminal: "That's good. Be careful."

[0288] 3. Emotional engine response

[0289] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[0290] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[0291] 4. Monitoring and Anomaly Detection

[0292] The device monitors surrounding sounds and movements using sensors and microphones. These sensors analyze and record everyday sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[0293] Terminal: "Suzuki-san, are you okay?"

[0294] If there is no response, try again after a few minutes.

[0295] If there is still no reaction, it is determined to be abnormal.

[0296] 5. Emergency call

[0297] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[0298] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0299] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[0300] 6. Solving the housing problem for elderly people

[0301] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[0302] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0303] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[0304] Specific scenario example

[0305] Example 1: Morning conversation scenario

[0306] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0307] User: "Good morning, I slept well last night."

[0308] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0309] Example 2: Emotion Recognition Scenario

[0310] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[0311] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[0312] Terminal: "That's concerning. Should I call for help right away?"

[0313] Example 3: Anomaly detection scenario

[0314] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0315] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0316] Device: Further no response, so report abnormality to server

[0317] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0318] As described above, the present invention is a multi-functional system that watches over elderly people to ensure that they can live their daily lives safely and with peace of mind, responds quickly in emergencies, and uses an emotion engine to provide flexible responses that correspond to the user's emotions.

[0319] The processing flow will be explained below.

[0320] Step 1:

[0321] The user turns on the device and starts the application, which displays a guide for initial setup.

[0322] Step 2:

[0323] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[0324] Step 3:

[0325] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[0326] Step 4:

[0327] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[0328] Step 5:

[0329] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0330] Step 6:

[0331] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[0332] Step 7:

[0333] The device analyzes the user's tone of voice and word choice and uses an emotion engine to recognize the user's emotions.

[0334] Step 8:

[0335] The device generates the following responses depending on the user's emotions. For example, if the user speaks in an anxious voice, the device responds, "Mr. Suzuki, are you worried about something today? Please tell me."

[0336] Step 9:

[0337] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[0338] Step 10:

[0339] If an abnormality is detected, such as a loud noise or a long period of silence, the device will call out to the user, saying something like, "Mr. Suzuki, are you OK?"

[0340] Step 11:

[0341] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[0342] Step 12:

[0343] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[0344] Step 13:

[0345] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0346] Step 14:

[0347] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[0348] Step 15:

[0349] The device continuously records daily living situation data and periodically transmits it to the server.

[0350] Step 16:

[0351] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[0352] Step 17:

[0353] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[0354] The above are the specific processing steps of this system, which monitors the user's life and provides flexible responses according to their emotions.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Today's elderly people face a variety of problems in their daily lives. These include feelings of loneliness and anxiety about their safety, and they often find themselves unable to receive the support they need promptly. Furthermore, it is difficult for their families and relatives to understand the elderly's living situation, which risks delaying appropriate responses. There is a need to develop a system that can solve these problems and enable the elderly to live their daily lives with peace of mind.

[0358] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving and saving the user's initial setting information, a means for actively calling out to the user periodically and holding a conversation, a means for analyzing the user's tone of voice, facial expression, and words selected to identify emotions, a means for monitoring surrounding sounds and movements and detecting abnormalities, and a means for notifying an emergency contact when an abnormality is detected. This enables the system to appropriately monitor the living conditions of the elderly and automatically take appropriate measures as necessary.

[0359] "Initial setup information" refers to basic information that a user enters when they first start using the system, including language settings, name, personality settings, and emergency contact information.

[0360] "Active calling" is a function that enables the system to communicate with the user on a regular basis by proactively speaking to the user.

[0361] "Means for conducting conversation" refers to a function for exchanging information with a user using voice or text.

[0362] "Voice tone" refers to characteristics such as pitch, strength, and rhythm of a user's voice when they speak, and is used to identify emotions.

[0363] "Facial expression" refers to facial movements and changes in facial muscles, and is an important factor in identifying emotions.

[0364] "Selected words" refer to words and phrases that users use in conversation and are used to identify emotions.

[0365] "Means for identifying emotions" is a function that determines the user's emotional state by analyzing the user's tone of voice, facial expressions, and words they choose.

[0366] "Means for monitoring surrounding sounds and movements" refers to a function that uses sensors and microphones to detect environmental sounds and movements and discover abnormalities.

[0367] "Means for detecting abnormalities" refers to a function that recognizes an abnormal situation when the system meets pre-set conditions (e.g., a loud abnormal noise or a long period of silence).

[0368] "Emergency contacts" are people to whom notifications are sent when an abnormality is detected, including family members and local medical institutions.

[0369] "Life situation data" refers to various information related to the user's daily life, such as behavioral patterns and health status.

[0370] A "report" is a document in the form of a report generated based on data collected by the system, detailing the user's living situation.

[0371] "Personality setting" refers to the user pre-setting their own character and personality within the system, and is used to customize the content of conversations.

[0372] This system monitors the lives of the elderly, enriching their daily lives while also responding quickly to emergencies. Furthermore, by incorporating an emotion engine that recognizes the user's emotions and responds appropriately based on those emotions, the system offers a more detailed and human-like response.

[0373] Basic system configuration

[0374] The system mainly includes the following elements:

[0375] User

[0376] Device (e.g., AI speaker, smartphone)

[0377] server

[0378] Emotion Engine

[0379] Sensors (e.g. microphone, motion sensor)

[0380] 1. Initial Setup

[0381] When a user uses the system for the first time, they install and launch the application on their device (such as an AI speaker or smartphone). When the application launches, the user is prompted to enter initial information, including language settings, name, personality settings, and emergency contact information. The device then sends the entered information to the server.

[0382] The server stores the received initial setup information in a database for future access. Specifically, the "initial setup information" is basic information that a user enters when they start using the system, such as language settings, name, personality settings, and emergency contact information.

[0383] 2. Daily communication

[0384] By using the system's periodic communication function, the device actively engages with the user and engages in daily conversations. For example, the following conversation takes place every morning at 8:00:

[0385] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0386] User: "Good morning, I'm going shopping today."

[0387] Terminal: "That's good. Be careful."

[0388] This allows users to reduce feelings of loneliness in their daily lives and receive psychological support through the system.

[0389] 3. Emotional engine response

[0390] The device analyzes the user's tone of voice, facial expressions, and choice of words, and uses an emotion engine to identify the user's emotions. For example, if the user speaks with anxiety:

[0391] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[0392] A concrete example of this "means for identifying emotions" is to analyze changes in the pitch and tone of a user's voice and the words they choose when they are feeling anxious or worried, and then use the emotion engine to determine that they are feeling "anxiety." This allows the system to respond appropriately to the user's psychological state.

[0393] 4. Monitoring and Anomaly Detection

[0394] The device uses sensors and microphones to monitor surrounding sounds and movements. Specifically, the sensors analyze and record everyday sounds and movements in real time. The system detects specific anomalies and takes immediate action:

[0395] Terminal: "Suzuki-san, are you OK?" (If there is no response, try again after a few minutes.)

[0396] If there is still no response, it will be determined that something is wrong and an emergency call will be made.

[0397] 5. Emergency call

[0398] If an abnormality is detected, the device immediately reports this information to the server, which then sends a notification to registered emergency contacts, such as:

[0399] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0400] If necessary, preparations are being made to notify local medical institutions and welfare facilities.

[0401] 6. Solving the housing problem for elderly people

[0402] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[0403] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0404] This report is provided to families, rental owners, facilities, etc. to prove that users are living safely.

[0405] Examples of prompts include:

[0406] Example 1: Morning conversation scenario

[0407] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0408] User: "Good morning, I slept well last night."

[0409] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0410] Example 2: Emotion Recognition Scenario

[0411] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[0412] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[0413] Terminal: "That's concerning. Should I call for help right away?"

[0414] Example 3: Anomaly detection scenario

[0415] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0416] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0417] Device: Further no response, so report abnormality to server

[0418] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0419] This system not only enables elderly people to live their daily lives safely and with peace of mind, but also enables appropriate and prompt responses when necessary. Furthermore, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotions, providing psychological support to the user.

[0420] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0421] Step 1: Entering user information and initial settings

[0422] Input: The user enters initial configuration information (e.g., language preference, name, personality preference, emergency contact information) into the application.

[0423] How it works: The user launches the application on their device and enters the required information following the prompts.

[0424] Data processing: The terminal organizes the input information into a data structure and sends it to the server.

[0425] Output: Initial configuration information sent to the server.

[0426] Step 2: Save the initial configuration information

[0427] Input: Initial setting information sent from the terminal.

[0428] What it does: The server stores the received information in a database.

[0429] Data processing: The received initial setting information is organized into appropriate fields for storage in a database.

[0430] Output: Initial configuration information stored in the database.

[0431] Step 3: Regularly check in and keep the conversation going

[0432] Input: User schedule and past conversation history read from the server.

[0433] What it does: The device automatically calls the user at a set time (e.g., 8:00 every morning).

[0434] Data calculation: The server generates optimal conversation prompts from past data and sends them to the device.

[0435] Output: A specific call to action for the user.

[0436] Examples:

[0437] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0438] User: "Good morning, I'm going shopping today."

[0439] Terminal: "That's good. Be careful."

[0440] Step 4: Identify the emotion

[0441] Input: Data on the user's tone of voice, speech, and facial expressions.

[0442] Operation: The device sends collected voice data to the emotion engine.

[0443] Data Calculation: The emotion engine analyzes tone of voice and selected words to identify emotions.

[0444] Output: Identified emotional state.

[0445] Examples:

[0446] The user says in an anxious voice, "I'm not feeling well."

[0447] The device will identify "anxiety" through its emotion engine and respond, "That sounds worrying. Would you like us to call for support right away?"

[0448] Step 5: Surrounding area monitoring and anomaly detection

[0449] Input: Environmental sound data and movement data from sensors and microphones.

[0450] Operation: The device monitors surrounding sounds and movements in real time.

[0451] Data processing: Analyze collected data to detect abnormal patterns (e.g., loud noises or long periods of silence).

[0452] Output: Alert information if an abnormality is detected.

[0453] Examples:

[0454] Terminal: Detects an abnormal noise and calls out, "Suzuki-san, are you okay?"

[0455] If there is no response, call again after a few minutes, and if there is still no response, it is determined to be an "abnormality."

[0456] Step 6: Call Emergency Services

[0457] Input: Alert information that an anomaly has been detected.

[0458] Action: The server notifies emergency contacts.

[0459] Data processing: Check emergency contact information and convert the anomaly detection content into a message.

[0460] Output: Notification sent to emergency contacts.

[0461] Examples:

[0462] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[0463] Step 7: Recording living situation data and generating reports

[0464] Input: Daily activity data sent from your device.

[0465] Operation: The server continuously records the living situation data.

[0466] Data Computing: Analyzes collected data and generates periodic reports.

[0467] Output: Living situation data compiled as a report.

[0468] Examples:

[0469] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and send it to the relevant parties.

[0470] (Application example 2)

[0471] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0472] In recent years, the number of elderly people living alone has been increasing, creating a need for monitoring systems that allow elderly people to live their daily lives safely and with peace of mind. However, conventional monitoring systems simply detect abnormalities and are unable to understand the emotions and psychological state of the elderly or provide appropriate responses. Therefore, a system that can not only respond to emergencies but also enrich the daily lives of elderly people and provide a sense of security is needed.

[0473] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0474] In this invention, the server includes means for receiving and storing the user's initial setting information, means for periodically actively calling out to the user and engaging in conversation, means for monitoring surrounding sounds and movements to detect abnormalities, means for notifying an emergency contact when an abnormality is detected, and means for understanding the user's emotions using voice recognition and an emotion engine and responding appropriately, allowing the elderly to live their daily lives with peace of mind and enabling quick and appropriate responses in emergencies.

[0475] A "user" is an individual who performs initial setup to use the system.

[0476] "Preferences Information" refers to basic information such as a user's name, language preference, personality preference, and emergency contact information.

[0477] "Actively calling out" refers to the system proactively speaking to the user and initiating a dialogue.

[0478] "Speech recognition" refers to the technology of analyzing a user's voice and understanding its content.

[0479] An "emotion engine" refers to technology that analyzes a user's emotional state based on their tone of voice, facial expressions, and choice of words.

[0480] "Surrounding sounds and movements" refers to sounds and movements of the user's body within the user's living environment.

[0481] "Means for detecting abnormalities" refers to a system that uses sensors and microphones to analyze surrounding sounds and movements in real time and identify abnormal situations.

[0482] "Emergency contact" is contact information registered by the user to receive notifications in the event of an emergency.

[0483] "Means for emergency notification" refers to a system that sends a notification to emergency contacts or appropriate authorities when an abnormality is detected.

[0484] "Lifestyle data" refers to data related to the user's daily activities and health condition.

[0485] "Means for generating reports" refers to a system that periodically creates reports summarizing the current status of the user based on collected living situation data.

[0486] "Third Party" refers to the user's family, medical institution, or other party providing necessary assistance.

[0487] "Personality settings" refers to a feature that allows users to set their own personality and preferences for the system.

[0488] "Means for responding appropriately" refers to technology that enables the system to generate appropriate responses depending on the user's emotional state and the content of the conversation.

[0489] The present invention is a system that assumes that the user is an elderly person, monitors their daily life, and responds quickly in the event of an emergency. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, providing a more detailed and human-like response.

[0490] 1. Initial Setup

[0491] When a user uses the system for the first time, they install and launch the application on their smartphone or other device. The application displays a guide for initial setup and prompts them to enter the following information:

[0492] Language settings

[0493] name

[0494] Personality settings (character and preferences)

[0495] Emergency Contact Information

[0496] The terminal transmits this initial setting information to the server, and the server stores the received information in a database.

[0497] 2. Daily communication

[0498] Based on the settings, the system will actively call out to the user and have a conversation with them periodically. For example, every morning at 8:00, the following conversation will take place:

[0499] Terminal: "Good morning, <username>. What are your plans for today?"

[0500] User: "Good morning, I'm going shopping today."

[0501] Terminal: "That's good. Be careful on your way."

[0502] 3. Emotional Engine Response

[0503] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[0504] Terminal: "Hey <username>, is there something bothering you today? Let me know what you think."

[0505] 4. Monitoring and Anomaly Detection

[0506] The device uses sensors and microphones to monitor surrounding sounds and movements, analyzing them in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies by following these steps:

[0507] Terminal: "<username>, are you okay?"

[0508] If there is no response, try again after a few minutes.

[0509] If there is still no reaction, it is determined to be abnormal.

[0510] 5. Emergency call

[0511] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[0512] "An abnormality has been detected in <user name>'s status. Please check immediately."

[0513] If necessary, local medical institutions and welfare facilities will also be notified.

[0514] 6. Solving the housing problem for elderly people

[0515] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[0516] "<User name> has been living a stable daily life for the past week, with no notable abnormalities."

[0517] This report can be provided to family and relatives as needed to prove that the user is living safely.

[0518] Hardware and software used

[0519] EmotionRecognition: Emotion recognition engine

[0520] VoiceInterface: Voice Interface

[0521] SensorManager: Environmental sensor management

[0522] notify_emergency_contacts: Emergency contacts

[0523] generate_report: Report generation function

[0524] Specific examples

[0525] For example, consider the following scenario:

[0526] Terminal: "Good morning, <username>. Did you sleep well last night?"

[0527] User: "Good morning, I slept well last night."

[0528] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0529] Prompt Sentence Examples

[0530] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[0531] information:

[0532] Watching over the daily lives of the elderly

[0533] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[0534] Notify emergency contacts and medical institutions when an abnormality is detected

[0535] Generate regular lifestyle reports and provide them to families and owners.

[0536] The above is a specific embodiment for carrying out the present invention.

[0537] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0538] Step 1:

[0539] Receiving and saving initial configuration information

[0540] A user installs the application on a device such as a smartphone and enters initial information (language settings, name, personality settings, emergency contact information). The device sends this information to the server, which stores it in a database.

[0541] Input: Language settings, name, personality settings, emergency contact information

[0542] Output: Initial configuration information stored in the server database

[0543] Step 2:

[0544] Beginning of daily communication

[0545] Based on the settings, the device will actively call out to the user periodically. For example, every morning at 8:00, it might say, "Good morning, <user name>. What are your plans for today?" The device will then analyze the user's response with a speech recognition engine and generate an appropriate response.

[0546] Input: User's spoken response

[0547] Output: Speech recognition results, voice response from the device

[0548] Step 3:

[0549] Emotion engine response

[0550] The device uses an emotion engine to analyze the user's tone of voice and choice of words to identify their emotional state. For example, if the user speaks in an anxious voice, the device can respond based on that by saying, "Hey <username>, are you worried about something today? Let me know."

[0551] Input: User's voice data

[0552] Output: Emotional state identification result, appropriate response voice

[0553] Step 4:

[0554] Surrounding sound and movement monitoring and anomaly detection

[0555] The device uses sensors and microphones to monitor surrounding sounds and movements in real time. If an abnormal sound or movement is detected (loud noises or long periods of silence), the device will:

[0556] First, say, "<user name>, are you okay?"

[0557] If there is no response, try again after a few minutes.

[0558] If there is still no reaction, it is determined to be abnormal.

[0559] Input: ambient sound and movement data

[0560] Output: Anomaly detection result, voice call from device

[0561] Step 5:

[0562] emergency call

[0563] If an abnormality is detected, the device immediately reports the information to the server, which then sends a notification to registered emergency contacts and, if necessary, to local medical institutions and welfare facilities.

[0564] Input: Anomaly detection results

[0565] Output: Notification to emergency contacts

[0566] Step 6:

[0567] Generate scheduled reports

[0568] The server continuously records the user's living situation data and periodically generates reports, which are provided to family members and other interested parties as needed.

[0569] Input: User's life situation data

[0570] Output: Living situation report

[0571] Specific examples

[0572] For example, consider the following scenario:

[0573] In step 2, the device says, "Good morning, <user name>. Did you sleep well last night?", and the user responds, "Good morning, I slept well last night." In step 3, the device responds, "That's good. The weather is nice today, so it would be nice to go for a walk in the daytime."

[0574] Prompt Sentence Examples

[0575] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[0576] information:

[0577] Watching over the daily lives of the elderly

[0578] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[0579] Notify emergency contacts and medical institutions when an abnormality is detected

[0580] Generate regular lifestyle reports and provide them to families and owners.

[0581] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0582] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0583] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0584] [Second embodiment]

[0585] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0586] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0587] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0588] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0589] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0590] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0591] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0592] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0593] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0594] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0595] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0596] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0597] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and responding quickly to emergencies. Specific embodiments of the present invention will be described in detail below.

[0598] 1. Initial Setup

[0599] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[0600] Language settings

[0601] name

[0602] Personality settings (e.g., character, personality)

[0603] Emergency Contact Information

[0604] The device sends this initial setup information to the server, which stores it in a database for later access.

[0605] 2. Daily communication

[0606] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[0607] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0608] User: "Good morning, I'm going shopping today."

[0609] Terminal: "That's good. Be careful."

[0610] In this way, the terminal generates conversation content based on the user's personality settings, providing a more friendly conversation.

[0611] 3. Monitoring and Anomaly Detection

[0612] The device has built-in sensors and microphones to monitor surrounding sounds and movements. These sensors analyze and record daily sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[0613] Terminal: "Suzuki-san, are you okay?"

[0614] If there is no response, try again after a few minutes.

[0615] If there is still no reaction, it is determined to be abnormal.

[0616] 4. Emergency call

[0617] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[0618] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0619] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[0620] 5. Solving elderly housing problems

[0621] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[0622] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0623] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[0624] Specific scenario example

[0625] Example 1: Morning conversation scenario

[0626] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0627] User: "Good morning, I slept well last night."

[0628] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0629] Example 2: Anomaly detection scenario

[0630] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0631] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0632] Device: Further no response, so report abnormality to server

[0633] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0634] As described above, the present invention provides a multi-functional system that watches over elderly people to enable them to live their daily lives safely and with peace of mind, and responds quickly in emergencies.

[0635] The processing flow will be explained below.

[0636] Step 1:

[0637] The user turns on the device and starts the application, which displays a guide for initial setup.

[0638] Step 2:

[0639] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[0640] Step 3:

[0641] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[0642] Step 4:

[0643] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[0644] Step 5:

[0645] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0646] Step 6:

[0647] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[0648] Step 7:

[0649] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[0650] Step 8:

[0651] If an abnormality is detected, such as a loud noise or a long period of silence, the device will ask the user, "Are you OK?"

[0652] Step 9:

[0653] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[0654] Step 10:

[0655] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[0656] Step 11:

[0657] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0658] Step 12:

[0659] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[0660] Step 13:

[0661] The device continuously records daily living situation data and periodically transmits it to the server.

[0662] Step 14:

[0663] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[0664] Step 15:

[0665] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[0666] The above are the specific processing steps of this system.

[0667] Example 1

[0668] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0669] Current elderly care systems often lack sufficient functionality to properly monitor the surrounding environment and respond quickly in emergencies. Furthermore, due to a lack of daily communication with users to build trust, users may find it difficult to use the system and feel uneasy. Another issue is the lack of a means to report users' living conditions to a third party and prove the safety of the elderly.

[0670] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0671] In this invention, the server includes a means for receiving the user's initial setting information and storing it in data storage, a means for actively calling and conversing with the user based on timing, and a means for monitoring surrounding sounds and movements with sensors to detect abnormalities. This allows for appropriate monitoring of the user's living environment and enables rapid response in emergencies. Furthermore, daily communication through active calling based on timing builds a relationship of trust with the user, allowing them to use the system with peace of mind. The system also provides a means for verifying the safety of the elderly by recording the user's living situation data, analyzing it, generating reports, and providing them to third parties.

[0672] "Users" refer to people who use the system, especially elderly people.

[0673] "Initial setting information" refers to information that a user enters when using the system for the first time, and includes language settings, name, personality settings, emergency contact information, etc.

[0674] "Data storage" refers to a storage device for saving a user's initial setting information and living situation data, and refers to a hard disk built into a server or cloud storage.

[0675] "Based on timing" means acting according to pre-set times or conditions.

[0676] "Actively calling out" refers to the system proactively asking messages or questions to the user.

[0677] "Conversing" refers to engaging in voice or text-based communication between a user and a system.

[0678] "Sensor" refers to a device for detecting surrounding sounds and movements, and includes sound sensors and motion sensors.

[0679] "Abnormal" refers to conditions or situations that the system determines to be deviations from normal life patterns, such as loud, unusual noises or long periods of silence.

[0680] "Emergency Contact" refers to a person or organization registered to receive notifications when an abnormality is detected in the user's behavior.

[0681] "Life situation data" is data that records the user's daily life situation, and includes conversation history and sensor data.

[0682] "Analysis" refers to analyzing collected data to extract useful information and patterns.

[0683] A "report" is a written or electronic document summarizing the analyzed living situation data and provided to a third party to confirm the user's safety.

[0684] The present invention is a monitoring system for elderly people that supports their daily lives and provides technology for rapid response in emergencies. This system is used by users using devices such as smartphones and AI speakers. Specific embodiments for carrying out the invention are described below.

[0685] 1. Initial Setup

[0686] The user installs and launches a dedicated application on their smartphone or AI speaker. This application displays a guide for the user to input initial setup information. The initial setup information includes:

[0687] Language settings

[0688] name

[0689] Personality settings (character, personality, etc.)

[0690] Emergency Contact Information

[0691] After entering this initial setup information, the device sends it to a server via the Internet and stores it in data storage. Specific database management systems used on the server include MySQL and PostgreSQL.

[0692] 2. Daily communication

[0693] The device actively calls the user periodically and engages in daily communication. For example, the following conversation takes place every morning at 8:00:

[0694] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0695] User: "Good morning. I'm going shopping today."

[0696] Terminal: "That's good. Be careful."

[0697] This communication is done using voice recognition software (e.g., Google Speech-to-Text) and generative AI models, which generate more friendly conversational content based on the user's personality settings.

[0698] 3. Monitoring and Anomaly Detection

[0699] The device monitors the user's surroundings using built-in sensors (e.g., sound sensors, motion sensors) and microphones. This allows it to record and analyze everyday sounds and movement patterns in real time. When certain abnormal conditions occur (e.g., loud noises or long periods of silence), the device detects the abnormality as follows:

[0700] Terminal: "Suzuki-san, are you okay?"

[0701] Call again after a few minutes, and if there is still no response, assume that something is wrong.

[0702] 4. Emergency call

[0703] If an abnormality is detected, the device immediately sends this information to the server, which then sends a notification to the registered emergency contacts via SMS or email (e.g., Twilio, SendGrid). The notification content is as follows:

[0704] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0705] 5. Solving elderly housing problems

[0706] The device records the user's daily life situation and sends it to the server. The server analyzes the received data and generates periodic reports. These reports are generated using data analysis software (e.g., Python's Pandas library). The report contents are provided as follows:

[0707] Server: "Mr. Suzuki has been living his daily life stably for the past week, with no notable abnormalities."

[0708] This allows family members and other people involved to confirm the safety of the user.

[0709] Prompt Sentence Examples

[0710] Here are some example prompts to use when inputting a scenario into a generative AI model:

[0711] "We want to create a system that helps elderly people live their daily lives with peace of mind. Please generate scenarios that involve daily conversations, anomaly detection, and emergency calls. Please include specific examples of morning conversations and anomaly detection."

[0712] The above is a detailed description of the mode for carrying out the invention, and by using this system, elderly people can live their daily lives safely and with peace of mind.

[0713] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0714] System program processing flow

[0715] Initial Setup

[0716] Step 1:

[0717] Users install and launch a dedicated application on their smartphone or AI speaker.

[0718] Input: Install and launch the application

[0719] Output: The screen where the application starts

[0720] Step 2:

[0721] The terminal displays an initial setting screen for the application and prompts the user to input initial setting information.

[0722] What happens: A user enters language preferences, name, personality preferences, and emergency contact information into an application.

[0723] Input: User-entered initial configuration information

[0724] Output: The entered information is displayed on the terminal.

[0725] Step 3:

[0726] The terminal transmits the input initial setting information to a server via the Internet.

[0727] Specific operation: Sends information using the device's network communication function

[0728] Input: Initial setting information entered into the terminal

[0729] Output: Initial configuration information sent to the server

[0730] Step 4:

[0731] The server stores the received initial setting information in a data storage.

[0732] Specific operation: Use a database management system (MySQL or PostgreSQL)

[0733] Input: Received initial setting information

[0734] Output: Initial settings information stored in the database

[0735] Daily communication

[0736] Step 1:

[0737] The device uses an internal clock and timer function to call the user at the set time.

[0738] Specific behavior: Send a message to the user every morning at 8am

[0739] Input: Set time information

[0740] Output: A message to the user

[0741] Step 2:

[0742] The device uses voice recognition software (Google Speech-to-Text) to analyze the user's responses and generate appropriate replies.

[0743] What it does: Converts user speech to text and generates a response using a generative AI model.

[0744] Input: User's voice response

[0745] Output: Terminal-generated reply message

[0746] Specific working example:

[0747] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0748] User: "Good morning. I'm going shopping today."

[0749] Terminal: "That's good. Be careful."

[0750] Monitoring and Anomaly Detection

[0751] Step 1:

[0752] The device uses built-in sensors (sound sensor, motion sensor) and microphone to monitor the user's surroundings.

[0753] Specific operation: Sensors monitor the living environment in real time

[0754] Input: Sound and movement data from the surrounding area

[0755] Output: Monitoring data

[0756] Step 2:

[0757] The device analyzes the collected sensor data and detects abnormalities when certain abnormal conditions occur (loud noises, long periods of silence).

[0758] Specific operation: Real-time data analysis

[0759] Input: Sensor data

[0760] Output: Anomaly detection results

[0761] Specific working example:

[0762] Terminal: "Suzuki-san, are you okay?"

[0763] If there is no response, try again in a few minutes.

[0764] If there is still no reaction, it is considered abnormal.

[0765] emergency call

[0766] Step 1:

[0767] If an abnormality is detected, the terminal immediately transmits the information to the server.

[0768] Specific operation: Sends anomaly information via internet connection

[0769] Input: Anomaly detection information

[0770] Output: Anomaly information sent to the server

[0771] Step 2:

[0772] The server sends a notification to the registered emergency contact based on the received abnormality information.

[0773] Specific operation: Uses SMS and email sending systems (Twilio, SendGrid)

[0774] Input: Anomaly information

[0775] Output: Notification message to emergency contacts

[0776] Specific working example:

[0777] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[0778] Solving elderly housing problems

[0779] Step 1:

[0780] The terminal records the user's daily life situation and periodically transmits it to the server.

[0781] Specific operation: Uses the device's local storage and internet connection

[0782] Input: Daily life situation data

[0783] Output: Status data sent to the server

[0784] Step 2:

[0785] The server analyzes the received data and generates periodic reports.

[0786] Specific operation: Data analysis software (Python's Pandas library)

[0787] Input: Received living situation data

[0788] Output: Generated report

[0789] Specific working example:

[0790] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and provide it to his family and related parties.

[0791] The above is the specific flow of the system program processing and details of each step.

[0792] (Application example 1)

[0793] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] There is a need to ensure rapid response in emergencies while monitoring the safety of elderly people's daily lives. However, many current systems have low accuracy in detecting abnormalities, and visual and auditory monitoring in particular is insufficient. Furthermore, they are limited to simple warning sounds and notifications and lack functionality for daily communication with users, which can lead to a decline in the quality of daily life. Another issue is the lack of detailed recording and reporting of abnormal situations and reliable notification to emergency contacts.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0796] In this invention, the server includes means for receiving and saving the user's initial setting information, means for periodically actively calling and conversing with the user, means for monitoring surrounding sounds and movements and detecting abnormalities, means for notifying an emergency contact when an abnormality is detected, means for monitoring audio, video, and motion data using sensors and cameras, and means for checking multiple times if there is no response and determining that there is an abnormality if there is still no response. This makes it possible to comprehensively monitor the lives of the elderly and respond quickly and effectively in emergencies.

[0797] The "means for receiving and saving user initial setting information" is a function for collecting information that a user inputs when using the system for the first time and storing it in a database.

[0798] "Means for actively calling out to and conversing with the user on a regular basis" refers to a function in which the system speaks to and communicates with the user based on a pre-set schedule.

[0799] "Means for monitoring surrounding sounds and movements and detecting abnormalities" refers to a system that uses sound sensors and motion sensors to constantly monitor environmental information and detect abnormalities.

[0800] "Means of notifying emergency contacts when an abnormality is detected" is a function that sends warnings and notifications to pre-registered emergency contacts when the system recognizes an abnormality.

[0801] "Means for monitoring audio, video, and motion data using sensors and cameras" refers to a function that utilizes sensors and cameras to acquire and monitor data such as audio, video, and motion.

[0802] "Means for making multiple confirmations if there is no response, and determining that an abnormality exists if there is still no response" is a function that makes multiple confirmation calls if the user does not respond within a certain period of time, and determines that an abnormality exists if no response is still received.

[0803] The present invention is a system for comprehensively monitoring the lives of elderly people and responding quickly and effectively in emergencies. The system includes a means for receiving and storing user initial setup information, including name, emergency contact information, and personality settings. This allows the device to respond appropriately based on the user's unique information.

[0804] The system has a means to actively call out to the user periodically and engage in conversation. For example, every morning at 8:00, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?" This conversation is generated based on the user's personality settings, making it more friendly.

[0805] The device is also equipped with a means of monitoring surrounding sounds and movements to detect abnormalities. Audio, video, and motion data are monitored using sensors and cameras. If an abnormality is detected, the device immediately reports this information to the server. The server then sends a notification to the registered emergency contacts. In addition, if there is no response, the device will check multiple times and, if there is still no response, will determine that an abnormality has occurred. This ensures a prompt response when an elderly person is in a dangerous situation.

[0806] The user's living situation data is constantly recorded and generated as a report. This report is automatically generated using a generative AI model and provided to family members or medical institutions as needed. This allows those involved to constantly keep track of the user's health and living situation.

[0807] The following are specific examples of the present invention.

[0808] Example 1:

[0809] 1. The user puts on the smart glasses and launches the application.

[0810] 2. The system speaks to the user, "Good morning, Suzuki-san. What are your plans for today?"

[0811] 3. The user replies, "Good morning, I'm going shopping today."

[0812] 4. The system responds, "That's good. Be careful."

[0813] Example 2:

[0814] 1. The system monitors surrounding sounds and activity.

[0815] 2. Suddenly, a loud noise occurs and the system detects an abnormality.

[0816] 3. The system calls out, "Mr. Suzuki, are you okay?" but there is no response.

[0817] 4. He then calls out a second time, "Mr. Suzuki, please respond," but there is still no response.

[0818] 5. The system reports the abnormality to the server, and the server sends a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0819] An example of a prompt sentence to input to the generative AI model is as follows:

[0820] "Apply the invention of the elderly monitoring system to create an application scenario for a security service using smart glasses."

[0821] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0822] Step 1:

[0823] The device receives and stores the user's initial setting information. The initial setting information (name, emergency contact, personality settings) is received from the user, sent to the server, and stored in a database. This allows the device to respond appropriately based on the user's unique information.

[0824] Step 2:

[0825] It periodically actively calls out to the user and starts a conversation. The user's name and personality settings are obtained from the server as input, and the generative AI model generates the conversation content as output, which the device then relays to the user via voice. For example, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0826] Step 3:

[0827] Sensors and cameras are used to monitor audio, video, and motion data. The device monitors surrounding sounds and motion in real time, taking environmental data as input. It analyzes the data to detect abnormalities, and if no abnormalities are found, records the data as normal, and if an abnormality is detected, moves on to the next step.

[0828] Step 4:

[0829] If an abnormality is detected, a response is checked. The terminal calls out, "Mr. Suzuki, are you OK?", but if there is no response from the user as input, the check is made multiple times (for example, again after 5 minutes). The output confirms that there is no response, and once this step is complete, the process proceeds to the next step.

[0830] Step 5:

[0831] The abnormality is reported to the server and an emergency contact is notified. As input, the device sends "no response" data to the server, and as output, the server sends a notification to the registered emergency contact. The content of the notification is "An abnormality has been detected in Suzuki's situation. Please check immediately."

[0832] Step 6:

[0833] It records the user's living situation data and generates a report. Data from sensors and cameras is sent as input to a server, and a living situation report is automatically generated using a generative AI model. The report generated as output is provided to family members and medical institutions. This allows those involved to constantly keep track of the user's health condition and living situation.

[0834] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0835] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and quickly responding to emergencies. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, thereby providing a more detailed and human-like response. Specific embodiments of the present invention are described in detail below.

[0836] 1. Initial Setup

[0837] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[0838] Language settings

[0839] name

[0840] Personality settings (character, personality)

[0841] Emergency Contact Information

[0842] The device sends this initial setup information to the server, which stores it in a database for later access.

[0843] 2. Daily communication

[0844] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[0845] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0846] User: "Good morning, I'm going shopping today."

[0847] Terminal: "That's good. Be careful."

[0848] 3. Emotional engine response

[0849] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[0850] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[0851] 4. Monitoring and Anomaly Detection

[0852] The device monitors surrounding sounds and movements using sensors and microphones. These sensors analyze and record everyday sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[0853] Terminal: "Suzuki-san, are you okay?"

[0854] If there is no response, try again after a few minutes.

[0855] If there is still no reaction, it is determined to be abnormal.

[0856] 5. Emergency call

[0857] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[0858] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0859] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[0860] 6. Solving the housing problem for elderly people

[0861] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[0862] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0863] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[0864] Specific scenario example

[0865] Example 1: Morning conversation scenario

[0866] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0867] User: "Good morning, I slept well last night."

[0868] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0869] Example 2: Emotion Recognition Scenario

[0870] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[0871] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[0872] Terminal: "That's concerning. Should I call for help right away?"

[0873] Example 3: Anomaly detection scenario

[0874] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0875] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0876] Device: Further no response, so report abnormality to server

[0877] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0878] As described above, the present invention is a multi-functional system that watches over elderly people to ensure that they can live their daily lives safely and with peace of mind, responds quickly in emergencies, and uses an emotion engine to provide flexible responses that correspond to the user's emotions.

[0879] The processing flow will be explained below.

[0880] Step 1:

[0881] The user turns on the device and starts the application, which displays a guide for initial setup.

[0882] Step 2:

[0883] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[0884] Step 3:

[0885] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[0886] Step 4:

[0887] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[0888] Step 5:

[0889] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[0890] Step 6:

[0891] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[0892] Step 7:

[0893] The device analyzes the user's tone of voice and word choice and uses an emotion engine to recognize the user's emotions.

[0894] Step 8:

[0895] The device generates the following responses depending on the user's emotions. For example, if the user speaks in an anxious voice, the device responds, "Mr. Suzuki, are you worried about something today? Please tell me."

[0896] Step 9:

[0897] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[0898] Step 10:

[0899] If an abnormality is detected, such as a loud noise or a long period of silence, the device will call out to the user, saying something like, "Mr. Suzuki, are you OK?"

[0900] Step 11:

[0901] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[0902] Step 12:

[0903] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[0904] Step 13:

[0905] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0906] Step 14:

[0907] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[0908] Step 15:

[0909] The device continuously records daily living situation data and periodically transmits it to the server.

[0910] Step 16:

[0911] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[0912] Step 17:

[0913] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[0914] The above are the specific processing steps of this system, which monitors the user's life and provides flexible responses according to their emotions.

[0915] Example 2

[0916] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0917] Today's elderly people face a variety of problems in their daily lives. These include feelings of loneliness and anxiety about their safety, and they often find themselves unable to receive the support they need promptly. Furthermore, it is difficult for their families and relatives to understand the elderly's living situation, which risks delaying appropriate responses. There is a need to develop a system that can solve these problems and enable the elderly to live their daily lives with peace of mind.

[0918] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving and saving the user's initial setting information, a means for actively calling out to the user periodically and holding a conversation, a means for analyzing the user's tone of voice, facial expression, and words selected to identify emotions, a means for monitoring surrounding sounds and movements and detecting abnormalities, and a means for notifying an emergency contact when an abnormality is detected. This enables the system to appropriately monitor the living conditions of the elderly and automatically take appropriate measures as necessary.

[0919] "Initial setup information" refers to basic information that a user enters when they first start using the system, including language settings, name, personality settings, and emergency contact information.

[0920] "Active calling" is a function that enables the system to communicate with the user on a regular basis by proactively speaking to the user.

[0921] "Means for conducting conversation" refers to a function for exchanging information with a user using voice or text.

[0922] "Voice tone" refers to characteristics such as pitch, strength, and rhythm of a user's voice when they speak, and is used to identify emotions.

[0923] "Facial expression" refers to facial movements and changes in facial muscles, and is an important factor in identifying emotions.

[0924] "Selected words" refer to words and phrases that users use in conversation and are used to identify emotions.

[0925] "Means for identifying emotions" is a function that determines the user's emotional state by analyzing the user's tone of voice, facial expressions, and words they choose.

[0926] "Means for monitoring surrounding sounds and movements" refers to a function that uses sensors and microphones to detect environmental sounds and movements and discover abnormalities.

[0927] "Means for detecting abnormalities" refers to a function that recognizes an abnormal situation when the system meets pre-set conditions (e.g., a loud abnormal noise or a long period of silence).

[0928] "Emergency contacts" are people to whom notifications are sent when an abnormality is detected, including family members and local medical institutions.

[0929] "Life situation data" refers to various information related to the user's daily life, such as behavioral patterns and health status.

[0930] A "report" is a document in the form of a report generated based on data collected by the system, detailing the user's living situation.

[0931] "Personality setting" refers to the user pre-setting their own character and personality within the system, and is used to customize the content of conversations.

[0932] This system monitors the lives of the elderly, enriching their daily lives while also responding quickly to emergencies. Furthermore, by incorporating an emotion engine that recognizes the user's emotions and responds appropriately based on those emotions, the system offers a more detailed and human-like response.

[0933] Basic system configuration

[0934] The system mainly includes the following elements:

[0935] User

[0936] Device (e.g., AI speaker, smartphone)

[0937] server

[0938] Emotion Engine

[0939] Sensors (e.g. microphone, motion sensor)

[0940] 1. Initial Setup

[0941] When a user uses the system for the first time, they install and launch the application on their device (such as an AI speaker or smartphone). When the application launches, the user is prompted to enter initial information, including language settings, name, personality settings, and emergency contact information. The device then sends the entered information to the server.

[0942] The server stores the received initial setup information in a database for future access. Specifically, the "initial setup information" is basic information that a user enters when they start using the system, such as language settings, name, personality settings, and emergency contact information.

[0943] 2. Daily communication

[0944] By using the system's periodic communication function, the device actively engages with the user and engages in daily conversations. For example, the following conversation takes place every morning at 8:00:

[0945] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0946] User: "Good morning, I'm going shopping today."

[0947] Terminal: "That's good. Be careful."

[0948] This allows users to reduce feelings of loneliness in their daily lives and receive psychological support through the system.

[0949] 3. Emotional engine response

[0950] The device analyzes the user's tone of voice, facial expressions, and choice of words, and uses an emotion engine to identify the user's emotions. For example, if the user speaks with anxiety:

[0951] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[0952] A concrete example of this "means for identifying emotions" is to analyze changes in the pitch and tone of a user's voice and the words they choose when they are feeling anxious or worried, and then use the emotion engine to determine that they are feeling "anxiety." This allows the system to respond appropriately to the user's psychological state.

[0953] 4. Monitoring and Anomaly Detection

[0954] The device uses sensors and microphones to monitor surrounding sounds and movements. Specifically, the sensors analyze and record everyday sounds and movements in real time. The system detects specific anomalies and takes immediate action:

[0955] Terminal: "Suzuki-san, are you OK?" (If there is no response, try again after a few minutes.)

[0956] If there is still no response, it will be determined that something is wrong and an emergency call will be made.

[0957] 5. Emergency call

[0958] If an abnormality is detected, the device immediately reports this information to the server, which then sends a notification to registered emergency contacts, such as:

[0959] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[0960] If necessary, preparations are being made to notify local medical institutions and welfare facilities.

[0961] 6. Solving the housing problem for elderly people

[0962] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[0963] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[0964] This report is provided to families, rental owners, facilities, etc. to prove that users are living safely.

[0965] Examples of prompts include:

[0966] Example 1: Morning conversation scenario

[0967] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[0968] User: "Good morning, I slept well last night."

[0969] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[0970] Example 2: Emotion Recognition Scenario

[0971] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[0972] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[0973] Terminal: "That's concerning. Should I call for help right away?"

[0974] Example 3: Anomaly detection scenario

[0975] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[0976] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[0977] Device: Further no response, so report abnormality to server

[0978] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[0979] This system not only enables elderly people to live their daily lives safely and with peace of mind, but also enables appropriate and prompt responses when necessary. Furthermore, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotions, providing psychological support to the user.

[0980] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0981] Step 1: Entering user information and initial settings

[0982] Input: The user enters initial configuration information (e.g., language preference, name, personality preference, emergency contact information) into the application.

[0983] How it works: The user launches the application on their device and enters the required information following the prompts.

[0984] Data processing: The terminal organizes the input information into a data structure and sends it to the server.

[0985] Output: Initial configuration information sent to the server.

[0986] Step 2: Save the initial configuration information

[0987] Input: Initial setting information sent from the terminal.

[0988] What it does: The server stores the received information in a database.

[0989] Data processing: The received initial setting information is organized into appropriate fields for storage in a database.

[0990] Output: Initial configuration information stored in the database.

[0991] Step 3: Regularly check in and keep the conversation going

[0992] Input: User schedule and past conversation history read from the server.

[0993] What it does: The device automatically calls the user at a set time (e.g., 8:00 every morning).

[0994] Data calculation: The server generates optimal conversation prompts from past data and sends them to the device.

[0995] Output: A specific call to action for the user.

[0996] Examples:

[0997] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[0998] User: "Good morning, I'm going shopping today."

[0999] Terminal: "That's good. Be careful."

[1000] Step 4: Identify the emotion

[1001] Input: Data on the user's tone of voice, speech, and facial expressions.

[1002] Operation: The device sends collected voice data to the emotion engine.

[1003] Data Calculation: The emotion engine analyzes tone of voice and selected words to identify emotions.

[1004] Output: Identified emotional state.

[1005] Examples:

[1006] The user says in an anxious voice, "I'm not feeling well."

[1007] The device will identify "anxiety" through its emotion engine and respond, "That sounds worrying. Would you like us to call for support right away?"

[1008] Step 5: Surrounding area monitoring and anomaly detection

[1009] Input: Environmental sound data and movement data from sensors and microphones.

[1010] Operation: The device monitors surrounding sounds and movements in real time.

[1011] Data processing: Analyze collected data to detect abnormal patterns (e.g., loud noises or long periods of silence).

[1012] Output: Alert information if an abnormality is detected.

[1013] Examples:

[1014] Terminal: Detects an abnormal noise and calls out, "Suzuki-san, are you okay?"

[1015] If there is no response, call again after a few minutes, and if there is still no response, it is determined to be an "abnormality."

[1016] Step 6: Call Emergency Services

[1017] Input: Alert information that an anomaly has been detected.

[1018] Action: The server notifies emergency contacts.

[1019] Data processing: Check emergency contact information and convert the anomaly detection content into a message.

[1020] Output: Notification sent to emergency contacts.

[1021] Examples:

[1022] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[1023] Step 7: Recording living situation data and generating reports

[1024] Input: Daily activity data sent from your device.

[1025] Operation: The server continuously records the living situation data.

[1026] Data Computing: Analyzes collected data and generates periodic reports.

[1027] Output: Living situation data compiled as a report.

[1028] Examples:

[1029] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and send it to the relevant parties.

[1030] (Application example 2)

[1031] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1032] In recent years, the number of elderly people living alone has been increasing, creating a need for monitoring systems that allow elderly people to live their daily lives safely and with peace of mind. However, conventional monitoring systems simply detect abnormalities and are unable to understand the emotions and psychological state of the elderly or provide appropriate responses. Therefore, a system that can not only respond to emergencies but also enrich the daily lives of elderly people and provide a sense of security is needed.

[1033] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1034] In this invention, the server includes means for receiving and storing the user's initial setting information, means for periodically actively calling out to the user and engaging in conversation, means for monitoring surrounding sounds and movements to detect abnormalities, means for notifying an emergency contact when an abnormality is detected, and means for understanding the user's emotions using voice recognition and an emotion engine and responding appropriately, allowing the elderly to live their daily lives with peace of mind and enabling quick and appropriate responses in emergencies.

[1035] A "user" is an individual who performs initial setup to use the system.

[1036] "Preferences Information" refers to basic information such as a user's name, language preference, personality preference, and emergency contact information.

[1037] "Actively calling out" refers to the system proactively speaking to the user and initiating a dialogue.

[1038] "Speech recognition" refers to the technology of analyzing a user's voice and understanding its content.

[1039] An "emotion engine" refers to technology that analyzes a user's emotional state based on their tone of voice, facial expressions, and choice of words.

[1040] "Surrounding sounds and movements" refers to sounds and movements of the user's body within the user's living environment.

[1041] "Means for detecting abnormalities" refers to a system that uses sensors and microphones to analyze surrounding sounds and movements in real time and identify abnormal situations.

[1042] "Emergency contact" is contact information registered by the user to receive notifications in the event of an emergency.

[1043] "Means for emergency notification" refers to a system that sends a notification to emergency contacts or appropriate authorities when an abnormality is detected.

[1044] "Lifestyle data" refers to data related to the user's daily activities and health condition.

[1045] "Means for generating reports" refers to a system that periodically creates reports summarizing the current status of the user based on collected living situation data.

[1046] "Third Party" refers to the user's family, medical institution, or other party providing necessary assistance.

[1047] "Personality settings" refers to a feature that allows users to set their own personality and preferences for the system.

[1048] "Means for responding appropriately" refers to technology that enables the system to generate appropriate responses depending on the user's emotional state and the content of the conversation.

[1049] The present invention is a system that assumes that the user is an elderly person, monitors their daily life, and responds quickly in the event of an emergency. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, providing a more detailed and human-like response.

[1050] 1. Initial Setup

[1051] When a user uses the system for the first time, they install and launch the application on their smartphone or other device. The application displays a guide for initial setup and prompts them to enter the following information:

[1052] Language settings

[1053] name

[1054] Personality settings (character and preferences)

[1055] Emergency Contact Information

[1056] The terminal transmits this initial setting information to the server, and the server stores the received information in a database.

[1057] 2. Daily communication

[1058] Based on the settings, the system will actively call out to the user and have a conversation with them periodically. For example, every morning at 8:00, the following conversation will take place:

[1059] Terminal: "Good morning, <username>. What are your plans for today?"

[1060] User: "Good morning, I'm going shopping today."

[1061] Terminal: "That's good. Be careful on your way."

[1062] 3. Emotional Engine Response

[1063] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[1064] Terminal: "Hey <username>, is there something bothering you today? Let me know what you think."

[1065] 4. Monitoring and Anomaly Detection

[1066] The device uses sensors and microphones to monitor surrounding sounds and movements, analyzing them in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies by following these steps:

[1067] Terminal: "<username>, are you okay?"

[1068] If there is no response, try again after a few minutes.

[1069] If there is still no reaction, it is determined to be abnormal.

[1070] 5. Emergency call

[1071] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1072] "An abnormality has been detected in <user name>'s status. Please check immediately."

[1073] If necessary, local medical institutions and welfare facilities will also be notified.

[1074] 6. Solving the housing problem for elderly people

[1075] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[1076] "<User name> has been living a stable daily life for the past week, with no notable abnormalities."

[1077] This report can be provided to family and relatives as needed to prove that the user is living safely.

[1078] Hardware and software used

[1079] EmotionRecognition: Emotion recognition engine

[1080] VoiceInterface: Voice Interface

[1081] SensorManager: Environmental sensor management

[1082] notify_emergency_contacts: Emergency contacts

[1083] generate_report: Report generation function

[1084] Specific examples

[1085] For example, consider the following scenario:

[1086] Terminal: "Good morning, <username>. Did you sleep well last night?"

[1087] User: "Good morning, I slept well last night."

[1088] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1089] Prompt Sentence Examples

[1090] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[1091] information:

[1092] Watching over the daily lives of the elderly

[1093] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[1094] Notify emergency contacts and medical institutions when an abnormality is detected

[1095] Generate regular lifestyle reports and provide them to families and owners.

[1096] The above is a specific embodiment for carrying out the present invention.

[1097] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1098] Step 1:

[1099] Receiving and saving initial configuration information

[1100] A user installs the application on a device such as a smartphone and enters initial information (language settings, name, personality settings, emergency contact information). The device sends this information to the server, which stores it in a database.

[1101] Input: Language settings, name, personality settings, emergency contact information

[1102] Output: Initial configuration information stored in the server database

[1103] Step 2:

[1104] Beginning of daily communication

[1105] Based on the settings, the device will actively call out to the user periodically. For example, every morning at 8:00, it might say, "Good morning, <user name>. What are your plans for today?" The device will then analyze the user's response with a speech recognition engine and generate an appropriate response.

[1106] Input: User's spoken response

[1107] Output: Speech recognition results, voice response from the device

[1108] Step 3:

[1109] Emotion engine response

[1110] The device uses an emotion engine to analyze the user's tone of voice and choice of words to identify their emotional state. For example, if the user speaks in an anxious voice, the device can respond based on that by saying, "Hey <username>, are you worried about something today? Let me know."

[1111] Input: User's voice data

[1112] Output: Emotional state identification result, appropriate response voice

[1113] Step 4:

[1114] Surrounding sound and movement monitoring and anomaly detection

[1115] The device uses sensors and microphones to monitor surrounding sounds and movements in real time. If an abnormal sound or movement is detected (loud noises or long periods of silence), the device will:

[1116] First, say, "<user name>, are you okay?"

[1117] If there is no response, try again after a few minutes.

[1118] If there is still no reaction, it is determined to be abnormal.

[1119] Input: ambient sound and movement data

[1120] Output: Anomaly detection result, voice call from device

[1121] Step 5:

[1122] emergency call

[1123] If an abnormality is detected, the device immediately reports the information to the server, which then sends a notification to registered emergency contacts and, if necessary, to local medical institutions and welfare facilities.

[1124] Input: Anomaly detection results

[1125] Output: Notification to emergency contacts

[1126] Step 6:

[1127] Generate scheduled reports

[1128] The server continuously records the user's living situation data and periodically generates reports, which are provided to family members and other interested parties as needed.

[1129] Input: User's life situation data

[1130] Output: Living situation report

[1131] Specific examples

[1132] For example, consider the following scenario:

[1133] In step 2, the device says, "Good morning, <user name>. Did you sleep well last night?", and the user responds, "Good morning, I slept well last night." In step 3, the device responds, "That's good. The weather is nice today, so it would be nice to go for a walk in the daytime."

[1134] Prompt Sentence Examples

[1135] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[1136] information:

[1137] Watching over the daily lives of the elderly

[1138] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[1139] Notify emergency contacts and medical institutions when an abnormality is detected

[1140] Generate regular lifestyle reports and provide them to families and owners.

[1141] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1142] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1143] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1144] [Third embodiment]

[1145] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1146] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1147] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1148] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1149] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1150] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1151] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1152] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1153] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1155] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1156] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1157] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and responding quickly to emergencies. Specific embodiments of the present invention will be described in detail below.

[1158] 1. Initial Setup

[1159] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[1160] Language settings

[1161] name

[1162] Personality settings (e.g., character, personality)

[1163] Emergency Contact Information

[1164] The device sends this initial setup information to the server, which stores it in a database for later access.

[1165] 2. Daily communication

[1166] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[1167] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1168] User: "Good morning, I'm going shopping today."

[1169] Terminal: "That's good. Be careful."

[1170] In this way, the terminal generates conversation content based on the user's personality settings, providing a more friendly conversation.

[1171] 3. Monitoring and Anomaly Detection

[1172] The device has built-in sensors and microphones to monitor surrounding sounds and movements. These sensors analyze and record daily sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[1173] Terminal: "Suzuki-san, are you okay?"

[1174] If there is no response, try again after a few minutes.

[1175] If there is still no reaction, it is determined to be abnormal.

[1176] 4. Emergency call

[1177] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1178] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1179] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[1180] 5. Solving elderly housing problems

[1181] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[1182] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[1183] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[1184] Specific scenario example

[1185] Example 1: Morning conversation scenario

[1186] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[1187] User: "Good morning, I slept well last night."

[1188] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1189] Example 2: Anomaly detection scenario

[1190] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[1191] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[1192] Device: Further no response, so report abnormality to server

[1193] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1194] As described above, the present invention provides a multi-functional system that watches over elderly people to enable them to live their daily lives safely and with peace of mind, and responds quickly in emergencies.

[1195] The processing flow will be explained below.

[1196] Step 1:

[1197] The user turns on the device and starts the application, which displays a guide for initial setup.

[1198] Step 2:

[1199] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[1200] Step 3:

[1201] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[1202] Step 4:

[1203] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[1204] Step 5:

[1205] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[1206] Step 6:

[1207] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[1208] Step 7:

[1209] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[1210] Step 8:

[1211] If an abnormality is detected, such as a loud noise or a long period of silence, the device will ask the user, "Are you OK?"

[1212] Step 9:

[1213] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[1214] Step 10:

[1215] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[1216] Step 11:

[1217] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1218] Step 12:

[1219] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[1220] Step 13:

[1221] The device continuously records daily living situation data and periodically transmits it to the server.

[1222] Step 14:

[1223] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[1224] Step 15:

[1225] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[1226] The above are the specific processing steps of this system.

[1227] Example 1

[1228] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] Current elderly care systems often lack sufficient functionality to properly monitor the surrounding environment and respond quickly in emergencies. Furthermore, due to a lack of daily communication with users to build trust, users may find it difficult to use the system and feel uneasy. Another issue is the lack of a means to report users' living conditions to a third party and prove the safety of the elderly.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1231] In this invention, the server includes a means for receiving the user's initial setting information and storing it in data storage, a means for actively calling and conversing with the user based on timing, and a means for monitoring surrounding sounds and movements with sensors to detect abnormalities. This allows for appropriate monitoring of the user's living environment and enables rapid response in emergencies. Furthermore, daily communication through active calling based on timing builds a relationship of trust with the user, allowing them to use the system with peace of mind. The system also provides a means for verifying the safety of the elderly by recording the user's living situation data, analyzing it, generating reports, and providing them to third parties.

[1232] "Users" refer to people who use the system, especially elderly people.

[1233] "Initial setting information" refers to information that a user enters when using the system for the first time, and includes language settings, name, personality settings, emergency contact information, etc.

[1234] "Data storage" refers to a storage device for saving a user's initial setting information and living situation data, and refers to a hard disk built into a server or cloud storage.

[1235] "Based on timing" means acting according to pre-set times or conditions.

[1236] "Actively calling out" refers to the system proactively asking messages or questions to the user.

[1237] "Conversing" refers to engaging in voice or text-based communication between a user and a system.

[1238] "Sensor" refers to a device for detecting surrounding sounds and movements, and includes sound sensors and motion sensors.

[1239] "Abnormal" refers to conditions or situations that the system determines to be deviations from normal life patterns, such as loud, unusual noises or long periods of silence.

[1240] "Emergency Contact" refers to a person or organization registered to receive notifications when an abnormality is detected in the user's behavior.

[1241] "Life situation data" is data that records the user's daily life situation, and includes conversation history and sensor data.

[1242] "Analysis" refers to analyzing collected data to extract useful information and patterns.

[1243] A "report" is a written or electronic document summarizing the analyzed living situation data and provided to a third party to confirm the user's safety.

[1244] The present invention is a monitoring system for elderly people that supports their daily lives and provides technology for rapid response in emergencies. This system is used by users using devices such as smartphones and AI speakers. Specific embodiments for carrying out the invention are described below.

[1245] 1. Initial Setup

[1246] The user installs and launches a dedicated application on their smartphone or AI speaker. This application displays a guide for the user to input initial setup information. The initial setup information includes:

[1247] Language settings

[1248] name

[1249] Personality settings (character, personality, etc.)

[1250] Emergency Contact Information

[1251] After entering this initial setup information, the device sends it to a server via the Internet and stores it in data storage. Specific database management systems used on the server include MySQL and PostgreSQL.

[1252] 2. Daily communication

[1253] The device actively calls the user periodically and engages in daily communication. For example, the following conversation takes place every morning at 8:00:

[1254] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1255] User: "Good morning. I'm going shopping today."

[1256] Terminal: "That's good. Be careful."

[1257] This communication is done using voice recognition software (e.g., Google Speech-to-Text) and generative AI models, which generate more friendly conversational content based on the user's personality settings.

[1258] 3. Monitoring and Anomaly Detection

[1259] The device monitors the user's surroundings using built-in sensors (e.g., sound sensors, motion sensors) and microphones. This allows it to record and analyze everyday sounds and movement patterns in real time. When certain abnormal conditions occur (e.g., loud noises or long periods of silence), the device detects the abnormality as follows:

[1260] Terminal: "Suzuki-san, are you okay?"

[1261] Call again after a few minutes, and if there is still no response, assume that something is wrong.

[1262] 4. Emergency call

[1263] If an abnormality is detected, the device immediately sends this information to the server, which then sends a notification to the registered emergency contacts via SMS or email (e.g., Twilio, SendGrid). The notification content is as follows:

[1264] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1265] 5. Solving elderly housing problems

[1266] The device records the user's daily life situation and sends it to the server. The server analyzes the received data and generates periodic reports. These reports are generated using data analysis software (e.g., Python's Pandas library). The report contents are provided as follows:

[1267] Server: "Mr. Suzuki has been living his daily life stably for the past week, with no notable abnormalities."

[1268] This allows family members and other people involved to confirm the safety of the user.

[1269] Prompt Sentence Examples

[1270] Here are some example prompts to use when inputting a scenario into a generative AI model:

[1271] "We want to create a system that helps elderly people live their daily lives with peace of mind. Please generate scenarios that involve daily conversations, anomaly detection, and emergency calls. Please include specific examples of morning conversations and anomaly detection."

[1272] The above is a detailed description of the mode for carrying out the invention, and by using this system, elderly people can live their daily lives safely and with peace of mind.

[1273] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1274] System program processing flow

[1275] Initial Setup

[1276] Step 1:

[1277] Users install and launch a dedicated application on their smartphone or AI speaker.

[1278] Input: Install and launch the application

[1279] Output: The screen where the application starts

[1280] Step 2:

[1281] The terminal displays an initial setting screen for the application and prompts the user to input initial setting information.

[1282] What happens: A user enters language preferences, name, personality preferences, and emergency contact information into an application.

[1283] Input: User-entered initial configuration information

[1284] Output: The entered information is displayed on the terminal.

[1285] Step 3:

[1286] The terminal transmits the input initial setting information to a server via the Internet.

[1287] Specific operation: Sends information using the device's network communication function

[1288] Input: Initial setting information entered into the terminal

[1289] Output: Initial configuration information sent to the server

[1290] Step 4:

[1291] The server stores the received initial setting information in a data storage.

[1292] Specific operation: Use a database management system (MySQL or PostgreSQL)

[1293] Input: Received initial setting information

[1294] Output: Initial settings information stored in the database

[1295] Daily communication

[1296] Step 1:

[1297] The device uses an internal clock and timer function to call the user at the set time.

[1298] Specific behavior: Send a message to the user every morning at 8am

[1299] Input: Set time information

[1300] Output: A message to the user

[1301] Step 2:

[1302] The device uses voice recognition software (Google Speech-to-Text) to analyze the user's responses and generate appropriate replies.

[1303] What it does: Converts user speech to text and generates a response using a generative AI model.

[1304] Input: User's voice response

[1305] Output: Terminal-generated reply message

[1306] Specific working example:

[1307] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1308] User: "Good morning. I'm going shopping today."

[1309] Terminal: "That's good. Be careful."

[1310] Monitoring and Anomaly Detection

[1311] Step 1:

[1312] The device uses built-in sensors (sound sensor, motion sensor) and microphone to monitor the user's surroundings.

[1313] Specific operation: Sensors monitor the living environment in real time

[1314] Input: Sound and movement data from the surrounding area

[1315] Output: Monitoring data

[1316] Step 2:

[1317] The device analyzes the collected sensor data and detects abnormalities when certain abnormal conditions occur (loud noises, long periods of silence).

[1318] Specific operation: Real-time data analysis

[1319] Input: Sensor data

[1320] Output: Anomaly detection results

[1321] Specific working example:

[1322] Terminal: "Suzuki-san, are you okay?"

[1323] If there is no response, try again in a few minutes.

[1324] If there is still no reaction, it is considered abnormal.

[1325] emergency call

[1326] Step 1:

[1327] If an abnormality is detected, the terminal immediately transmits the information to the server.

[1328] Specific operation: Sends anomaly information via internet connection

[1329] Input: Anomaly detection information

[1330] Output: Anomaly information sent to the server

[1331] Step 2:

[1332] The server sends a notification to the registered emergency contact based on the received abnormality information.

[1333] Specific operation: Uses SMS and email sending systems (Twilio, SendGrid)

[1334] Input: Anomaly information

[1335] Output: Notification message to emergency contacts

[1336] Specific working example:

[1337] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[1338] Solving elderly housing problems

[1339] Step 1:

[1340] The terminal records the user's daily life situation and periodically transmits it to the server.

[1341] Specific operation: Uses the device's local storage and internet connection

[1342] Input: Daily life situation data

[1343] Output: Status data sent to the server

[1344] Step 2:

[1345] The server analyzes the received data and generates periodic reports.

[1346] Specific operation: Data analysis software (Python's Pandas library)

[1347] Input: Received living situation data

[1348] Output: Generated report

[1349] Specific working example:

[1350] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and provide it to his family and related parties.

[1351] The above is the specific flow of the system program processing and details of each step.

[1352] (Application example 1)

[1353] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1354] There is a need to ensure rapid response in emergencies while monitoring the safety of elderly people's daily lives. However, many current systems have low accuracy in detecting abnormalities, and visual and auditory monitoring in particular is insufficient. Furthermore, they are limited to simple warning sounds and notifications and lack functionality for daily communication with users, which can lead to a decline in the quality of daily life. Another issue is the lack of detailed recording and reporting of abnormal situations and reliable notification to emergency contacts.

[1355] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1356] In this invention, the server includes means for receiving and saving the user's initial setting information, means for periodically actively calling and conversing with the user, means for monitoring surrounding sounds and movements and detecting abnormalities, means for notifying an emergency contact when an abnormality is detected, means for monitoring audio, video, and motion data using sensors and cameras, and means for checking multiple times if there is no response and determining that there is an abnormality if there is still no response. This makes it possible to comprehensively monitor the lives of the elderly and respond quickly and effectively in emergencies.

[1357] The "means for receiving and saving user initial setting information" is a function for collecting information that a user inputs when using the system for the first time and storing it in a database.

[1358] "Means for actively calling out to and conversing with the user on a regular basis" refers to a function in which the system speaks to and communicates with the user based on a pre-set schedule.

[1359] "Means for monitoring surrounding sounds and movements and detecting abnormalities" refers to a system that uses sound sensors and motion sensors to constantly monitor environmental information and detect abnormalities.

[1360] "Means of notifying emergency contacts when an abnormality is detected" is a function that sends warnings and notifications to pre-registered emergency contacts when the system recognizes an abnormality.

[1361] "Means for monitoring audio, video, and motion data using sensors and cameras" refers to a function that utilizes sensors and cameras to acquire and monitor data such as audio, video, and motion.

[1362] "Means for making multiple confirmations if there is no response, and determining that an abnormality exists if there is still no response" is a function that makes multiple confirmation calls if the user does not respond within a certain period of time, and determines that an abnormality exists if no response is still received.

[1363] The present invention is a system for comprehensively monitoring the lives of elderly people and responding quickly and effectively in emergencies. The system includes a means for receiving and storing user initial setup information, including name, emergency contact information, and personality settings. This allows the device to respond appropriately based on the user's unique information.

[1364] The system has a means to actively call out to the user periodically and engage in conversation. For example, every morning at 8:00, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?" This conversation is generated based on the user's personality settings, making it more friendly.

[1365] The device is also equipped with a means of monitoring surrounding sounds and movements to detect abnormalities. Audio, video, and motion data are monitored using sensors and cameras. If an abnormality is detected, the device immediately reports this information to the server. The server then sends a notification to the registered emergency contacts. In addition, if there is no response, the device will check multiple times and, if there is still no response, will determine that an abnormality has occurred. This ensures a prompt response when an elderly person is in a dangerous situation.

[1366] The user's living situation data is constantly recorded and generated as a report. This report is automatically generated using a generative AI model and provided to family members or medical institutions as needed. This allows those involved to constantly keep track of the user's health and living situation.

[1367] The following are specific examples of the present invention.

[1368] Example 1:

[1369] 1. The user puts on the smart glasses and launches the application.

[1370] 2. The system speaks to the user, "Good morning, Suzuki-san. What are your plans for today?"

[1371] 3. The user replies, "Good morning, I'm going shopping today."

[1372] 4. The system responds, "That's good. Be careful."

[1373] Example 2:

[1374] 1. The system monitors surrounding sounds and activity.

[1375] 2. Suddenly, a loud noise occurs and the system detects an abnormality.

[1376] 3. The system calls out, "Mr. Suzuki, are you okay?" but there is no response.

[1377] 4. He then calls out a second time, "Mr. Suzuki, please respond," but there is still no response.

[1378] 5. The system reports the abnormality to the server, and the server sends a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1379] An example of a prompt sentence to input to the generative AI model is as follows:

[1380] "Apply the invention of the elderly monitoring system to create an application scenario for a security service using smart glasses."

[1381] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1382] Step 1:

[1383] The device receives and stores the user's initial setting information. The initial setting information (name, emergency contact, personality settings) is received from the user, sent to the server, and stored in a database. This allows the device to respond appropriately based on the user's unique information.

[1384] Step 2:

[1385] It periodically actively calls out to the user and starts a conversation. The user's name and personality settings are obtained from the server as input, and the generative AI model generates the conversation content as output, which the device then relays to the user via voice. For example, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?"

[1386] Step 3:

[1387] Sensors and cameras are used to monitor audio, video, and motion data. The device monitors surrounding sounds and motion in real time, taking environmental data as input. It analyzes the data to detect abnormalities, and if no abnormalities are found, records the data as normal, and if an abnormality is detected, moves on to the next step.

[1388] Step 4:

[1389] If an abnormality is detected, a response is checked. The terminal calls out, "Mr. Suzuki, are you OK?", but if there is no response from the user as input, the check is made multiple times (for example, again after 5 minutes). The output confirms that there is no response, and once this step is complete, the process proceeds to the next step.

[1390] Step 5:

[1391] The abnormality is reported to the server and an emergency contact is notified. As input, the device sends "no response" data to the server, and as output, the server sends a notification to the registered emergency contact. The content of the notification is "An abnormality has been detected in Suzuki's situation. Please check immediately."

[1392] Step 6:

[1393] It records the user's living situation data and generates a report. Data from sensors and cameras is sent as input to a server, and a living situation report is automatically generated using a generative AI model. The report generated as output is provided to family members and medical institutions. This allows those involved to constantly keep track of the user's health condition and living situation.

[1394] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1395] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and quickly responding to emergencies. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, thereby providing a more detailed and human-like response. Specific embodiments of the present invention are described in detail below.

[1396] 1. Initial Setup

[1397] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[1398] Language settings

[1399] name

[1400] Personality settings (character, personality)

[1401] Emergency Contact Information

[1402] The device sends this initial setup information to the server, which stores it in a database for later access.

[1403] 2. Daily communication

[1404] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[1405] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1406] User: "Good morning, I'm going shopping today."

[1407] Terminal: "That's good. Be careful."

[1408] 3. Emotional engine response

[1409] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[1410] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[1411] 4. Monitoring and Anomaly Detection

[1412] The device monitors surrounding sounds and movements using sensors and microphones. These sensors analyze and record everyday sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[1413] Terminal: "Suzuki-san, are you okay?"

[1414] If there is no response, try again after a few minutes.

[1415] If there is still no reaction, it is determined to be abnormal.

[1416] 5. Emergency call

[1417] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1418] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1419] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[1420] 6. Solving the housing problem for elderly people

[1421] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[1422] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[1423] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[1424] Specific scenario example

[1425] Example 1: Morning conversation scenario

[1426] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[1427] User: "Good morning, I slept well last night."

[1428] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1429] Example 2: Emotion Recognition Scenario

[1430] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[1431] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[1432] Terminal: "That's concerning. Should I call for help right away?"

[1433] Example 3: Anomaly detection scenario

[1434] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[1435] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[1436] Device: Further no response, so report abnormality to server

[1437] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1438] As described above, the present invention is a multi-functional system that watches over elderly people to ensure that they can live their daily lives safely and with peace of mind, responds quickly in emergencies, and uses an emotion engine to provide flexible responses that correspond to the user's emotions.

[1439] The processing flow will be explained below.

[1440] Step 1:

[1441] The user turns on the device and starts the application, which displays a guide for initial setup.

[1442] Step 2:

[1443] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[1444] Step 3:

[1445] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[1446] Step 4:

[1447] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[1448] Step 5:

[1449] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[1450] Step 6:

[1451] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[1452] Step 7:

[1453] The device analyzes the user's tone of voice and word choice and uses an emotion engine to recognize the user's emotions.

[1454] Step 8:

[1455] The device generates the following responses depending on the user's emotions. For example, if the user speaks in an anxious voice, the device responds, "Mr. Suzuki, are you worried about something today? Please tell me."

[1456] Step 9:

[1457] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[1458] Step 10:

[1459] If an abnormality is detected, such as a loud noise or a long period of silence, the device will call out to the user, saying something like, "Mr. Suzuki, are you OK?"

[1460] Step 11:

[1461] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[1462] Step 12:

[1463] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[1464] Step 13:

[1465] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1466] Step 14:

[1467] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[1468] Step 15:

[1469] The device continuously records daily living situation data and periodically transmits it to the server.

[1470] Step 16:

[1471] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[1472] Step 17:

[1473] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[1474] The above are the specific processing steps of this system, which monitors the user's life and provides flexible responses according to their emotions.

[1475] Example 2

[1476] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1477] Today's elderly people face a variety of problems in their daily lives. These include feelings of loneliness and anxiety about their safety, and they often find themselves unable to receive the support they need promptly. Furthermore, it is difficult for their families and relatives to understand the elderly's living situation, which risks delaying appropriate responses. There is a need to develop a system that can solve these problems and enable the elderly to live their daily lives with peace of mind.

[1478] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving and saving the user's initial setting information, a means for actively calling out to the user periodically and holding a conversation, a means for analyzing the user's tone of voice, facial expression, and words selected to identify emotions, a means for monitoring surrounding sounds and movements and detecting abnormalities, and a means for notifying an emergency contact when an abnormality is detected. This enables the system to appropriately monitor the living conditions of the elderly and automatically take appropriate measures as necessary.

[1479] "Initial setup information" refers to basic information that a user enters when they first start using the system, including language settings, name, personality settings, and emergency contact information.

[1480] "Active calling" is a function that enables the system to communicate with the user on a regular basis by proactively speaking to the user.

[1481] "Means for conducting conversation" refers to a function for exchanging information with a user using voice or text.

[1482] "Voice tone" refers to characteristics such as pitch, strength, and rhythm of a user's voice when they speak, and is used to identify emotions.

[1483] "Facial expression" refers to facial movements and changes in facial muscles, and is an important factor in identifying emotions.

[1484] "Selected words" refer to words and phrases that users use in conversation and are used to identify emotions.

[1485] "Means for identifying emotions" is a function that determines the user's emotional state by analyzing the user's tone of voice, facial expressions, and words they choose.

[1486] "Means for monitoring surrounding sounds and movements" refers to a function that uses sensors and microphones to detect environmental sounds and movements and discover abnormalities.

[1487] "Means for detecting abnormalities" refers to a function that recognizes an abnormal situation when the system meets pre-set conditions (e.g., a loud abnormal noise or a long period of silence).

[1488] "Emergency contacts" are people to whom notifications are sent when an abnormality is detected, including family members and local medical institutions.

[1489] "Life situation data" refers to various information related to the user's daily life, such as behavioral patterns and health status.

[1490] A "report" is a document in the form of a report generated based on data collected by the system, detailing the user's living situation.

[1491] "Personality setting" refers to the user pre-setting their own character and personality within the system, and is used to customize the content of conversations.

[1492] This system monitors the lives of the elderly, enriching their daily lives while also responding quickly to emergencies. Furthermore, by incorporating an emotion engine that recognizes the user's emotions and responds appropriately based on those emotions, the system offers a more detailed and human-like response.

[1493] Basic system configuration

[1494] The system mainly includes the following elements:

[1495] User

[1496] Device (e.g., AI speaker, smartphone)

[1497] server

[1498] Emotion Engine

[1499] Sensors (e.g. microphone, motion sensor)

[1500] 1. Initial Setup

[1501] When a user uses the system for the first time, they install and launch the application on their device (such as an AI speaker or smartphone). When the application launches, the user is prompted to enter initial information, including language settings, name, personality settings, and emergency contact information. The device then sends the entered information to the server.

[1502] The server stores the received initial setup information in a database for future access. Specifically, the "initial setup information" is basic information that a user enters when they start using the system, such as language settings, name, personality settings, and emergency contact information.

[1503] 2. Daily communication

[1504] By using the system's periodic communication function, the device actively engages with the user and engages in daily conversations. For example, the following conversation takes place every morning at 8:00:

[1505] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1506] User: "Good morning, I'm going shopping today."

[1507] Terminal: "That's good. Be careful."

[1508] This allows users to reduce feelings of loneliness in their daily lives and receive psychological support through the system.

[1509] 3. Emotional engine response

[1510] The device analyzes the user's tone of voice, facial expressions, and choice of words, and uses an emotion engine to identify the user's emotions. For example, if the user speaks with anxiety:

[1511] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[1512] A concrete example of this "means for identifying emotions" is to analyze changes in the pitch and tone of a user's voice and the words they choose when they are feeling anxious or worried, and then use the emotion engine to determine that they are feeling "anxiety." This allows the system to respond appropriately to the user's psychological state.

[1513] 4. Monitoring and Anomaly Detection

[1514] The device uses sensors and microphones to monitor surrounding sounds and movements. Specifically, the sensors analyze and record everyday sounds and movements in real time. The system detects specific anomalies and takes immediate action:

[1515] Terminal: "Suzuki-san, are you OK?" (If there is no response, try again after a few minutes.)

[1516] If there is still no response, it will be determined that something is wrong and an emergency call will be made.

[1517] 5. Emergency call

[1518] If an abnormality is detected, the device immediately reports this information to the server, which then sends a notification to registered emergency contacts, such as:

[1519] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1520] If necessary, preparations are being made to notify local medical institutions and welfare facilities.

[1521] 6. Solving the housing problem for elderly people

[1522] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[1523] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[1524] This report is provided to families, rental owners, facilities, etc. to prove that users are living safely.

[1525] Examples of prompts include:

[1526] Example 1: Morning conversation scenario

[1527] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[1528] User: "Good morning, I slept well last night."

[1529] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1530] Example 2: Emotion Recognition Scenario

[1531] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[1532] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[1533] Terminal: "That's concerning. Should I call for help right away?"

[1534] Example 3: Anomaly detection scenario

[1535] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[1536] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[1537] Device: Further no response, so report abnormality to server

[1538] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1539] This system not only enables elderly people to live their daily lives safely and with peace of mind, but also enables appropriate and prompt responses when necessary. Furthermore, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotions, providing psychological support to the user.

[1540] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1541] Step 1: Entering user information and initial settings

[1542] Input: The user enters initial configuration information (e.g., language preference, name, personality preference, emergency contact information) into the application.

[1543] How it works: The user launches the application on their device and enters the required information following the prompts.

[1544] Data processing: The terminal organizes the input information into a data structure and sends it to the server.

[1545] Output: Initial configuration information sent to the server.

[1546] Step 2: Save the initial configuration information

[1547] Input: Initial setting information sent from the terminal.

[1548] What it does: The server stores the received information in a database.

[1549] Data processing: The received initial setting information is organized into appropriate fields for storage in a database.

[1550] Output: Initial configuration information stored in the database.

[1551] Step 3: Regularly check in and keep the conversation going

[1552] Input: User schedule and past conversation history read from the server.

[1553] What it does: The device automatically calls the user at a set time (e.g., 8:00 every morning).

[1554] Data calculation: The server generates optimal conversation prompts from past data and sends them to the device.

[1555] Output: A specific call to action for the user.

[1556] Examples:

[1557] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1558] User: "Good morning, I'm going shopping today."

[1559] Terminal: "That's good. Be careful."

[1560] Step 4: Identify the emotion

[1561] Input: Data on the user's tone of voice, speech, and facial expressions.

[1562] Operation: The device sends collected voice data to the emotion engine.

[1563] Data Calculation: The emotion engine analyzes tone of voice and selected words to identify emotions.

[1564] Output: Identified emotional state.

[1565] Examples:

[1566] The user says in an anxious voice, "I'm not feeling well."

[1567] The device will identify "anxiety" through its emotion engine and respond, "That sounds worrying. Would you like us to call for support right away?"

[1568] Step 5: Surrounding area monitoring and anomaly detection

[1569] Input: Environmental sound data and movement data from sensors and microphones.

[1570] Operation: The device monitors surrounding sounds and movements in real time.

[1571] Data processing: Analyze collected data to detect abnormal patterns (e.g., loud noises or long periods of silence).

[1572] Output: Alert information if an abnormality is detected.

[1573] Examples:

[1574] Terminal: Detects an abnormal noise and calls out, "Suzuki-san, are you okay?"

[1575] If there is no response, call again after a few minutes, and if there is still no response, it is determined to be an "abnormality."

[1576] Step 6: Call Emergency Services

[1577] Input: Alert information that an anomaly has been detected.

[1578] Action: The server notifies emergency contacts.

[1579] Data processing: Check emergency contact information and convert the anomaly detection content into a message.

[1580] Output: Notification sent to emergency contacts.

[1581] Examples:

[1582] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[1583] Step 7: Recording living situation data and generating reports

[1584] Input: Daily activity data sent from your device.

[1585] Operation: The server continuously records the living situation data.

[1586] Data Computing: Analyzes collected data and generates periodic reports.

[1587] Output: Living situation data compiled as a report.

[1588] Examples:

[1589] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and send it to the relevant parties.

[1590] (Application example 2)

[1591] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1592] In recent years, the number of elderly people living alone has been increasing, creating a need for monitoring systems that allow elderly people to live their daily lives safely and with peace of mind. However, conventional monitoring systems simply detect abnormalities and are unable to understand the emotions and psychological state of the elderly or provide appropriate responses. Therefore, a system that can not only respond to emergencies but also enrich the daily lives of elderly people and provide a sense of security is needed.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1594] In this invention, the server includes means for receiving and storing the user's initial setting information, means for periodically actively calling out to the user and engaging in conversation, means for monitoring surrounding sounds and movements to detect abnormalities, means for notifying an emergency contact when an abnormality is detected, and means for understanding the user's emotions using voice recognition and an emotion engine and responding appropriately, allowing the elderly to live their daily lives with peace of mind and enabling quick and appropriate responses in emergencies.

[1595] A "user" is an individual who performs initial setup to use the system.

[1596] "Preferences Information" refers to basic information such as a user's name, language preference, personality preference, and emergency contact information.

[1597] "Actively calling out" refers to the system proactively speaking to the user and initiating a dialogue.

[1598] "Speech recognition" refers to the technology of analyzing a user's voice and understanding its content.

[1599] An "emotion engine" refers to technology that analyzes a user's emotional state based on their tone of voice, facial expressions, and choice of words.

[1600] "Surrounding sounds and movements" refers to sounds and movements of the user's body within the user's living environment.

[1601] "Means for detecting abnormalities" refers to a system that uses sensors and microphones to analyze surrounding sounds and movements in real time and identify abnormal situations.

[1602] "Emergency contact" is contact information registered by the user to receive notifications in the event of an emergency.

[1603] "Means for emergency notification" refers to a system that sends a notification to emergency contacts or appropriate authorities when an abnormality is detected.

[1604] "Lifestyle data" refers to data related to the user's daily activities and health condition.

[1605] "Means for generating reports" refers to a system that periodically creates reports summarizing the current status of the user based on collected living situation data.

[1606] "Third Party" refers to the user's family, medical institution, or other party providing necessary assistance.

[1607] "Personality settings" refers to a feature that allows users to set their own personality and preferences for the system.

[1608] "Means for responding appropriately" refers to technology that enables the system to generate appropriate responses depending on the user's emotional state and the content of the conversation.

[1609] The present invention is a system that assumes that the user is an elderly person, monitors their daily life, and responds quickly in the event of an emergency. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, providing a more detailed and human-like response.

[1610] 1. Initial Setup

[1611] When a user uses the system for the first time, they install and launch the application on their smartphone or other device. The application displays a guide for initial setup and prompts them to enter the following information:

[1612] Language settings

[1613] name

[1614] Personality settings (character and preferences)

[1615] Emergency Contact Information

[1616] The terminal transmits this initial setting information to the server, and the server stores the received information in a database.

[1617] 2. Daily communication

[1618] Based on the settings, the system will actively call out to the user and have a conversation with them periodically. For example, every morning at 8:00, the following conversation will take place:

[1619] Terminal: "Good morning, <username>. What are your plans for today?"

[1620] User: "Good morning, I'm going shopping today."

[1621] Terminal: "That's good. Be careful on your way."

[1622] 3. Emotional Engine Response

[1623] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[1624] Terminal: "Hey <username>, is there something bothering you today? Let me know what you think."

[1625] 4. Monitoring and Anomaly Detection

[1626] The device uses sensors and microphones to monitor surrounding sounds and movements, analyzing them in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies by following these steps:

[1627] Terminal: "<username>, are you okay?"

[1628] If there is no response, try again after a few minutes.

[1629] If there is still no reaction, it is determined to be abnormal.

[1630] 5. Emergency call

[1631] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1632] "An abnormality has been detected in <user name>'s status. Please check immediately."

[1633] If necessary, local medical institutions and welfare facilities will also be notified.

[1634] 6. Solving the housing problem for elderly people

[1635] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[1636] "<User name> has been living a stable daily life for the past week, with no notable abnormalities."

[1637] This report can be provided to family and relatives as needed to prove that the user is living safely.

[1638] Hardware and software used

[1639] EmotionRecognition: Emotion recognition engine

[1640] VoiceInterface: Voice Interface

[1641] SensorManager: Environmental sensor management

[1642] notify_emergency_contacts: Emergency contacts

[1643] generate_report: Report generation function

[1644] Specific examples

[1645] For example, consider the following scenario:

[1646] Terminal: "Good morning, <username>. Did you sleep well last night?"

[1647] User: "Good morning, I slept well last night."

[1648] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1649] Prompt Sentence Examples

[1650] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[1651] information:

[1652] Watching over the daily lives of the elderly

[1653] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[1654] Notify emergency contacts and medical institutions when an abnormality is detected

[1655] Generate regular lifestyle reports and provide them to families and owners.

[1656] The above is a specific embodiment for carrying out the present invention.

[1657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1658] Step 1:

[1659] Receiving and saving initial configuration information

[1660] A user installs the application on a device such as a smartphone and enters initial information (language settings, name, personality settings, emergency contact information). The device sends this information to the server, which stores it in a database.

[1661] Input: Language settings, name, personality settings, emergency contact information

[1662] Output: Initial configuration information stored in the server database

[1663] Step 2:

[1664] Beginning of daily communication

[1665] Based on the settings, the device will actively call out to the user periodically. For example, every morning at 8:00, it might say, "Good morning, <user name>. What are your plans for today?" The device will then analyze the user's response with a speech recognition engine and generate an appropriate response.

[1666] Input: User's spoken response

[1667] Output: Speech recognition results, voice response from the device

[1668] Step 3:

[1669] Emotion engine response

[1670] The device uses an emotion engine to analyze the user's tone of voice and choice of words to identify their emotional state. For example, if the user speaks in an anxious voice, the device can respond based on that by saying, "Hey <username>, are you worried about something today? Let me know."

[1671] Input: User's voice data

[1672] Output: Emotional state identification result, appropriate response voice

[1673] Step 4:

[1674] Surrounding sound and movement monitoring and anomaly detection

[1675] The device uses sensors and microphones to monitor surrounding sounds and movements in real time. If an abnormal sound or movement is detected (loud noises or long periods of silence), the device will:

[1676] First, say, "<user name>, are you okay?"

[1677] If there is no response, try again after a few minutes.

[1678] If there is still no reaction, it is determined to be abnormal.

[1679] Input: ambient sound and movement data

[1680] Output: Anomaly detection result, voice call from device

[1681] Step 5:

[1682] emergency call

[1683] If an abnormality is detected, the device immediately reports the information to the server, which then sends a notification to registered emergency contacts and, if necessary, to local medical institutions and welfare facilities.

[1684] Input: Anomaly detection results

[1685] Output: Notification to emergency contacts

[1686] Step 6:

[1687] Generate scheduled reports

[1688] The server continuously records the user's living situation data and periodically generates reports, which are provided to family members and other interested parties as needed.

[1689] Input: User's life situation data

[1690] Output: Living situation report

[1691] Specific examples

[1692] For example, consider the following scenario:

[1693] In step 2, the device says, "Good morning, <user name>. Did you sleep well last night?", and the user responds, "Good morning, I slept well last night." In step 3, the device responds, "That's good. The weather is nice today, so it would be nice to go for a walk in the daytime."

[1694] Prompt Sentence Examples

[1695] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[1696] information:

[1697] Watching over the daily lives of the elderly

[1698] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[1699] Notify emergency contacts and medical institutions when an abnormality is detected

[1700] Generate regular lifestyle reports and provide them to families and owners.

[1701] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1702] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1703] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1704] [Fourth embodiment]

[1705] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1706] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1707] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1708] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1709] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1710] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1711] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1712] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1713] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1714] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1715] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1716] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1717] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1718] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and responding quickly to emergencies. Specific embodiments of the present invention will be described in detail below.

[1719] 1. Initial Setup

[1720] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[1721] Language settings

[1722] name

[1723] Personality settings (e.g., character, personality)

[1724] Emergency Contact Information

[1725] The device sends this initial setup information to the server, which stores it in a database for later access.

[1726] 2. Daily communication

[1727] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[1728] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1729] User: "Good morning, I'm going shopping today."

[1730] Terminal: "That's good. Be careful."

[1731] In this way, the terminal generates conversation content based on the user's personality settings, providing a more friendly conversation.

[1732] 3. Monitoring and Anomaly Detection

[1733] The device has built-in sensors and microphones to monitor surrounding sounds and movements. These sensors analyze and record daily sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[1734] Terminal: "Suzuki-san, are you okay?"

[1735] If there is no response, try again after a few minutes.

[1736] If there is still no reaction, it is determined to be abnormal.

[1737] 4. Emergency call

[1738] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1739] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1740] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[1741] 5. Solving elderly housing problems

[1742] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[1743] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[1744] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[1745] Specific scenario example

[1746] Example 1: Morning conversation scenario

[1747] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[1748] User: "Good morning, I slept well last night."

[1749] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1750] Example 2: Anomaly detection scenario

[1751] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[1752] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[1753] Device: Further no response, so report abnormality to server

[1754] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1755] As described above, the present invention provides a multi-functional system that watches over elderly people to enable them to live their daily lives safely and with peace of mind, and responds quickly in emergencies.

[1756] The processing flow will be explained below.

[1757] Step 1:

[1758] The user turns on the device and starts the application, which displays a guide for initial setup.

[1759] Step 2:

[1760] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[1761] Step 3:

[1762] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[1763] Step 4:

[1764] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[1765] Step 5:

[1766] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[1767] Step 6:

[1768] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[1769] Step 7:

[1770] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[1771] Step 8:

[1772] If an abnormality is detected, such as a loud noise or a long period of silence, the device will ask the user, "Are you OK?"

[1773] Step 9:

[1774] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[1775] Step 10:

[1776] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[1777] Step 11:

[1778] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1779] Step 12:

[1780] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[1781] Step 13:

[1782] The device continuously records daily living situation data and periodically transmits it to the server.

[1783] Step 14:

[1784] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[1785] Step 15:

[1786] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[1787] The above are the specific processing steps of this system.

[1788] Example 1

[1789] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1790] Current elderly care systems often lack sufficient functionality to properly monitor the surrounding environment and respond quickly in emergencies. Furthermore, due to a lack of daily communication with users to build trust, users may find it difficult to use the system and feel uneasy. Another issue is the lack of a means to report users' living conditions to a third party and prove the safety of the elderly.

[1791] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1792] In this invention, the server includes a means for receiving the user's initial setting information and storing it in data storage, a means for actively calling and conversing with the user based on timing, and a means for monitoring surrounding sounds and movements with sensors to detect abnormalities. This allows for appropriate monitoring of the user's living environment and enables rapid response in emergencies. Furthermore, daily communication through active calling based on timing builds a relationship of trust with the user, allowing them to use the system with peace of mind. The system also provides a means for verifying the safety of the elderly by recording the user's living situation data, analyzing it, generating reports, and providing them to third parties.

[1793] "Users" refer to people who use the system, especially elderly people.

[1794] "Initial setting information" refers to information that a user enters when using the system for the first time, and includes language settings, name, personality settings, emergency contact information, etc.

[1795] "Data storage" refers to a storage device for saving a user's initial setting information and living situation data, and refers to a hard disk built into a server or cloud storage.

[1796] "Based on timing" means acting according to pre-set times or conditions.

[1797] "Actively calling out" refers to the system proactively asking messages or questions to the user.

[1798] "Conversing" refers to engaging in voice or text-based communication between a user and a system.

[1799] "Sensor" refers to a device for detecting surrounding sounds and movements, and includes sound sensors and motion sensors.

[1800] "Abnormal" refers to conditions or situations that the system determines to be deviations from normal life patterns, such as loud, unusual noises or long periods of silence.

[1801] "Emergency Contact" refers to a person or organization registered to receive notifications when an abnormality is detected in the user's behavior.

[1802] "Life situation data" is data that records the user's daily life situation, and includes conversation history and sensor data.

[1803] "Analysis" refers to analyzing collected data to extract useful information and patterns.

[1804] A "report" is a written or electronic document summarizing the analyzed living situation data and provided to a third party to confirm the user's safety.

[1805] The present invention is a monitoring system for elderly people that supports their daily lives and provides technology for rapid response in emergencies. This system is used by users using devices such as smartphones and AI speakers. Specific embodiments for carrying out the invention are described below.

[1806] 1. Initial Setup

[1807] The user installs and launches a dedicated application on their smartphone or AI speaker. This application displays a guide for the user to input initial setup information. The initial setup information includes:

[1808] Language settings

[1809] name

[1810] Personality settings (character, personality, etc.)

[1811] Emergency Contact Information

[1812] After entering this initial setup information, the device sends it to a server via the Internet and stores it in data storage. Specific database management systems used on the server include MySQL and PostgreSQL.

[1813] 2. Daily communication

[1814] The device actively calls the user periodically and engages in daily communication. For example, the following conversation takes place every morning at 8:00:

[1815] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1816] User: "Good morning. I'm going shopping today."

[1817] Terminal: "That's good. Be careful."

[1818] This communication is done using voice recognition software (e.g., Google Speech-to-Text) and generative AI models, which generate more friendly conversational content based on the user's personality settings.

[1819] 3. Monitoring and Anomaly Detection

[1820] The device monitors the user's surroundings using built-in sensors (e.g., sound sensors, motion sensors) and microphones. This allows it to record and analyze everyday sounds and movement patterns in real time. When certain abnormal conditions occur (e.g., loud noises or long periods of silence), the device detects the abnormality as follows:

[1821] Terminal: "Suzuki-san, are you okay?"

[1822] Call again after a few minutes, and if there is still no response, assume that something is wrong.

[1823] 4. Emergency call

[1824] If an abnormality is detected, the device immediately sends this information to the server, which then sends a notification to the registered emergency contacts via SMS or email (e.g., Twilio, SendGrid). The notification content is as follows:

[1825] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1826] 5. Solving elderly housing problems

[1827] The device records the user's daily life situation and sends it to the server. The server analyzes the received data and generates periodic reports. These reports are generated using data analysis software (e.g., Python's Pandas library). The report contents are provided as follows:

[1828] Server: "Mr. Suzuki has been living his daily life stably for the past week, with no notable abnormalities."

[1829] This allows family members and other people involved to confirm the safety of the user.

[1830] Prompt Sentence Examples

[1831] Here are some example prompts to use when inputting a scenario into a generative AI model:

[1832] "We want to create a system that helps elderly people live their daily lives with peace of mind. Please generate scenarios that involve daily conversations, anomaly detection, and emergency calls. Please include specific examples of morning conversations and anomaly detection."

[1833] The above is a detailed description of the mode for carrying out the invention, and by using this system, elderly people can live their daily lives safely and with peace of mind.

[1834] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1835] System program processing flow

[1836] Initial Setup

[1837] Step 1:

[1838] Users install and launch a dedicated application on their smartphone or AI speaker.

[1839] Input: Install and launch the application

[1840] Output: The screen where the application starts

[1841] Step 2:

[1842] The terminal displays an initial setting screen for the application and prompts the user to input initial setting information.

[1843] What happens: A user enters language preferences, name, personality preferences, and emergency contact information into an application.

[1844] Input: User-entered initial configuration information

[1845] Output: The entered information is displayed on the terminal.

[1846] Step 3:

[1847] The terminal transmits the input initial setting information to a server via the Internet.

[1848] Specific operation: Sends information using the device's network communication function

[1849] Input: Initial setting information entered into the terminal

[1850] Output: Initial configuration information sent to the server

[1851] Step 4:

[1852] The server stores the received initial setting information in a data storage.

[1853] Specific operation: Use a database management system (MySQL or PostgreSQL)

[1854] Input: Received initial setting information

[1855] Output: Initial settings information stored in the database

[1856] Daily communication

[1857] Step 1:

[1858] The device uses an internal clock and timer function to call the user at the set time.

[1859] Specific behavior: Send a message to the user every morning at 8am

[1860] Input: Set time information

[1861] Output: A message to the user

[1862] Step 2:

[1863] The device uses voice recognition software (Google Speech-to-Text) to analyze the user's responses and generate appropriate replies.

[1864] What it does: Converts user speech to text and generates a response using a generative AI model.

[1865] Input: User's voice response

[1866] Output: Terminal-generated reply message

[1867] Specific working example:

[1868] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1869] User: "Good morning. I'm going shopping today."

[1870] Terminal: "That's good. Be careful."

[1871] Monitoring and Anomaly Detection

[1872] Step 1:

[1873] The device uses built-in sensors (sound sensor, motion sensor) and microphone to monitor the user's surroundings.

[1874] Specific operation: Sensors monitor the living environment in real time

[1875] Input: Sound and movement data from the surrounding area

[1876] Output: Monitoring data

[1877] Step 2:

[1878] The device analyzes the collected sensor data and detects abnormalities when certain abnormal conditions occur (loud noises, long periods of silence).

[1879] Specific operation: Real-time data analysis

[1880] Input: Sensor data

[1881] Output: Anomaly detection results

[1882] Specific working example:

[1883] Terminal: "Suzuki-san, are you okay?"

[1884] If there is no response, try again in a few minutes.

[1885] If there is still no reaction, it is considered abnormal.

[1886] emergency call

[1887] Step 1:

[1888] If an abnormality is detected, the terminal immediately transmits the information to the server.

[1889] Specific operation: Sends anomaly information via internet connection

[1890] Input: Anomaly detection information

[1891] Output: Anomaly information sent to the server

[1892] Step 2:

[1893] The server sends a notification to the registered emergency contact based on the received abnormality information.

[1894] Specific operation: Uses SMS and email sending systems (Twilio, SendGrid)

[1895] Input: Anomaly information

[1896] Output: Notification message to emergency contacts

[1897] Specific working example:

[1898] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[1899] Solving elderly housing problems

[1900] Step 1:

[1901] The terminal records the user's daily life situation and periodically transmits it to the server.

[1902] Specific operation: Uses the device's local storage and internet connection

[1903] Input: Daily life situation data

[1904] Output: Status data sent to the server

[1905] Step 2:

[1906] The server analyzes the received data and generates periodic reports.

[1907] Specific operation: Data analysis software (Python's Pandas library)

[1908] Input: Received living situation data

[1909] Output: Generated report

[1910] Specific working example:

[1911] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and provide it to his family and related parties.

[1912] The above is the specific flow of the system program processing and details of each step.

[1913] (Application example 1)

[1914] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1915] There is a need to ensure rapid response in emergencies while monitoring the safety of elderly people's daily lives. However, many current systems have low accuracy in detecting abnormalities, and visual and auditory monitoring in particular is insufficient. Furthermore, they are limited to simple warning sounds and notifications and lack functionality for daily communication with users, which can lead to a decline in the quality of daily life. Another issue is the lack of detailed recording and reporting of abnormal situations and reliable notification to emergency contacts.

[1916] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1917] In this invention, the server includes means for receiving and saving the user's initial setting information, means for periodically actively calling and conversing with the user, means for monitoring surrounding sounds and movements and detecting abnormalities, means for notifying an emergency contact when an abnormality is detected, means for monitoring audio, video, and motion data using sensors and cameras, and means for checking multiple times if there is no response and determining that there is an abnormality if there is still no response. This makes it possible to comprehensively monitor the lives of the elderly and respond quickly and effectively in emergencies.

[1918] The "means for receiving and saving user initial setting information" is a function for collecting information that a user inputs when using the system for the first time and storing it in a database.

[1919] "Means for actively calling out to and conversing with the user on a regular basis" refers to a function in which the system speaks to and communicates with the user based on a pre-set schedule.

[1920] "Means for monitoring surrounding sounds and movements and detecting abnormalities" refers to a system that uses sound sensors and motion sensors to constantly monitor environmental information and detect abnormalities.

[1921] "Means of notifying emergency contacts when an abnormality is detected" is a function that sends warnings and notifications to pre-registered emergency contacts when the system recognizes an abnormality.

[1922] "Means for monitoring audio, video, and motion data using sensors and cameras" refers to a function that utilizes sensors and cameras to acquire and monitor data such as audio, video, and motion.

[1923] "Means for making multiple confirmations if there is no response, and determining that an abnormality exists if there is still no response" is a function that makes multiple confirmation calls if the user does not respond within a certain period of time, and determines that an abnormality exists if no response is still received.

[1924] The present invention is a system for comprehensively monitoring the lives of elderly people and responding quickly and effectively in emergencies. The system includes a means for receiving and storing user initial setup information, including name, emergency contact information, and personality settings. This allows the device to respond appropriately based on the user's unique information.

[1925] The system has a means to actively call out to the user periodically and engage in conversation. For example, every morning at 8:00, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?" This conversation is generated based on the user's personality settings, making it more friendly.

[1926] The device is also equipped with a means of monitoring surrounding sounds and movements to detect abnormalities. Audio, video, and motion data are monitored using sensors and cameras. If an abnormality is detected, the device immediately reports this information to the server. The server then sends a notification to the registered emergency contacts. In addition, if there is no response, the device will check multiple times and, if there is still no response, will determine that an abnormality has occurred. This ensures a prompt response when an elderly person is in a dangerous situation.

[1927] The user's living situation data is constantly recorded and generated as a report. This report is automatically generated using a generative AI model and provided to family members or medical institutions as needed. This allows those involved to constantly keep track of the user's health and living situation.

[1928] The following are specific examples of the present invention.

[1929] Example 1:

[1930] 1. The user puts on the smart glasses and launches the application.

[1931] 2. The system speaks to the user, "Good morning, Suzuki-san. What are your plans for today?"

[1932] 3. The user replies, "Good morning, I'm going shopping today."

[1933] 4. The system responds, "That's good. Be careful."

[1934] Example 2:

[1935] 1. The system monitors surrounding sounds and activity.

[1936] 2. Suddenly, a loud noise occurs and the system detects an abnormality.

[1937] 3. The system calls out, "Mr. Suzuki, are you okay?" but there is no response.

[1938] 4. He then calls out a second time, "Mr. Suzuki, please respond," but there is still no response.

[1939] 5. The system reports the abnormality to the server, and the server sends a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1940] An example of a prompt sentence to input to the generative AI model is as follows:

[1941] "Apply the invention of the elderly monitoring system to create an application scenario for a security service using smart glasses."

[1942] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1943] Step 1:

[1944] The device receives and stores the user's initial setting information. The initial setting information (name, emergency contact, personality settings) is received from the user, sent to the server, and stored in a database. This allows the device to respond appropriately based on the user's unique information.

[1945] Step 2:

[1946] It periodically actively calls out to the user and starts a conversation. The user's name and personality settings are obtained from the server as input, and the generative AI model generates the conversation content as output, which the device then relays to the user via voice. For example, it might say, "Good morning, Mr. / Ms. X. What are your plans for today?"

[1947] Step 3:

[1948] Sensors and cameras are used to monitor audio, video, and motion data. The device monitors surrounding sounds and motion in real time, taking environmental data as input. It analyzes the data to detect abnormalities, and if no abnormalities are found, records the data as normal, and if an abnormality is detected, moves on to the next step.

[1949] Step 4:

[1950] If an abnormality is detected, a response is checked. The terminal calls out, "Mr. Suzuki, are you OK?", but if there is no response from the user as input, the check is made multiple times (for example, again after 5 minutes). The output confirms that there is no response, and once this step is complete, the process proceeds to the next step.

[1951] Step 5:

[1952] The abnormality is reported to the server and an emergency contact is notified. As input, the device sends "no response" data to the server, and as output, the server sends a notification to the registered emergency contact. The content of the notification is "An abnormality has been detected in Suzuki's situation. Please check immediately."

[1953] Step 6:

[1954] It records the user's living situation data and generates a report. Data from sensors and cameras is sent as input to a server, and a living situation report is automatically generated using a generative AI model. The report generated as output is provided to family members and medical institutions. This allows those involved to constantly keep track of the user's health condition and living situation.

[1955] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1956] The present invention is a system for monitoring the lives of elderly users, enriching their daily lives, and quickly responding to emergencies. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, thereby providing a more detailed and human-like response. Specific embodiments of the present invention are described in detail below.

[1957] 1. Initial Setup

[1958] When a user uses the system for the first time, they install and launch the application on their device (e.g., an AI speaker or smartphone). The application displays a guide for initial setup and prompts them to enter the following information:

[1959] Language settings

[1960] name

[1961] Personality settings (character, personality)

[1962] Emergency Contact Information

[1963] The device sends this initial setup information to the server, which stores it in a database for later access.

[1964] 2. Daily communication

[1965] Based on the settings, the system will actively contact the user periodically and engage in daily conversations. For example, every morning at 8:00 AM, the following conversation will take place:

[1966] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[1967] User: "Good morning, I'm going shopping today."

[1968] Terminal: "That's good. Be careful."

[1969] 3. Emotional engine response

[1970] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[1971] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[1972] 4. Monitoring and Anomaly Detection

[1973] The device monitors surrounding sounds and movements using sensors and microphones. These sensors analyze and record everyday sounds and movement patterns in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies through the following steps:

[1974] Terminal: "Suzuki-san, are you okay?"

[1975] If there is no response, try again after a few minutes.

[1976] If there is still no reaction, it is determined to be abnormal.

[1977] 5. Emergency call

[1978] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[1979] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[1980] Additionally, preparations are in place to notify local medical and welfare facilities if necessary.

[1981] 6. Solving the housing problem for elderly people

[1982] The system continuously records data on the user's daily life and generates periodic reports based on that data. For example, the following reports are generated:

[1983] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[1984] This report is provided to families, rental owners, and facilities as needed to prove that users are living safely.

[1985] Specific scenario example

[1986] Example 1: Morning conversation scenario

[1987] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[1988] User: "Good morning, I slept well last night."

[1989] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[1990] Example 2: Emotion Recognition Scenario

[1991] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[1992] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[1993] Terminal: "That's concerning. Should I call for help right away?"

[1994] Example 3: Anomaly detection scenario

[1995] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[1996] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[1997] Device: Further no response, so report abnormality to server

[1998] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[1999] As described above, the present invention is a multi-functional system that watches over elderly people to ensure that they can live their daily lives safely and with peace of mind, responds quickly in emergencies, and uses an emotion engine to provide flexible responses that correspond to the user's emotions.

[2000] The processing flow will be explained below.

[2001] Step 1:

[2002] The user turns on the device and starts the application, which displays a guide for initial setup.

[2003] Step 2:

[2004] The device prompts the user to enter language settings, name, personality settings (character, character), and emergency contact information.

[2005] Step 3:

[2006] The user enters the necessary information and completes the setup. Once the setup information is complete, the device sends this information to the server.

[2007] Step 4:

[2008] The server stores the received initial setting information in the database, after which the system goes into daily use mode.

[2009] Step 5:

[2010] Every morning at a set time, the device will actively call out to the user, for example, "Good morning, Mr. / Ms. X. What are your plans for today?"

[2011] Step 6:

[2012] When the user responds to the question, the device analyzes the content and generates the next question and answer, for example, suggesting, "The weather is nice, why don't you go for a walk?"

[2013] Step 7:

[2014] The device analyzes the user's tone of voice and word choice and uses an emotion engine to recognize the user's emotions.

[2015] Step 8:

[2016] The device generates the following responses depending on the user's emotions. For example, if the user speaks in an anxious voice, the device responds, "Mr. Suzuki, are you worried about something today? Please tell me."

[2017] Step 9:

[2018] The device monitors surrounding sounds and movements using sensors and microphones, and analyzes everyday sounds and movement patterns in real time.

[2019] Step 10:

[2020] If an abnormality is detected, such as a loud noise or a long period of silence, the device will call out to the user, saying something like, "Mr. Suzuki, are you OK?"

[2021] Step 11:

[2022] If the user does not respond to the terminal's prompt within a certain period of time (for example, five minutes), the terminal will repeat the prompt.

[2023] Step 12:

[2024] If there is no response to the repeated call, the device determines that an abnormality has occurred and reports this information to the server.

[2025] Step 13:

[2026] The server will send a notification to the registered emergency contact stating, "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[2027] Step 14:

[2028] If necessary, the server will send similar notifications to local medical institutions and welfare facilities to encourage them to take prompt action.

[2029] Step 15:

[2030] The device continuously records daily living situation data and periodically transmits it to the server.

[2031] Step 16:

[2032] Based on the received living situation data, the server generates a report indicating the safety and stability of the user's life.

[2033] Step 17:

[2034] The generated report is provided from the server to third parties such as family members, rental property owners, facilities, etc. For example, it may report, "Mr. Suzuki has been living a stable daily life every day for the past week, with no notable abnormalities."

[2035] The above are the specific processing steps of this system, which monitors the user's life and provides flexible responses according to their emotions.

[2036] Example 2

[2037] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2038] Today's elderly people face a variety of problems in their daily lives. These include feelings of loneliness and anxiety about their safety, and they often find themselves unable to receive the support they need promptly. Furthermore, it is difficult for their families and relatives to understand the elderly's living situation, which risks delaying appropriate responses. There is a need to develop a system that can solve these problems and enable the elderly to live their daily lives with peace of mind.

[2039] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving and saving the user's initial setting information, a means for actively calling out to the user periodically and holding a conversation, a means for analyzing the user's tone of voice, facial expression, and words selected to identify emotions, a means for monitoring surrounding sounds and movements and detecting abnormalities, and a means for notifying an emergency contact when an abnormality is detected. This enables the system to appropriately monitor the living conditions of the elderly and automatically take appropriate measures as necessary.

[2040] "Initial setup information" refers to basic information that a user enters when they first start using the system, including language settings, name, personality settings, and emergency contact information.

[2041] "Active calling" is a function that enables the system to communicate with the user on a regular basis by proactively speaking to the user.

[2042] "Means for conducting conversation" refers to a function for exchanging information with a user using voice or text.

[2043] "Voice tone" refers to characteristics such as pitch, strength, and rhythm of a user's voice when they speak, and is used to identify emotions.

[2044] "Facial expression" refers to facial movements and changes in facial muscles, and is an important factor in identifying emotions.

[2045] "Selected words" refer to words and phrases that users use in conversation and are used to identify emotions.

[2046] "Means for identifying emotions" is a function that determines the user's emotional state by analyzing the user's tone of voice, facial expressions, and words they choose.

[2047] "Means for monitoring surrounding sounds and movements" refers to a function that uses sensors and microphones to detect environmental sounds and movements and discover abnormalities.

[2048] "Means for detecting abnormalities" refers to a function that recognizes an abnormal situation when the system meets pre-set conditions (e.g., a loud abnormal noise or a long period of silence).

[2049] "Emergency contacts" are people to whom notifications are sent when an abnormality is detected, including family members and local medical institutions.

[2050] "Life situation data" refers to various information related to the user's daily life, such as behavioral patterns and health status.

[2051] A "report" is a document in the form of a report generated based on data collected by the system, detailing the user's living situation.

[2052] "Personality setting" refers to the user pre-setting their own character and personality within the system, and is used to customize the content of conversations.

[2053] This system monitors the lives of the elderly, enriching their daily lives while also responding quickly to emergencies. Furthermore, by incorporating an emotion engine that recognizes the user's emotions and responds appropriately based on those emotions, the system offers a more detailed and human-like response.

[2054] Basic system configuration

[2055] The system mainly includes the following elements:

[2056] User

[2057] Device (e.g., AI speaker, smartphone)

[2058] server

[2059] Emotion Engine

[2060] Sensors (e.g. microphone, motion sensor)

[2061] 1. Initial Setup

[2062] When a user uses the system for the first time, they install and launch the application on their device (such as an AI speaker or smartphone). When the application launches, the user is prompted to enter initial information, including language settings, name, personality settings, and emergency contact information. The device then sends the entered information to the server.

[2063] The server stores the received initial setup information in a database for future access. Specifically, the "initial setup information" is basic information that a user enters when they start using the system, such as language settings, name, personality settings, and emergency contact information.

[2064] 2. Daily communication

[2065] By using the system's periodic communication function, the device actively engages with the user and engages in daily conversations. For example, the following conversation takes place every morning at 8:00:

[2066] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[2067] User: "Good morning, I'm going shopping today."

[2068] Terminal: "That's good. Be careful."

[2069] This allows users to reduce feelings of loneliness in their daily lives and receive psychological support through the system.

[2070] 3. Emotional engine response

[2071] The device analyzes the user's tone of voice, facial expressions, and choice of words, and uses an emotion engine to identify the user's emotions. For example, if the user speaks with anxiety:

[2072] Terminal: "Mr. Suzuki, is there anything that is bothering you today? Please tell me."

[2073] A concrete example of this "means for identifying emotions" is to analyze changes in the pitch and tone of a user's voice and the words they choose when they are feeling anxious or worried, and then use the emotion engine to determine that they are feeling "anxiety." This allows the system to respond appropriately to the user's psychological state.

[2074] 4. Monitoring and Anomaly Detection

[2075] The device uses sensors and microphones to monitor surrounding sounds and movements. Specifically, the sensors analyze and record everyday sounds and movements in real time. The system detects specific anomalies and takes immediate action:

[2076] Terminal: "Suzuki-san, are you OK?" (If there is no response, try again after a few minutes.)

[2077] If there is still no response, it will be determined that something is wrong and an emergency call will be made.

[2078] 5. Emergency call

[2079] If an abnormality is detected, the device immediately reports this information to the server, which then sends a notification to registered emergency contacts, such as:

[2080] "An abnormality has been detected in Mr. / Ms. XX's condition. Please check immediately."

[2081] If necessary, preparations are being made to notify local medical institutions and welfare facilities.

[2082] 6. Solving the housing problem for elderly people

[2083] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[2084] "Mr. Suzuki has been living his daily life stably every day for the past week, and there have been no notable abnormalities."

[2085] This report is provided to families, rental owners, facilities, etc. to prove that users are living safely.

[2086] Examples of prompts include:

[2087] Example 1: Morning conversation scenario

[2088] Terminal: "Good morning, Mr. / Ms. XX. Did you sleep well last night?"

[2089] User: "Good morning, I slept well last night."

[2090] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[2091] Example 2: Emotion Recognition Scenario

[2092] Terminal: "Good morning, Mr. / Ms. XX. Is there anything I can help you with?"

[2093] User: "Yes, I haven't been feeling well since yesterday," says an anxious voice.

[2094] Terminal: "That's concerning. Should I call for help right away?"

[2095] Example 3: Anomaly detection scenario

[2096] Terminal: Abnormal noise detected in the surrounding area. "Suzuki-san, are you okay? Is something wrong?"

[2097] Terminal: No response for 5 minutes. "Suzuki-san, please respond."

[2098] Device: Further no response, so report abnormality to server

[2099] Server: Send a notification to the emergency contact saying, "An abnormality has been detected in Suzuki's condition. Please check immediately."

[2100] This system not only enables elderly people to live their daily lives safely and with peace of mind, but also enables appropriate and prompt responses when necessary. Furthermore, by utilizing an emotion engine, it is possible to respond flexibly according to the user's emotions, providing psychological support to the user.

[2101] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2102] Step 1: Entering user information and initial settings

[2103] Input: The user enters initial configuration information (e.g., language preference, name, personality preference, emergency contact information) into the application.

[2104] How it works: The user launches the application on their device and enters the required information following the prompts.

[2105] Data processing: The terminal organizes the input information into a data structure and sends it to the server.

[2106] Output: Initial configuration information sent to the server.

[2107] Step 2: Save the initial configuration information

[2108] Input: Initial setting information sent from the terminal.

[2109] What it does: The server stores the received information in a database.

[2110] Data processing: The received initial setting information is organized into appropriate fields for storage in a database.

[2111] Output: Initial configuration information stored in the database.

[2112] Step 3: Regularly check in and keep the conversation going

[2113] Input: User schedule and past conversation history read from the server.

[2114] What it does: The device automatically calls the user at a set time (e.g., 8:00 every morning).

[2115] Data calculation: The server generates optimal conversation prompts from past data and sends them to the device.

[2116] Output: A specific call to action for the user.

[2117] Examples:

[2118] Terminal: "Good morning, Mr. / Ms. XX. What are your plans for today?"

[2119] User: "Good morning, I'm going shopping today."

[2120] Terminal: "That's good. Be careful."

[2121] Step 4: Identify the emotion

[2122] Input: Data on the user's tone of voice, speech, and facial expressions.

[2123] Operation: The device sends collected voice data to the emotion engine.

[2124] Data Calculation: The emotion engine analyzes tone of voice and selected words to identify emotions.

[2125] Output: Identified emotional state.

[2126] Examples:

[2127] The user says in an anxious voice, "I'm not feeling well."

[2128] The device will identify "anxiety" through its emotion engine and respond, "That sounds worrying. Would you like us to call for support right away?"

[2129] Step 5: Surrounding area monitoring and anomaly detection

[2130] Input: Environmental sound data and movement data from sensors and microphones.

[2131] Operation: The device monitors surrounding sounds and movements in real time.

[2132] Data processing: Analyze collected data to detect abnormal patterns (e.g., loud noises or long periods of silence).

[2133] Output: Alert information if an abnormality is detected.

[2134] Examples:

[2135] Terminal: Detects an abnormal noise and calls out, "Suzuki-san, are you okay?"

[2136] If there is no response, call again after a few minutes, and if there is still no response, it is determined to be an "abnormality."

[2137] Step 6: Call Emergency Services

[2138] Input: Alert information that an anomaly has been detected.

[2139] Action: The server notifies emergency contacts.

[2140] Data processing: Check emergency contact information and convert the anomaly detection content into a message.

[2141] Output: Notification sent to emergency contacts.

[2142] Examples:

[2143] Server: Send a notification to the emergency contact stating, "An abnormality has been detected in Mr. / Ms. X's condition. Please check immediately."

[2144] Step 7: Recording living situation data and generating reports

[2145] Input: Daily activity data sent from your device.

[2146] Operation: The server continuously records the living situation data.

[2147] Data Computing: Analyzes collected data and generates periodic reports.

[2148] Output: Living situation data compiled as a report.

[2149] Examples:

[2150] Server: Generate a report stating, "Mr. Suzuki has been living his daily life stably every day for the past week, with no notable abnormalities," and send it to the relevant parties.

[2151] (Application example 2)

[2152] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2153] In recent years, the number of elderly people living alone has been increasing, creating a need for monitoring systems that allow elderly people to live their daily lives safely and with peace of mind. However, conventional monitoring systems simply detect abnormalities and are unable to understand the emotions and psychological state of the elderly or provide appropriate responses. Therefore, a system that can not only respond to emergencies but also enrich the daily lives of elderly people and provide a sense of security is needed.

[2154] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2155] In this invention, the server includes means for receiving and storing the user's initial setting information, means for periodically actively calling out to the user and engaging in conversation, means for monitoring surrounding sounds and movements to detect abnormalities, means for notifying an emergency contact when an abnormality is detected, and means for understanding the user's emotions using voice recognition and an emotion engine and responding appropriately, allowing the elderly to live their daily lives with peace of mind and enabling quick and appropriate responses in emergencies.

[2156] A "user" is an individual who performs initial setup to use the system.

[2157] "Preferences Information" refers to basic information such as a user's name, language preference, personality preference, and emergency contact information.

[2158] "Actively calling out" refers to the system proactively speaking to the user and initiating a dialogue.

[2159] "Speech recognition" refers to the technology of analyzing a user's voice and understanding its content.

[2160] An "emotion engine" refers to technology that analyzes a user's emotional state based on their tone of voice, facial expressions, and choice of words.

[2161] "Surrounding sounds and movements" refers to sounds and movements of the user's body within the user's living environment.

[2162] "Means for detecting abnormalities" refers to a system that uses sensors and microphones to analyze surrounding sounds and movements in real time and identify abnormal situations.

[2163] "Emergency contact" is contact information registered by the user to receive notifications in the event of an emergency.

[2164] "Means for emergency notification" refers to a system that sends a notification to emergency contacts or appropriate authorities when an abnormality is detected.

[2165] "Lifestyle data" refers to data related to the user's daily activities and health condition.

[2166] "Means for generating reports" refers to a system that periodically creates reports summarizing the current status of the user based on collected living situation data.

[2167] "Third Party" refers to the user's family, medical institution, or other party providing necessary assistance.

[2168] "Personality settings" refers to a feature that allows users to set their own personality and preferences for the system.

[2169] "Means for responding appropriately" refers to technology that enables the system to generate appropriate responses depending on the user's emotional state and the content of the conversation.

[2170] The present invention is a system that assumes that the user is an elderly person, monitors their daily life, and responds quickly in the event of an emergency. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions and responds appropriately to those emotions, providing a more detailed and human-like response.

[2171] 1. Initial Setup

[2172] When a user uses the system for the first time, they install and launch the application on their smartphone or other device. The application displays a guide for initial setup and prompts them to enter the following information:

[2173] Language settings

[2174] name

[2175] Personality settings (character and preferences)

[2176] Emergency Contact Information

[2177] The terminal transmits this initial setting information to the server, and the server stores the received information in a database.

[2178] 2. Daily communication

[2179] Based on the settings, the system will actively call out to the user and have a conversation with them periodically. For example, every morning at 8:00, the following conversation will take place:

[2180] Terminal: "Good morning, <username>. What are your plans for today?"

[2181] User: "Good morning, I'm going shopping today."

[2182] Terminal: "That's good. Be careful on your way."

[2183] 3. Emotional Engine Response

[2184] The device analyzes the user's tone of voice, facial expressions, and word choices to identify their emotions using an emotion engine. This data is used to better tailor responses. For example, if the user sounds anxious, the device might respond as follows:

[2185] Terminal: "Hey <username>, is there something bothering you today? Let me know what you think."

[2186] 4. Monitoring and Anomaly Detection

[2187] The device uses sensors and microphones to monitor surrounding sounds and movements, analyzing them in real time. When certain conditions occur (such as loud noises or prolonged periods of silence), the device detects anomalies by following these steps:

[2188] Terminal: "<username>, are you okay?"

[2189] If there is no response, try again after a few minutes.

[2190] If there is still no reaction, it is determined to be abnormal.

[2191] 5. Emergency call

[2192] If an abnormality is detected, the device immediately reports this information to the server, which then sends the following notification to registered emergency contacts (such as family members):

[2193] "An abnormality has been detected in <user name>'s status. Please check immediately."

[2194] If necessary, local medical institutions and welfare facilities will also be notified.

[2195] 6. Solving the housing problem for elderly people

[2196] The system continuously records data on the user's daily life and generates periodic reports based on that data, such as:

[2197] "<User name> has been living a stable daily life for the past week, with no notable abnormalities."

[2198] This report can be provided to family and relatives as needed to prove that the user is living safely.

[2199] Hardware and software used

[2200] EmotionRecognition: Emotion recognition engine

[2201] VoiceInterface: Voice Interface

[2202] SensorManager: Environmental sensor management

[2203] notify_emergency_contacts: Emergency contacts

[2204] generate_report: Report generation function

[2205] Specific examples

[2206] For example, consider the following scenario:

[2207] Terminal: "Good morning, <username>. Did you sleep well last night?"

[2208] User: "Good morning, I slept well last night."

[2209] Terminal: "That's good. The weather is nice today, so it'll be nice to go for a walk in the daytime."

[2210] Prompt Sentence Examples

[2211] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[2212] information:

[2213] Watching over the daily lives of the elderly

[2214] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[2215] Notify emergency contacts and medical institutions when an abnormality is detected

[2216] Generate regular lifestyle reports and provide them to families and owners.

[2217] The above is a specific embodiment for carrying out the present invention.

[2218] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2219] Step 1:

[2220] Receiving and saving initial configuration information

[2221] A user installs the application on a device such as a smartphone and enters initial information (language settings, name, personality settings, emergency contact information). The device sends this information to the server, which stores it in a database.

[2222] Input: Language settings, name, personality settings, emergency contact information

[2223] Output: Initial configuration information stored in the server database

[2224] Step 2:

[2225] Beginning of daily communication

[2226] Based on the settings, the device will actively call out to the user periodically. For example, every morning at 8:00, it might say, "Good morning, <user name>. What are your plans for today?" The device will then analyze the user's response with a speech recognition engine and generate an appropriate response.

[2227] Input: User's spoken response

[2228] Output: Speech recognition results, voice response from the device

[2229] Step 3:

[2230] Emotion engine response

[2231] The device uses an emotion engine to analyze the user's tone of voice and choice of words to identify their emotional state. For example, if the user speaks in an anxious voice, the device can respond based on that by saying, "Hey <username>, are you worried about something today? Let me know."

[2232] Input: User's voice data

[2233] Output: Emotional state identification result, appropriate response voice

[2234] Step 4:

[2235] Surrounding sound and movement monitoring and anomaly detection

[2236] The device uses sensors and microphones to monitor surrounding sounds and movements in real time. If an abnormal sound or movement is detected (loud noises or long periods of silence), the device will:

[2237] First, say, "<user name>, are you okay?"

[2238] If there is no response, try again after a few minutes.

[2239] If there is still no reaction, it is determined to be abnormal.

[2240] Input: ambient sound and movement data

[2241] Output: Anomaly detection result, voice call from device

[2242] Step 5:

[2243] emergency call

[2244] If an abnormality is detected, the device immediately reports the information to the server, which then sends a notification to registered emergency contacts and, if necessary, to local medical institutions and welfare facilities.

[2245] Input: Anomaly detection results

[2246] Output: Notification to emergency contacts

[2247] Step 6:

[2248] Generate scheduled reports

[2249] The server continuously records the user's living situation data and periodically generates reports, which are provided to family members and other interested parties as needed.

[2250] Input: User's life situation data

[2251] Output: Living situation report

[2252] Specific examples

[2253] For example, consider the following scenario:

[2254] In step 2, the device says, "Good morning, <user name>. Did you sleep well last night?", and the user responds, "Good morning, I slept well last night." In step 3, the device responds, "That's good. The weather is nice today, so it would be nice to go for a walk in the daytime."

[2255] Prompt Sentence Examples

[2256] Please create a concrete example of an elderly monitoring security service using the information below. Please include a detailed description of the features, hardware and software used, and data processing details.

[2257] information:

[2258] Watching over the daily lives of the elderly

[2259] Understand the emotions of the elderly and respond appropriately using voice recognition and an emotion engine

[2260] Notify emergency contacts and medical institutions when an abnormality is detected

[2261] Generate regular lifestyle reports and provide them to families and owners.

[2262] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2263] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2264] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2265] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2266] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2267] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2268] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2269] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2270] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2271] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2272] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2273] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2274] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2275] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2276] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2277] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2278] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2279] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2280] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2281] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2282] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2283] The following is further disclosed regarding the above embodiment.

[2284] (Claim 1)

[2285] means for receiving and storing user preference information;

[2286] A means for actively calling out to users on a regular basis and engaging in conversation;

[2287] A means of monitoring surrounding sounds and movements and detecting abnormalities;

[2288] A means of notifying emergency contacts if an abnormality is detected;

[2289] A system including:

[2290] (Claim 2)

[2291] means for recording user life situation data and generating reports;

[2292] a means for providing the generated reports to a third party;

[2293] The system of claim 1 further comprising:

[2294] (Claim 3)

[2295] means for accepting a user's personality setting and generating conversation content based on the personality;

[2296] The system of claim 1 further comprising:

[2297] "Example 1"

[2298] (Claim 1)

[2299] means for receiving and storing user initial setting information in a data storage;

[2300] A means for actively calling out to a user and engaging in conversation based on timing;

[2301] A means of monitoring surrounding sounds and movements with sensors and detecting abnormalities,

[2302] means for transmitting information to an emergency contact if an anomaly is detected;

[2303] A system including:

[2304] (Claim 2)

[2305] means for recording and analyzing user's living situation data to generate reports;

[2306] a means for providing the generated reports to a third party;

[2307] The system of claim 1 further comprising:

[2308] (Claim 3)

[2309] means for receiving user personality information and structuring conversation content based on the information;

[2310] The system of claim 1 further comprising:

[2311] "Application Example 1"

[2312] (Claim 1)

[2313] means for receiving and storing user preference information;

[2314] A means for actively calling out to users on a regular basis and engaging in conversation;

[2315] A means of monitoring surrounding sounds and movements and detecting abnormalities;

[2316] A means of notifying emergency contacts if an abnormality is detected;

[2317] a means for monitoring audio, video and motion data using sensors and cameras;

[2318] If there is no response, multiple checks are made, and if there is still no response, a method is used to determine that an abnormality has occurred.

[2319] A system including:

[2320] (Claim 2)

[2321] means for recording user life situation data and generating reports;

[2322] a means for providing the generated reports to a third party;

[2323] A means for generating conversation content tailored to the user using a generative AI model;

[2324] A means for engaging in everyday conversation based on the generated conversation content;

[2325] The system of claim 1 further comprising:

[2326] (Claim 3)

[2327] means for accepting a user's personality setting and generating conversation content based on the personality;

[2328] A way to input prompts into the generative AI model to customize the conversation;

[2329] The system of claim 1 further comprising:

[2330] "Example 2: Combining Emotion Engines"

[2331] (Claim 1)

[2332] means for receiving and storing user preference information;

[2333] A means for actively calling out to users on a regular basis and engaging in conversation;

[2334] A means of identifying emotions by analyzing the user's tone of voice, facial expressions, and choice of words;

[2335] A means of monitoring surrounding sounds and movements and detecting abnormalities;

[2336] A means of notifying emergency contacts if an abnormality is detected;

[2337] A system including:

[2338] (Claim 2) ...

Claims

1. means for receiving and storing user preference information; A means for actively calling out to users on a regular basis and engaging in conversation; A means of monitoring surrounding sounds and movements and detecting abnormalities; A means of notifying emergency contacts if an abnormality is detected; A system including:

2. means for recording user life situation data and generating reports; a means for providing the generated reports to a third party; The system of claim 1 further comprising:

3. means for accepting a user's personality setting and generating conversation content based on the personality; The system of claim 1 further comprising:

Images