System

Abstract

A system is provided.SOLUTION: A system including an AI avatar, means for displaying a face photo of a family member or a friend of an elderly person, means for interacting with the elderly person through the AI avatar, means for maintaining cognitive function of the elderly person through interaction or a game, means for collecting activity data of the elderly person and transmitting the activity data to a server in real time, means for notifying a family member of the activity data of the elderly person, means for detecting and notifying forgetting to lock a door or an abnormality in association with a home appliance or a system of a house, and means for remotely operating the home appliance or the system of the house of the elderly person.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's aging society, the number of elderly people living alone is increasing, and loneliness and declining cognitive function are becoming serious problems. Furthermore, it is difficult for family members living far away to monitor the daily lives of elderly people, and there are concerns about security risks due to forgetting to lock their homes or operating household appliances incorrectly. The purpose of this invention is to provide a system that improves the quality of life of elderly people and allows family members to monitor them with peace of mind. [Means for solving the problem]

[0005] The present invention includes a system that displays an AI avatar and facial photos of the elderly person's family or friends, a system for interacting with the elderly person through the AI ​​avatar, a system for maintaining the elderly person's cognitive function through interactions and games, and a system for collecting the elderly person's activity data and transmitting it to a server in real time. The system also includes a system for notifying family members of the elderly person's activity data, a system for linking with home appliances and home systems to detect and notify users of forgotten locks and abnormalities, and a system for remotely controlling home appliances and home systems. This system allows the elderly person to have fun stimulating their cognitive function, while family members can remotely check their safety and survival.

[0006] An "AI avatar" is a virtual character equipped with artificial intelligence that is used to interact with and play games with the elderly.

[0007] The "face photos" are actual photos of the elderly person's family and friends, and are displayed on the screen to make the elderly feel closer to them.

[0008] "Cognitive function" refers to the mental abilities of older adults, such as memory, attention, judgment, and language comprehension.

[0009] "Activity data" is a record of the behavior and reactions of elderly people obtained through conversations with AI avatars and games.

[0010] The "server" is a central processing unit that collects and analyzes data sent from user terminals and sensor units, and sends notifications to family members.

[0011] "Notifications" are messages and alerts that inform family members of the elderly person's activity status or any abnormalities that may occur.

[0012] A "sensor unit" is a device that collects information such as house keys, electricity, temperature, and motion detection, and controls them as needed.

[0013] "Remote control" refers to the act of family members remotely controlling an elderly person's home appliances and systems via the Internet.

[0014] "Forgetting to lock" refers to elderly people forgetting to lock their homes, and the system includes a means to detect and notify them of this. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Specific embodiments for implementation are described in detail below.

[0037] System configuration

[0038] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[0039] 1. User Terminal Embodiments

[0040] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0041] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0042] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0043] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0044] 4. Data collection: The device collects data on interactions and game results and periodically sends it to the server.

[0045] 2. Server Implementation

[0046] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0047] 1. Receiving and storing data: The server receives the activity data sent from the user terminal and stores it in a database.

[0048] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a drop in activity over a certain period of time.

[0049] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[0050] 3. Sensor Unit Embodiment

[0051] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0052] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[0053] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0054] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[0055] Specific examples

[0056] Example 1: Everyday usage scenario

[0057] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[0058] Example 2: Anomaly detection and response

[0059] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. This ensures the safety of the elderly and allows the family to go about their daily lives with peace of mind.

[0060] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety.

[0061] The processing flow will be explained below.

[0062] Specific processing steps of the program

[0063] User terminal processing

[0064] Step 1:

[0065] The user starts the terminal.

[0066] Step 2:

[0067] The terminal displays a login screen and prompts the user to enter their ID and password.

[0068] Step 3:

[0069] The user enters their ID and password and clicks the login button.

[0070] Step 4:

[0071] The terminal sends the entered ID and password to the server.

[0072] Step 5:

[0073] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[0074] Step 6:

[0075] The device displays an AI avatar based on the data it receives.

[0076] Step 7:

[0077] The user selects the interaction or game.

[0078] Step 8:

[0079] The device executes the selected dialogue or game logic and interacts with the user.

[0080] Step 9:

[0081] The device collects user input and behavioral data.

[0082] Step 10:

[0083] The terminal transmits the collected data to the server at regular intervals.

[0084] Server Processing

[0085] Step 1:

[0086] The server checks the ID and password sent by the user against the database.

[0087] Step 2:

[0088] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[0089] Step 3:

[0090] The server periodically receives the user's behavior data transmitted from the terminal.

[0091] Step 4:

[0092] The server analyzes the received data and stores it in a database.

[0093] Step 5:

[0094] The server analyzes the user's activity based on the stored data.

[0095] Step 6:

[0096] If necessary, the server will send activity notifications and alerts to family members.

[0097] Sensor unit processing

[0098] Step 1:

[0099] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[0100] Step 2:

[0101] The sensor unit transmits the acquired data to the server.

[0102] Step 3:

[0103] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[0104] Step 4:

[0105] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[0106] Step 5:

[0107] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[0108] Step 6:

[0109] The sensor unit follows the received command and performs the required action (for example, locking the door).

[0110] Example 1

[0111] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Declining cognitive function, increasing loneliness, and safety issues significantly impair the quality of life of older adults. Effective and sustainable solutions to these problems are needed. However, existing technologies have not provided a system that comprehensively addresses the complex issues that older adults face in their daily lives. In particular, there is a need for a system that simultaneously maintains and improves cognitive function, ensures safety, and reduces loneliness.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0114] In this invention, the server includes a means for displaying an AI avatar and photos of the elderly person's family or friends, a means for interacting with the elderly person through the AI ​​avatar, a means for maintaining and improving the elderly person's cognitive function through interactions and games, a means for collecting the elderly person's activity data and periodically transmitting it to the server, a means for notifying family members of the elderly person's activity data, a means for linking with home appliances and housing systems to detect and notify if the elderly person has forgotten to lock the door or if there is an abnormality, and a means for remotely operating the elderly person's home appliances and housing systems. This makes it possible to maintain and improve the elderly person's cognitive function, reduce loneliness, and ensure their safety.

[0115] An "AI avatar" is a virtual character equipped with artificial intelligence, designed to stimulate cognitive function and reduce feelings of loneliness through dialogue and communication with the elderly.

[0116] The "photos" are facial images of the elderly person's family or friends, intended to give the elderly a sense of familiarity and security.

[0117] The "dialogue function" is a function that allows an AI avatar to have everyday conversations with elderly people and maintain and improve their cognitive function.

[0118] The "game function" allows seniors to participate in activities such as simple puzzles and memory games, with the aim of further maintaining and improving cognitive function.

[0119] "Activity data" refers to information such as the elderly person's daily behavior, conversations, and game results, and is analyzed to detect abnormalities.

[0120] The "server" is a computer system that functions as the central part of the system, centrally managing elderly people's activity data and conducting analysis and notifications.

[0121] "Data collection" is the process by which user terminals and sensor units gather information about the elderly person's daily activities and use it for later analysis.

[0122] The "notification function" is a function that reports activity data and notifies abnormalities to the elderly person's family members, and sends alerts using email, SMS, push notifications, etc.

[0123] "Home devices" refers to electrical appliances and equipment used in the homes of seniors that can be remotely controlled and monitored.

[0124] "Housing systems" refer to facilities that provide basic functions such as locks, lighting, and temperature control for elderly people's homes, and are monitored and controlled through sensors.

[0125] A "sensor unit" refers to an individual sensor device that is used to continuously monitor the status of a home appliance or home system.

[0126] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function and improve safety of the elderly through the management of AI avatars and home appliances. Specific embodiments for carrying out the invention are described in detail below.

[0127] System configuration

[0128] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[0129] 1. User Terminal Embodiments

[0130] The user devices consist of smartphones and tablets used by the elderly. The devices have the following functions:

[0131] 1. Displaying AI avatars

[0132] The device launches a built-in app and displays an AI avatar on the screen, using photos of the elderly person's family and friends to create a sense of familiarity.

[0133] 2. Interactive features

[0134] The device engages in everyday conversations with the elderly through an AI avatar, generating dialogue using a natural language processing engine (e.g., GPT-4) to reduce feelings of loneliness and stimulate cognitive function.

[0135] 3. Game Features

[0136] The device provides activities such as simple puzzles and memory games, and is implemented using Unity and React Native to maintain and improve cognitive function in the elderly.

[0137] 4. Data Collection

[0138] The device collects activity data such as conversation results and game results, and periodically sends it to the server using the HTTP protocol.

[0139] 2. Server Implementation

[0140] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0141] 1. Data Receipt and Storage

[0142] The server receives activity data sent from the user's device and stores it in a database (e.g., MySQL, MongoDB).

[0143] 2. Data Analysis

[0144] The server analyzes the collected data using machine learning algorithms (e.g., Scikit-learn, TensorFlow), and if an anomaly is detected, it sends a notification to the family.

[0145] 3. Notification function

[0146] The server periodically reports the elderly person's activities to the family, and in case of an emergency, it sends an immediate notification via email, SMS, or push notification (such as Firebase Cloud Messaging).

[0147] 3. Sensor Unit Embodiment

[0148] The sensor unit integrates various sensors in the home to monitor the status of home appliances and residential systems, and can take action if necessary.

[0149] 1. Monitoring function

[0150] Monitor locks, lights, temperature, and activity of home appliances and systems. Collect information using IoT devices (e.g., Arduino, Raspberry Pi).

[0151] 2. Data transmission

[0152] The monitoring results are sent to the server periodically using MQTT as the communication protocol.

[0153] 3. Control Function

[0154] According to instructions from the server, it remotely controls household devices and locks / unlocks doors. Instructions given by family members using their smartphones are transmitted to each sensor via the server.

[0155] Specific examples

[0156] Example 1: Everyday usage scenario

[0157] Every morning, the user (elderly person) turns on their smartphone, and an AI avatar holding a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person replies, "I'm feeling fine today." While continuing the normal dialogue, the avatar suggests, "Would you like to play a little memory game?" The user plays the game, and the results are recorded on the device and periodically sent to the server. The server analyzes the results and notifies the family of "today's activity status."

[0158] Example 2: Anomaly detection and response

[0159] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family, and when the family member issues a command to "lock" the door from their smartphone, the command is transmitted via the server to the sensor unit, which then locks the door. This ensures the safety of the elderly, allowing the family to go about their daily lives with peace of mind.

[0160] Prompt Sentence Examples

[0161] Prompt 1: Use a user terminal

[0162] "Please tell us the process steps and specific actions of how an elderly person would turn on their smartphone every morning and interact with the AI ​​avatar displayed on the screen."

[0163] Prompt 2: Anomaly detection

[0164] "Please explain in detail the specific steps and technology used by the sensor unit to notify family members if their home is left unlocked."

[0165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0166] Step 1: Start the user device and display the AI ​​avatar

[0167] 1. The user starts up the smartphone

[0168] Input: User powers on the smartphone

[0169] Operation: Initialize the smartphone and transition to the home screen

[0170] Output: Home screen display

[0171] 2. Launch the AI ​​avatar app

[0172] Input: Tap the AI ​​Avatar app icon on the home screen

[0173] What it does: The app launches and displays an avatar showing a photo of the senior's family or friend.

[0174] Output: Display of AI avatar

[0175] Step 2: Execute the interaction

[0176] 1. Initial interaction with an AI avatar

[0177] Input: AI avatar app launch complete

[0178] Action: AI avatar says "Good morning, how are you feeling today?"

[0179] Output: Questions for the elderly

[0180] 2. Receiving and analyzing elderly people's responses

[0181] Input: Elderly person responds "I'm fine today"

[0182] How it works: The device uses a speech recognition feature (such as the Google Speech-to-Text API) to convert the voice data into text, and then uses a generative AI model (such as GPT-4) to analyze the response.

[0183] Output: Generate appropriate answers based on the analysis results

[0184] Step 3: Providing game functionality

[0185] 1. Game suggestions

[0186] Input:Continue the dialogue

[0187] How it works: The AI ​​avatar suggests, "Would you like to play a little memory game?"

[0188] Output: Game suggestions for seniors

[0189] 2. Running the game

[0190] Input: Seniors start the game

[0191] Action: The device displays a memory game using Unity and React Native and waits for user interaction.

[0192] Output: Displaying the game interface and collecting user operation data

[0193] Step 4: Collect data and send it to the server

[0194] 1. Data collection

[0195] Input: Interaction results and game results

[0196] What it does: The device temporarily stores the collected data in its internal storage.

[0197] Output: Temporarily saved activity data

[0198] 2. Data transmission

[0199] Input: Temporarily saved activity data

[0200] What happens: The device sends data to the server using the HTTP protocol.

[0201] Output: Activity data sent to the server

[0202] Step 5: Receiving and analyzing data on the server

[0203] 1. Receiving Data

[0204] Input: Activity data sent from the terminal

[0205] What happens: The server receives an HTTP request and retrieves the data.

[0206] Output: Received activity data

[0207] 2. Data storage

[0208] Input: Received activity data

[0209] What happens: The server saves the data to a database (MySQL or MongoDB).

[0210] Output: Activity data stored in the database

[0211] 3. Data Analysis

[0212] Input: Activity data stored in the database

[0213] How it works: The server analyzes the data using machine learning algorithms (Scikit-learn or TensorFlow) and detects anomalies.

[0214] Output: Analysis results and anomaly detection information

[0215] Step 6: Notify family members

[0216] 1. Generate a notification

[0217] Input: Analysis results and anomaly detection information

[0218] Action: The server generates a notification message for the family.

[0219] Output: Notification message

[0220] 2. Sending notifications

[0221] Input: Notification message

[0222] How it works: The server sends notifications via email, SMS, or push notifications (Firebase Cloud Messaging).

[0223] Output: Notification sent to family members

[0224] Step 7: Sensor-by-sensor monitoring and control

[0225] 1. Performing monitoring functions

[0226] Input: Status of home appliances and systems

[0227] Operation: Sensor units constantly monitor locks, lighting, temperature, and operation. Data is acquired using IoT devices (Arduino and Raspberry Pi).

[0228] Output: Monitoring results

[0229] 2. Data Transmission

[0230] Input: Monitoring results

[0231] Operation: The sensor unit sends the monitoring results to the server using the MQTT protocol.

[0232] Output: Monitoring data sent to the server

[0233] 3. Receiving control instructions

[0234] Input: Control instructions from family members or the server

[0235] Operation: The server sends control instructions to each sensor, and the sensor operates the home appliance.

[0236] Output: Operation results of home appliances

[0237] (Application example 1)

[0238] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0239] In addition to systems that improve the quality of life for the elderly, there is a need for means to manage worker safety. In particular, in work environments such as factories, it is necessary to ensure safety by timely monitoring the physical condition of workers and the surrounding environment. In conventional systems, the elderly management system and the worker safety management system exist independently and require separate management, making efficient integrated management difficult. Furthermore, if real-time data collection and analysis results are not immediately notified, there is a risk of delays in appropriate response. The present invention aims to solve these problems and provide an integrated system that simultaneously improves the safety and quality of life of both the elderly and workers.

[0240] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0241] In this invention, the server includes a means for collecting elderly people's activity data and transmitting it to the server in real time, a means for collecting sensor data and transmitting it to the server, and a means for analyzing the collected data and notifying the safety manager. This improves the quality of life of elderly people and makes it possible to constantly monitor the physical condition and environmental conditions of workers and ensure their safety.

[0242] An "AI avatar" is a virtual character that uses artificial intelligence technology and is a means of interacting with the elderly or workers.

[0243] "Photos of family or friends" are images used to display AI avatars to increase familiarity with the elderly.

[0244] "Means for dialogue" refers to a mechanism that allows users and AI avatars to communicate through voice or text.

[0245] "Methods for maintaining cognitive function" are methods of providing mental stimulation through dialogue, games, etc. to maintain the cognitive abilities of elderly people.

[0246] "Means for collecting activity data and transmitting it to a server in real time" refers to technology that continuously collects activity information on elderly people and workers and instantly transmits that information to a cloud server.

[0247] The "means of notifying family members of activity data" refers to a method for informing family members of the elderly person's activity status.

[0248] "Means for detecting and notifying of forgotten locks and abnormalities" refers to technology that uses sensors in the home or work environment to monitor the status of keys and abnormalities, and issues alerts as necessary.

[0249] "Means for remotely controlling home appliances and home systems" refers to technology for remotely controlling home appliances and home systems via the Internet.

[0250] "Means for monitoring physical condition and environmental conditions and issuing warnings if abnormalities are detected" refers to a method in which sensors are used to detect abnormalities in workers' health and working environment and send appropriate alerts.

[0251] "Means for analyzing collected data and notifying safety managers" refers to methods for analyzing data obtained from sensors and reporting important information to safety managers in real time.

[0252] This invention is a system that comprehensively improves the quality of life for the elderly and manages worker safety. The system is composed of an AI avatar, a user terminal, a server, a sensor unit, and a communication network. Specific embodiments are described below.

[0253] System configuration

[0254] 1. User Terminal Embodiments

[0255] The user device can be, for example, a smartphone or tablet used by the elderly, or smart glasses used by workers. The device has the following functions:

[0256] Displaying AI avatars: The device displays AI avatars on the screen. The AI ​​avatars are set with photos of the elderly's family and friends, making them feel closer to the elderly. They can also be used to assist workers.

[0257] Conversational function: The device will engage in everyday conversations with the user through an AI avatar, thereby reducing feelings of loneliness among the elderly and stimulating their cognitive functions. It will also provide work instructions to workers.

[0258] Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0259] Data collection: The terminal collects the results of conversations and games, as well as the worker's physical condition and work data, and periodically transmits them to the server.

[0260] 2. Server Implementation

[0261] The server is the center of the system, centrally managing user activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0262] Data reception and storage: The server receives the activity data sent from the user terminals and sensor units and stores it in a database.

[0263] Data Analysis: The server analyzes the collected data and sends notifications to family members or safety managers if anomalies are detected, such as a drop in activity over a certain period of time.

[0264] Notification function: The server periodically reports user activity to family members and safety managers. In the event of an emergency, an alert is sent to prompt immediate action.

[0265] 3. Sensor Unit Embodiment

[0266] The sensor unit integrates various sensors in the home or work environment to monitor their status and take action if necessary.

[0267] Monitoring function: The sensor unit monitors temperature, movement, lock status, etc. within the home or work environment.

[0268] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0269] Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking.

[0270] Specific examples

[0271] Example 1: Daily usage scenario for elderly people

[0272] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[0273] Example 2: Worker Safety Management Scenario

[0274] A worker puts on the smart glasses and begins work. An AI avatar speaks to the worker through the glasses, asking, "What's the next step?" If the worker replies, "Assembly," the avatar displays the correct work procedure. The sensor unit monitors the temperature of the work environment and the worker's physical condition, and immediately displays a warning if there is an abnormality. The data is sent to a server and analyzed as appropriate. Based on the analysis results, a notification is sent to the safety manager.

[0275] A prompt sentence from a generative AI model as an example of use

[0276] Prompt statement:

[0277] "We are considering a voice assistant that will provide safety warnings while working in a factory. Please provide example dialogue."

[0278] As described above, this system not only improves the quality of life for the elderly, but also ensures the safety of working people, providing an integrated solution that meets the needs of both.

[0279] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0280] Step 1:

[0281] The device collects the user's speech and converts it into text data using a speech recognition function. The input for this process is the user's speech, and the output is text data.

[0282] Step 2:

[0283] The device sends the collected text data to the AI ​​avatar. The AI ​​avatar generates appropriate dialogue content based on this text data. It uses a generative AI model to generate dialogue based on the prompt text. The input is text data and the output is dialogue content. Specifically, when the user says, "Tell me the next step," the AI ​​avatar uses the generative AI model to answer, "Yes, the next step is assembling the parts."

[0284] Step 3:

[0285] The device outputs the generated dialogue content to the user as voice or text. The input for this process is the dialogue content data received from the AI ​​avatar, and the output is the dialogue result presented to the user. Specifically, the text generated using speech synthesis technology is played aloud or displayed on the smart glasses display.

[0286] Step 4:

[0287] The sensor unit continuously collects environmental data and user health data. This includes various sensors such as temperature sensors and heart rate monitors. The input of this process is the data obtained from the various sensors, and the output is the collected environmental data and health data.

[0288] Step 5:

[0289] The sensor unit sends the collected data to a server. The server stores the received data in a database and analyzes it in real time. The input to this process is the data sent from the sensor unit, and the output is the stored data and the analysis results. For example, if the temperature exceeds the allowable range within a certain period of time, this information is recorded.

[0290] Step 6:

[0291] The server sends a notification if an abnormality is detected based on the analysis results. For example, if the temperature in the work environment exceeds a set standard, the server will send an alert to the safety manager. The input to this process is the analysis result data, and the output is the notification data. Specifically, the alert is sent via email or an in-application notification function.

[0292] Step 7:

[0293] Users (workers and elderly people) receive notifications and alerts and take necessary measures. The input to this process is notification data from the server, and the output is the corresponding action. For example, if an alert about an abnormal temperature is received, the user will temporarily suspend their work and take appropriate measures.

[0294] The above process will create a system that comprehensively improves the quality of life for the elderly and manages worker safety.

[0295] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0296] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[0297] System configuration

[0298] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[0299] 1. User Terminal Embodiments

[0300] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0301] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0302] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0303] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0304] 4. Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[0305] 5. Data collection: The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically sends them to the server.

[0306] 2. Server Implementation

[0307] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0308] 1. Receiving and storing data: The server receives the activity data and emotion data sent from the user terminal and stores it in a database.

[0309] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[0310] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[0311] 3. Sensor Unit Embodiment

[0312] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0313] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[0314] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0315] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[0316] Specific examples

[0317] Example 1: Everyday usage scenario

[0318] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[0319] Example 2: Anomaly detection and response

[0320] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[0321] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[0322] The processing flow will be explained below.

[0323] Specific processing steps of the program

[0324] User terminal processing

[0325] Step 1:

[0326] The user starts the terminal.

[0327] Step 2:

[0328] The terminal displays a login screen and prompts the user to enter their ID and password.

[0329] Step 3:

[0330] The user enters their ID and password and clicks the login button.

[0331] Step 4:

[0332] The terminal sends the entered ID and password to the server.

[0333] Step 5:

[0334] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[0335] Step 6:

[0336] The device displays an AI avatar based on the data it receives.

[0337] Step 7:

[0338] The user selects the interaction or game.

[0339] Step 8:

[0340] The device executes the selected dialogue or game logic and interacts with the user, capturing the user's voice and facial expressions with a camera.

[0341] Step 9:

[0342] The device transmits the acquired voice and facial expression data to an emotion engine to analyze the user's emotional state.

[0343] Step 10:

[0344] The emotion engine returns the analysis results to the device, and the device dynamically changes the dialogue and game content based on these results.

[0345] Step 11:

[0346] The terminal collects user input data, behavioral data, and emotional data.

[0347] Step 12:

[0348] The terminal transmits the collected data to the server at regular intervals.

[0349] Server Processing

[0350] Step 1:

[0351] The server checks the ID and password sent by the user against the database.

[0352] Step 2:

[0353] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[0354] Step 3:

[0355] The server periodically receives the user's behavioral data and emotion data transmitted from the terminal.

[0356] Step 4:

[0357] The server analyzes the received data and stores it in a database.

[0358] Step 5:

[0359] The server analyzes the user's activity and emotional state based on the stored data.

[0360] Step 6:

[0361] If desired, the server will send notifications and alerts to family members about their activity and emotional state.

[0362] Sensor unit processing

[0363] Step 1:

[0364] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[0365] Step 2:

[0366] The sensor unit transmits the acquired data to the server.

[0367] Step 3:

[0368] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[0369] Step 4:

[0370] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[0371] Step 5:

[0372] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[0373] Step 6:

[0374] The sensor unit follows the received command and performs the required action (for example, locking the door).

[0375] Specific examples

[0376] Example 1: Everyday usage scenario

[0377] Step 1:

[0378] Every morning, the user turns on the tablet and an AI avatar with a photo of their grandchild appears on the screen.

[0379] Step 2:

[0380] The AI ​​avatar speaks, "Good morning, how are you feeling today?"

[0381] Step 3:

[0382] The user responds, "I'm feeling good today." At this time, the device records the user's voice and analyzes their emotional state using an emotion engine.

[0383] Step 4:

[0384] While engaging in normal dialogue, the avatar suggests a simple memory game.

[0385] Step 5:

[0386] The user plays the game and the results and emotional state are recorded on the terminal.

[0387] Step 6:

[0388] The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[0389] Example 2: Anomaly detection and response

[0390] Step 1:

[0391] The sensor unit detects when the house is unlocked and sends this information to a server.

[0392] Step 2:

[0393] The server immediately notifies the family.

[0394] Step 3:

[0395] When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, and the door is locked.

[0396] Step 4:

[0397] If the emotion engine detects anxiety in the elderly person, a notification will be sent to the family member stating that the elderly person is feeling anxious.

[0398] Example 2

[0399] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0400] There is a need to maintain the cognitive functions of the elderly, reduce feelings of loneliness, and ensure safety within the home. However, conventional systems have had difficulty adequately meeting these complex needs. In addition, there is a lack of means to properly analyze the emotional state and promptly notify family members. A new system is needed to solve these issues.

[0401] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0402] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly person to grasp their emotional state, a means for collecting activity data and emotional data of the elderly person and periodically transmitting it to the server, and a means for the server to analyze the received data and send a notification to family members if an abnormality is detected. This makes it possible to maintain the elderly person's cognitive function, reduce loneliness, and ensure safety within the home.

[0403] 1. "AI Avatar" is a virtual character used to interact with seniors, representing photographs of the user's family and friends.

[0404] 2. "Means for conducting dialogue" refers to technical devices and software that use AI avatars to engage in natural language dialogue with older people.

[0405] 3. "Cognitive maintenance strategies" are technologies that provide games and activities designed to stimulate and maintain cognitive abilities in older adults.

[0406] 4. "Means for understanding emotional states" refers to analytical technologies and devices that analyze the voices and facial expressions of elderly people to understand their emotions.

[0407] 5. "Means for collecting activity data and emotional data and periodically transmitting it to a server" refers to technology that uses terminals or sensor units to record the daily activities and emotional states of elderly people and transmits them to a server via a network.

[0408] 6. "The server analyzes the received data and notifies the family if an abnormality is detected" refers to a technology that analyzes data on the server side and immediately notifies the family if an abnormality is detected.

[0409] 7. "Means for linking with home appliances and home systems to detect and notify when locks are left locked or other abnormalities" refers to technology that uses sensor units in the home to monitor the status of home appliances and the home, and notifies if an abnormality is detected.

[0410] 8. "Means for remotely controlling home appliances and home systems" refers to technology that remotely operates and controls home appliances and various home facilities based on instructions from a server.

[0411] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[0412] System configuration

[0413] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[0414] 1. User Terminal Embodiments

[0415] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0416] 1. Displaying AI avatars:

[0417] The device displays an AI avatar on the screen, which is set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0418] 2. Interactive features:

[0419] The device uses natural language processing models (e.g., GPT-4) to engage in everyday conversations with the elderly, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0420] 3. Game Features:

[0421] The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0422] 4. Emotion recognition function:

[0423] The device analyzes the elderly person's voice and facial expressions and uses an emotion engine (e.g., Emotion API) to understand their emotional state.

[0424] 5. Data Collection:

[0425] The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically transmits them to the server.

[0426] 2. Server Implementation

[0427] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0428] 1. Data Receipt and Storage:

[0429] The server receives the activity data and emotion data sent from the user terminal and stores them in a database (e.g., MySQL).

[0430] 2. Data Analysis:

[0431] The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[0432] 3. Notification function:

[0433] The server periodically reports the elderly person's activities to the family, and in the event of an emergency, it sends an alert to prompt immediate action.

[0434] 3. Sensor Unit Embodiment

[0435] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0436] 1. Monitoring function:

[0437] The sensor unit monitors the house's locks, electricity, temperature, motion detection, and more.

[0438] 2. Data transmission:

[0439] The sensor unit periodically transmits the monitoring results to the server.

[0440] 3. Control function:

[0441] It receives instructions from the server and performs remote control of home appliances and locks / unlocks doors.

[0442] Specific examples

[0443] Example 1: Everyday usage scenario

[0444] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[0445] Example prompt sentence:

[0446] A user turns on the tablet, and an AI avatar with a photo of their grandchild speaks to them, saying, "Good morning, how are you feeling today?" When the user replies, "I'm feeling fine today," explain the system that performs voice analysis, records their emotional state, and sends the result to a server.

[0447] Example 2: Anomaly detection and response

[0448] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[0449] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[0450] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0451] Step 1: Start the user device

[0452] What it does: A user presses the power button on the tablet to start up the device.

[0453] Input: User presses and holds the power button.

[0454] Output: The device's home screen is displayed.

[0455] Specific behavior: The user presses and holds the tablet's power button, the boot screen appears, and then the home screen appears.

[0456] Step 2: Displaying the AI ​​Avatar

[0457] How it works: The device automatically launches the AI ​​avatar app from the home screen and displays an AI avatar on the screen, set to a photo of the elderly person's family or friends.

[0458] Enter: the app icon on your home screen.

[0459] Output: An AI avatar is displayed.

[0460] What it does: The device automatically launches the application and displays an AI avatar with a photo of your grandchild.

[0461] Step 3: Initiating everyday conversations and recognizing emotions

[0462] How it works: The device uses a natural language processing model (e.g., GPT-4) to have an AI avatar say, "Good morning, how are you feeling today?" The device then performs a voice analysis of the user's response and uses emotion recognition technology to determine their emotional state.

[0463] Input: A prompt to the generative AI model: "Good morning, how are you feeling today?" and the user's response.

[0464] Output: Emotional state (e.g., "Energetic").

[0465] Specific operation: The AI ​​avatar asks, "Good morning, how are you feeling today?", and the elderly person replies, "I'm feeling fine today." The device records the answer and uses its emotion engine to interpret it as "feeling fine."

[0466] Step 4: Propose and play the game

[0467] How it works: The device uses an AI avatar to ask the elderly, "Would you like to play a memory game today?" If the elderly agrees, the device starts the memory game.

[0468] Input: The game suggestion prompt, "Would you like to play a memory game today?" and the user's response.

[0469] Output: Game result.

[0470] Specific operation: The AI ​​avatar suggests, "Would you like to play a memory game today?" If the elderly person responds, "Sure," a simple memory game will appear on the screen.

[0471] Step 5: Collect and send data

[0472] How it works: The device collects data such as conversation content, game results, and emotion recognition results, and sends the collected data to a server via Wi-Fi.

[0473] Input: Dialogue recording, game results, emotion recognition results.

[0474] Output: Sending data to the server.

[0475] Specific operation: The device stores the conversation records, game results, and emotional state as data, and transmits them to the server via Wi-Fi.

[0476] Step 6: Server receives and stores data

[0477] Operation: The server receives data sent from the device and stores it in a database (e.g., MySQL).

[0478] Input: Data sent from the terminal.

[0479] Output: Save to database.

[0480] What happens: The server stores all received data as entries in a MySQL database.

[0481] Step 7: Data analysis and notification by the server

[0482] What it does: The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected.

[0483] Input: Record data from the database.

[0484] Output: Notification to family.

[0485] Specific operation: The server analyzes the data, and when it detects a decrease in activity, it sends a notification email to the family saying, "The elderly person's activity is decreasing."

[0486] Step 8: Anomaly detection and response

[0487] How it works: The sensor unit detects that the house doors are unlocked and sends this information to the server. The server immediately notifies the family, who then issue a command to "lock" the door via their smartphone. The sensor unit then remotely locks the door based on the received command.

[0488] Input: Anomaly detection data from the sensor unit, instructions from family members.

[0489] Output: Locks the door and notifies family members of the anxiety state.

[0490] Specific operation: The sensor detects that the door to the house is open and sends the information to the server. When a family member presses the "lock" button on their smartphone, the server sends a command to the sensor unit, which then locks the front door. If the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious."

[0491] These are the specific steps in the program processing of this system, which will help maintain cognitive function, reduce loneliness, and improve safety for the elderly.

[0492] (Application example 2)

[0493] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0494] In order to improve the quality of life of the elderly, it is important to maintain cognitive function and ensure their safety. However, because the time that family members can directly interact with the elderly is limited, a system that can remotely grasp the elderly's situation and respond appropriately is required. It is also important to analyze the elderly's emotional state and respond accordingly. Given this background, it is necessary to provide a system that can monitor the safety and emotional state of the elderly in real time and respond quickly to abnormalities or changes in emotion.

[0495] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0496] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly to grasp their emotional state, a means for monitoring the elderly's environment with a camera in the smart glasses, and a means for sending an alert to relatives when an abnormality is detected. This not only makes it possible to maintain the elderly's cognitive function and ensure their safety, but also enables immediate response to changes in their emotional state.

[0497] An "AI avatar" is a digital character equipped with artificial intelligence to interact with the elderly and display a photo of a relative or acquaintance.

[0498] "Means for displaying photographs of relatives or acquaintances" refers to technology that displays photographs of family members or friends on a digital display to help elderly people feel closer to them.

[0499] "Means for responding" is a function that uses recognition technology and voice synthesis technology to enable elderly people and AI avatars to communicate through natural conversation.

[0500] "Means for maintaining cognitive function in the elderly through entertainment" is a technology that aims to maintain and improve cognitive function in the elderly by providing activities such as simple games and puzzles.

[0501] "Means for collecting activity data and transmitting it to a central processing unit in real time" refers to a function that uses sensors and devices to record the elderly person's daily activities, voice, facial expressions, etc., and transmits the data to a server in real time.

[0502] The "means of notifying relatives of activity data" is a function that automatically analyzes collected data on elderly people and notifies relatives of abnormalities and necessary information.

[0503] The "means for detecting and notifying when someone forgets to lock their home or when there is an abnormality" is a technology that constantly monitors the elderly person's home appliances and home systems, and notifies relatives when it detects that someone has forgotten to lock their home or when there is an abnormality.

[0504] "Means for remotely controlling home appliances and home systems" refers to technology that allows relatives to remotely control home appliances and systems in the elderly person's home via devices such as smartphones.

[0505] "Means for monitoring the environment of the elderly using a camera in smart glasses" is a function that uses a camera installed in smart glasses to monitor the environment around the elderly in real time.

[0506] "Means for analyzing voice and facial expressions to grasp emotional states" refers to technology that analyzes voice and facial expression data of elderly people and uses an emotion engine to estimate and grasp their emotional states.

[0507] The "means for sending an alert to relatives when an abnormality is detected" is a system that sends an alert message to relatives when an abnormality is detected as a result of various monitoring performed on the elderly person.

[0508] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. In particular, this system has the ability to analyze the emotional state of the elderly using an emotion engine and utilize the analysis data. Specific embodiments are described below.

[0509] System configuration

[0510] The embodiment of the present invention is broadly composed of the following components.

[0511] 1. User Terminal Embodiments

[0512] The user devices are smartphones or smart glasses used by elderly people. The devices have the following main functions:

[0513] Display of AI avatar: The device displays an AI avatar on the display, which is set to a photo of the elderly person's relatives or acquaintances, giving the user a sense of familiarity.

[0514] Voice response function: The device responds naturally to the elderly through an AI avatar, helping to reduce feelings of loneliness and stimulate cognitive function.

[0515] Activity provision function: The device provides simple activities such as puzzles and games, which further maintain and improve cognitive function in the elderly.

[0516] Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[0517] Data collection and transmission: Responses, game results, and emotion recognition results are collected as data and periodically sent to the server.

[0518] 2. Server Implementation

[0519] The server is the center of the system, centrally managing elderly people's activity data and providing necessary notifications and analysis.

[0520] Data reception and storage: The server receives the activity data and emotion data sent from the user terminal and stores them in a database.

[0521] Data analysis: The server analyzes the collected data and sends notifications to relatives if any abnormalities are detected.

[0522] Notification function: The server periodically reports the elderly person's activity and emotional state to relatives, and sends immediate alerts in case of an emergency.

[0523] 3. Sensor Unit Embodiment

[0524] The sensor unit integrates various sensors in the home and monitors the status of home appliances and systems.

[0525] Monitoring functions: Sensors monitor locks, electricity, temperature, motion, and more within the home.

[0526] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0527] Control function: Receives instructions from the server and remotely controls and locks / unlocks home appliances.

[0528] Specific examples

[0529] Example 1: Everyday usage scenario

[0530] The user (elderly person) turns on the smart glasses every morning, and an AI avatar with a photo of a relative's face appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While making a normal response, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the relative of "today's activity status and emotional state."

[0531] Example 2: Anomaly detection and response

[0532] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies relatives. When a relative issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. If the emotion engine detects anxiety in the elderly person at this time, it also sends a notification to the relative that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also enables responses that take their emotional state into consideration.

[0533] Prompt Sentence Examples

[0534] "You will develop a function to detect falls among elderly people based on image data of high-risk abnormal behavior and falls, and image data of normal behavior. You will also create a system to analyze the emotional state of elderly people from images of their facial expressions, and collect and store that data."

[0535] In this way, the present invention provides a system that enables the maintenance of cognitive function and safety of elderly people, and instantly grasps their emotional state using an emotion engine.

[0536] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0537] Step 1:

[0538] The user turns on the smart glasses

[0539] Input: User action of pressing the power button on the smart glasses.

[0540] Operation: The smart glasses start up and each module of the system is initialized.

[0541] Output: The camera and microphone on the smart glasses start working, and the AI ​​avatar appears on the display.

[0542] Step 2:

[0543] The device captures the voice and facial expressions of the elderly person.

[0544] Input: Video data from the camera and audio data from the microphone of the smart glasses

[0545] How it works: The camera recognizes the elderly person's face and the microphone records their voice. This data is temporarily stored on the device.

[0546] Output: Video frames and audio clips of the elderly

[0547] Step 3:

[0548] The device analyzes your emotional state

[0549] Input: Captured video frames and audio clips

[0550] How it works: The device inputs this data into an emotion engine, which performs facial expression analysis and voice analysis. It then uses an emotion recognition model (e.g., a generative AI model using TensorFlow) to estimate the user's emotional state (e.g., happiness, sadness, anxiety, etc.).

[0551] Output: Estimated emotional state

[0552] Step 4:

[0553] The device responds to the elderly person through an AI avatar.

[0554] Input: The user's previous voice response and detected emotional state

[0555] How it works: The AI ​​avatar generates dialogue that responds to the user with the following responses, including the content of the response and speech synthesis.

[0556] Output: The following dialogue and synthesized speech

[0557] Step 5:

[0558] The device sends activity data to the server

[0559] Input: Responses, emotional state analysis results, elderly person activity data (game results, etc.)

[0560] How it works: The device packages the data and sends it over the internet to a server

[0561] Output: Elderly activity data sent to the server

[0562] Step 6:

[0563] The server receives and analyzes the data

[0564] Input: Elderly person's activity data sent from the device

[0565] How it works: The server stores the received data in a database and analyzes it for any abnormalities. If an abnormality is detected, an alert is sent to the relatives.

[0566] Output: Today's activity report or anomaly alert

[0567] Step 7:

[0568] The server monitors the data from the sensor units.

[0569] Input: Environmental data sent from sensor units in the home (temperature, lock status, etc.)

[0570] Operation: The server periodically receives this data and detects abnormal conditions.

[0571] Output: Alert notification when an abnormality occurs, log update when the status is normal

[0572] Step 8:

[0573] A relative remotely controls the device

[0574] Input: Operation command from smartphone (e.g., command to lock the door)

[0575] How it works: The server receives the command and sends appropriate control signals to the sensor units in the home.

[0576] Output: Home appliances and systems operate according to the relatives' instructions.

[0577] The above processing steps realize a system that monitors the living environment and emotional state of the elderly in real time, promptly notifies relatives when an abnormality occurs, and enables remote control.

[0578] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0580] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0589] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0594] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Specific embodiments for implementation are described in detail below.

[0595] System configuration

[0596] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[0597] 1. User Terminal Embodiments

[0598] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0599] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0600] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0601] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0602] 4. Data collection: The device collects data on interactions and game results and periodically sends it to the server.

[0603] 2. Server Implementation

[0604] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0605] 1. Receiving and storing data: The server receives the activity data sent from the user terminal and stores it in a database.

[0606] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a drop in activity over a certain period of time.

[0607] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[0608] 3. Sensor Unit Embodiment

[0609] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0610] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[0611] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0612] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[0613] Specific examples

[0614] Example 1: Everyday usage scenario

[0615] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[0616] Example 2: Anomaly detection and response

[0617] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. This ensures the safety of the elderly and allows the family to go about their daily lives with peace of mind.

[0618] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety.

[0619] The processing flow will be explained below.

[0620] Specific processing steps of the program

[0621] User terminal processing

[0622] Step 1:

[0623] The user starts the terminal.

[0624] Step 2:

[0625] The terminal displays a login screen and prompts the user to enter their ID and password.

[0626] Step 3:

[0627] The user enters their ID and password and clicks the login button.

[0628] Step 4:

[0629] The terminal sends the entered ID and password to the server.

[0630] Step 5:

[0631] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[0632] Step 6:

[0633] The device displays an AI avatar based on the data it receives.

[0634] Step 7:

[0635] The user selects the interaction or game.

[0636] Step 8:

[0637] The device executes the selected dialogue or game logic and interacts with the user.

[0638] Step 9:

[0639] The device collects user input and behavioral data.

[0640] Step 10:

[0641] The terminal transmits the collected data to the server at regular intervals.

[0642] Server Processing

[0643] Step 1:

[0644] The server checks the ID and password sent by the user against the database.

[0645] Step 2:

[0646] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[0647] Step 3:

[0648] The server periodically receives the user's behavior data transmitted from the terminal.

[0649] Step 4:

[0650] The server analyzes the received data and stores it in a database.

[0651] Step 5:

[0652] The server analyzes the user's activity based on the stored data.

[0653] Step 6:

[0654] If necessary, the server will send activity notifications and alerts to family members.

[0655] Sensor unit processing

[0656] Step 1:

[0657] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[0658] Step 2:

[0659] The sensor unit transmits the acquired data to the server.

[0660] Step 3:

[0661] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[0662] Step 4:

[0663] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[0664] Step 5:

[0665] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[0666] Step 6:

[0667] The sensor unit follows the received command and performs the required action (for example, locking the door).

[0668] Example 1

[0669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0670] Declining cognitive function, increasing loneliness, and safety issues significantly impair the quality of life of older adults. Effective and sustainable solutions to these problems are needed. However, existing technologies have not provided a system that comprehensively addresses the complex issues that older adults face in their daily lives. In particular, there is a need for a system that simultaneously maintains and improves cognitive function, ensures safety, and reduces loneliness.

[0671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0672] In this invention, the server includes a means for displaying an AI avatar and photos of the elderly person's family or friends, a means for interacting with the elderly person through the AI ​​avatar, a means for maintaining and improving the elderly person's cognitive function through interactions and games, a means for collecting the elderly person's activity data and periodically transmitting it to the server, a means for notifying family members of the elderly person's activity data, a means for linking with home appliances and housing systems to detect and notify if the elderly person has forgotten to lock the door or if there is an abnormality, and a means for remotely operating the elderly person's home appliances and housing systems. This makes it possible to maintain and improve the elderly person's cognitive function, reduce loneliness, and ensure their safety.

[0673] An "AI avatar" is a virtual character equipped with artificial intelligence, designed to stimulate cognitive function and reduce feelings of loneliness through dialogue and communication with the elderly.

[0674] The "photos" are facial images of the elderly person's family or friends, intended to give the elderly a sense of familiarity and security.

[0675] The "dialogue function" is a function that allows an AI avatar to have everyday conversations with elderly people and maintain and improve their cognitive function.

[0676] The "game function" allows seniors to participate in activities such as simple puzzles and memory games, with the aim of further maintaining and improving cognitive function.

[0677] "Activity data" refers to information such as the elderly person's daily behavior, conversations, and game results, and is analyzed to detect abnormalities.

[0678] The "server" is a computer system that functions as the central part of the system, centrally managing elderly people's activity data and conducting analysis and notifications.

[0679] "Data collection" is the process by which user terminals and sensor units gather information about the elderly person's daily activities and use it for later analysis.

[0680] The "notification function" is a function that reports activity data and notifies abnormalities to the elderly person's family members, and sends alerts using email, SMS, push notifications, etc.

[0681] "Home devices" refers to electrical appliances and equipment used in the homes of seniors that can be remotely controlled and monitored.

[0682] "Housing systems" refer to facilities that provide basic functions such as locks, lighting, and temperature control for elderly people's homes, and are monitored and controlled through sensors.

[0683] A "sensor unit" refers to an individual sensor device that is used to continuously monitor the status of a home appliance or home system.

[0684] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function and improve safety of the elderly through the management of AI avatars and home appliances. Specific embodiments for carrying out the invention are described in detail below.

[0685] System configuration

[0686] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[0687] 1. User Terminal Embodiments

[0688] The user devices consist of smartphones and tablets used by the elderly. The devices have the following functions:

[0689] 1. Displaying AI avatars

[0690] The device launches a built-in app and displays an AI avatar on the screen, using photos of the elderly person's family and friends to create a sense of familiarity.

[0691] 2. Interactive features

[0692] The device engages in everyday conversations with the elderly through an AI avatar, generating dialogue using a natural language processing engine (e.g., GPT-4) to reduce feelings of loneliness and stimulate cognitive function.

[0693] 3. Game Features

[0694] The device provides activities such as simple puzzles and memory games, and is implemented using Unity and React Native to maintain and improve cognitive function in the elderly.

[0695] 4. Data Collection

[0696] The device collects activity data such as conversation results and game results, and periodically sends it to the server using the HTTP protocol.

[0697] 2. Server Implementation

[0698] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0699] 1. Data Receipt and Storage

[0700] The server receives activity data sent from the user's device and stores it in a database (e.g., MySQL, MongoDB).

[0701] 2. Data Analysis

[0702] The server analyzes the collected data using machine learning algorithms (e.g., Scikit-learn, TensorFlow), and if an anomaly is detected, it sends a notification to the family.

[0703] 3. Notification function

[0704] The server periodically reports the elderly person's activities to the family, and in case of an emergency, it sends an immediate notification via email, SMS, or push notification (such as Firebase Cloud Messaging).

[0705] 3. Sensor Unit Embodiment

[0706] The sensor unit integrates various sensors in the home to monitor the status of home appliances and residential systems, and can take action if necessary.

[0707] 1. Monitoring function

[0708] Monitor locks, lights, temperature, and activity of home appliances and systems. Collect information using IoT devices (e.g., Arduino, Raspberry Pi).

[0709] 2. Data transmission

[0710] The monitoring results are sent to the server periodically using MQTT as the communication protocol.

[0711] 3. Control Function

[0712] According to instructions from the server, it remotely controls household devices and locks / unlocks doors. Instructions given by family members using their smartphones are transmitted to each sensor via the server.

[0713] Specific examples

[0714] Example 1: Everyday usage scenario

[0715] Every morning, the user (elderly person) turns on their smartphone, and an AI avatar holding a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person replies, "I'm feeling fine today." While continuing the normal dialogue, the avatar suggests, "Would you like to play a little memory game?" The user plays the game, and the results are recorded on the device and periodically sent to the server. The server analyzes the results and notifies the family of "today's activity status."

[0716] Example 2: Anomaly detection and response

[0717] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family, and when the family member issues a command to "lock" the door from their smartphone, the command is transmitted via the server to the sensor unit, which then locks the door. This ensures the safety of the elderly, allowing the family to go about their daily lives with peace of mind.

[0718] Prompt Sentence Examples

[0719] Prompt 1: Use a user terminal

[0720] "Please tell us the process steps and specific actions of how an elderly person would turn on their smartphone every morning and interact with the AI ​​avatar displayed on the screen."

[0721] Prompt 2: Anomaly detection

[0722] "Please explain in detail the specific steps and technology used by the sensor unit to notify family members if their home is left unlocked."

[0723] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0724] Step 1: Start the user device and display the AI ​​avatar

[0725] 1. The user starts up the smartphone

[0726] Input: User powers on the smartphone

[0727] Operation: Initialize the smartphone and transition to the home screen

[0728] Output: Home screen display

[0729] 2. Launch the AI ​​avatar app

[0730] Input: Tap the AI ​​Avatar app icon on the home screen

[0731] What it does: The app launches and displays an avatar showing a photo of the senior's family or friend.

[0732] Output: Display of AI avatar

[0733] Step 2: Execute the interaction

[0734] 1. Initial interaction with an AI avatar

[0735] Input: AI avatar app launch complete

[0736] Action: AI avatar says "Good morning, how are you feeling today?"

[0737] Output: Questions for the elderly

[0738] 2. Receiving and analyzing elderly people's responses

[0739] Input: Elderly person responds "I'm fine today"

[0740] How it works: The device uses a speech recognition feature (such as the Google Speech-to-Text API) to convert the voice data into text, and then uses a generative AI model (such as GPT-4) to analyze the response.

[0741] Output: Generate appropriate answers based on the analysis results

[0742] Step 3: Providing game functionality

[0743] 1. Game suggestions

[0744] Input:Continue the dialogue

[0745] How it works: The AI ​​avatar suggests, "Would you like to play a little memory game?"

[0746] Output: Game suggestions for seniors

[0747] 2. Running the game

[0748] Input: Seniors start the game

[0749] Action: The device displays a memory game using Unity and React Native and waits for user interaction.

[0750] Output: Displaying the game interface and collecting user operation data

[0751] Step 4: Collect data and send it to the server

[0752] 1. Data collection

[0753] Input: Interaction results and game results

[0754] What it does: The device temporarily stores the collected data in its internal storage.

[0755] Output: Temporarily saved activity data

[0756] 2. Data transmission

[0757] Input: Temporarily saved activity data

[0758] What happens: The device sends data to the server using the HTTP protocol.

[0759] Output: Activity data sent to the server

[0760] Step 5: Receiving and analyzing data on the server

[0761] 1. Receiving Data

[0762] Input: Activity data sent from the terminal

[0763] What happens: The server receives an HTTP request and retrieves the data.

[0764] Output: Received activity data

[0765] 2. Data storage

[0766] Input: Received activity data

[0767] What happens: The server saves the data to a database (MySQL or MongoDB).

[0768] Output: Activity data stored in the database

[0769] 3. Data Analysis

[0770] Input: Activity data stored in the database

[0771] How it works: The server analyzes the data using machine learning algorithms (Scikit-learn or TensorFlow) and detects anomalies.

[0772] Output: Analysis results and anomaly detection information

[0773] Step 6: Notify family members

[0774] 1. Generate a notification

[0775] Input: Analysis results and anomaly detection information

[0776] Action: The server generates a notification message for the family.

[0777] Output: Notification message

[0778] 2. Sending notifications

[0779] Input: Notification message

[0780] How it works: The server sends notifications via email, SMS, or push notifications (Firebase Cloud Messaging).

[0781] Output: Notification sent to family members

[0782] Step 7: Sensor-by-sensor monitoring and control

[0783] 1. Performing monitoring functions

[0784] Input: Status of home appliances and systems

[0785] Operation: Sensor units constantly monitor locks, lighting, temperature, and operation. Data is acquired using IoT devices (Arduino and Raspberry Pi).

[0786] Output: Monitoring results

[0787] 2. Data Transmission

[0788] Input: Monitoring results

[0789] Operation: The sensor unit sends the monitoring results to the server using the MQTT protocol.

[0790] Output: Monitoring data sent to the server

[0791] 3. Receiving control instructions

[0792] Input: Control instructions from family members or the server

[0793] Operation: The server sends control instructions to each sensor, and the sensor operates the home appliance.

[0794] Output: Operation results of home appliances

[0795] (Application example 1)

[0796] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0797] In addition to systems that improve the quality of life for the elderly, there is a need for means to manage worker safety. In particular, in work environments such as factories, it is necessary to ensure safety by timely monitoring the physical condition of workers and the surrounding environment. In conventional systems, the elderly management system and the worker safety management system exist independently and require separate management, making efficient integrated management difficult. Furthermore, if real-time data collection and analysis results are not immediately notified, there is a risk of delays in appropriate response. The present invention aims to solve these problems and provide an integrated system that simultaneously improves the safety and quality of life of both the elderly and workers.

[0798] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0799] In this invention, the server includes a means for collecting elderly people's activity data and transmitting it to the server in real time, a means for collecting sensor data and transmitting it to the server, and a means for analyzing the collected data and notifying the safety manager. This improves the quality of life of elderly people and makes it possible to constantly monitor the physical condition and environmental conditions of workers and ensure their safety.

[0800] An "AI avatar" is a virtual character that uses artificial intelligence technology and is a means of interacting with the elderly or workers.

[0801] "Photos of family or friends" are images used to display AI avatars to increase familiarity with the elderly.

[0802] "Means for dialogue" refers to a mechanism that allows users and AI avatars to communicate through voice or text.

[0803] "Methods for maintaining cognitive function" are methods of providing mental stimulation through dialogue, games, etc. to maintain the cognitive abilities of elderly people.

[0804] "Means for collecting activity data and transmitting it to a server in real time" refers to technology that continuously collects activity information on elderly people and workers and instantly transmits that information to a cloud server.

[0805] The "means of notifying family members of activity data" refers to a method for informing family members of the elderly person's activity status.

[0806] "Means for detecting and notifying of forgotten locks and abnormalities" refers to technology that uses sensors in the home or work environment to monitor the status of keys and abnormalities, and issues alerts as necessary.

[0807] "Means for remotely controlling home appliances and home systems" refers to technology for remotely controlling home appliances and home systems via the Internet.

[0808] "Means for monitoring physical condition and environmental conditions and issuing warnings if abnormalities are detected" refers to a method in which sensors are used to detect abnormalities in workers' health and working environment and send appropriate alerts.

[0809] "Means for analyzing collected data and notifying safety managers" refers to methods for analyzing data obtained from sensors and reporting important information to safety managers in real time.

[0810] This invention is a system that comprehensively improves the quality of life for the elderly and manages worker safety. The system is composed of an AI avatar, a user terminal, a server, a sensor unit, and a communication network. Specific embodiments are described below.

[0811] System configuration

[0812] 1. User Terminal Embodiments

[0813] The user device can be, for example, a smartphone or tablet used by the elderly, or smart glasses used by workers. The device has the following functions:

[0814] Displaying AI avatars: The device displays AI avatars on the screen. The AI ​​avatars are set with photos of the elderly's family and friends, making them feel closer to the elderly. They can also be used to assist workers.

[0815] Conversational function: The device will engage in everyday conversations with the user through an AI avatar, thereby reducing feelings of loneliness among the elderly and stimulating their cognitive functions. It will also provide work instructions to workers.

[0816] Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0817] Data collection: The terminal collects the results of conversations and games, as well as the worker's physical condition and work data, and periodically transmits them to the server.

[0818] 2. Server Implementation

[0819] The server is the center of the system, centrally managing user activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0820] Data reception and storage: The server receives the activity data sent from the user terminals and sensor units and stores it in a database.

[0821] Data Analysis: The server analyzes the collected data and sends notifications to family members or safety managers if anomalies are detected, such as a drop in activity over a certain period of time.

[0822] Notification function: The server periodically reports user activity to family members and safety managers. In the event of an emergency, an alert is sent to prompt immediate action.

[0823] 3. Sensor Unit Embodiment

[0824] The sensor unit integrates various sensors in the home or work environment to monitor their status and take action if necessary.

[0825] Monitoring function: The sensor unit monitors temperature, movement, lock status, etc. within the home or work environment.

[0826] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0827] Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking.

[0828] Specific examples

[0829] Example 1: Daily usage scenario for elderly people

[0830] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[0831] Example 2: Worker Safety Management Scenario

[0832] A worker puts on the smart glasses and begins work. An AI avatar speaks to the worker through the glasses, asking, "What's the next step?" If the worker replies, "Assembly," the avatar displays the correct work procedure. The sensor unit monitors the temperature of the work environment and the worker's physical condition, and immediately displays a warning if there is an abnormality. The data is sent to a server and analyzed as appropriate. Based on the analysis results, a notification is sent to the safety manager.

[0833] A prompt sentence from a generative AI model as an example of use

[0834] Prompt statement:

[0835] "We are considering a voice assistant that will provide safety warnings while working in a factory. Please provide example dialogue."

[0836] As described above, this system not only improves the quality of life for the elderly, but also ensures the safety of working people, providing an integrated solution that meets the needs of both.

[0837] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0838] Step 1:

[0839] The device collects the user's speech and converts it into text data using a speech recognition function. The input for this process is the user's speech, and the output is text data.

[0840] Step 2:

[0841] The device sends the collected text data to the AI ​​avatar. The AI ​​avatar generates appropriate dialogue content based on this text data. It uses a generative AI model to generate dialogue based on the prompt text. The input is text data and the output is dialogue content. Specifically, when the user says, "Tell me the next step," the AI ​​avatar uses the generative AI model to answer, "Yes, the next step is assembling the parts."

[0842] Step 3:

[0843] The device outputs the generated dialogue content to the user as voice or text. The input for this process is the dialogue content data received from the AI ​​avatar, and the output is the dialogue result presented to the user. Specifically, the text generated using speech synthesis technology is played aloud or displayed on the smart glasses display.

[0844] Step 4:

[0845] The sensor unit continuously collects environmental data and user health data. This includes various sensors such as temperature sensors and heart rate monitors. The input of this process is the data obtained from the various sensors, and the output is the collected environmental data and health data.

[0846] Step 5:

[0847] The sensor unit sends the collected data to a server. The server stores the received data in a database and analyzes it in real time. The input to this process is the data sent from the sensor unit, and the output is the stored data and the analysis results. For example, if the temperature exceeds the allowable range within a certain period of time, this information is recorded.

[0848] Step 6:

[0849] The server sends a notification if an abnormality is detected based on the analysis results. For example, if the temperature in the work environment exceeds a set standard, the server will send an alert to the safety manager. The input to this process is the analysis result data, and the output is the notification data. Specifically, the alert is sent via email or an in-application notification function.

[0850] Step 7:

[0851] Users (workers and elderly people) receive notifications and alerts and take necessary measures. The input to this process is notification data from the server, and the output is the corresponding action. For example, if an alert about an abnormal temperature is received, the user will temporarily suspend their work and take appropriate measures.

[0852] The above process will create a system that comprehensively improves the quality of life for the elderly and manages worker safety.

[0853] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0854] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[0855] System configuration

[0856] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[0857] 1. User Terminal Embodiments

[0858] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0859] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0860] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0861] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0862] 4. Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[0863] 5. Data collection: The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically sends them to the server.

[0864] 2. Server Implementation

[0865] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0866] 1. Receiving and storing data: The server receives the activity data and emotion data sent from the user terminal and stores it in a database.

[0867] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[0868] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[0869] 3. Sensor Unit Embodiment

[0870] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0871] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[0872] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[0873] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[0874] Specific examples

[0875] Example 1: Everyday usage scenario

[0876] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[0877] Example 2: Anomaly detection and response

[0878] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[0879] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[0880] The processing flow will be explained below.

[0881] Specific processing steps of the program

[0882] User terminal processing

[0883] Step 1:

[0884] The user starts the terminal.

[0885] Step 2:

[0886] The terminal displays a login screen and prompts the user to enter their ID and password.

[0887] Step 3:

[0888] The user enters their ID and password and clicks the login button.

[0889] Step 4:

[0890] The terminal sends the entered ID and password to the server.

[0891] Step 5:

[0892] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[0893] Step 6:

[0894] The device displays an AI avatar based on the data it receives.

[0895] Step 7:

[0896] The user selects the interaction or game.

[0897] Step 8:

[0898] The device executes the selected dialogue or game logic and interacts with the user, capturing the user's voice and facial expressions with a camera.

[0899] Step 9:

[0900] The device transmits the acquired voice and facial expression data to an emotion engine to analyze the user's emotional state.

[0901] Step 10:

[0902] The emotion engine returns the analysis results to the device, and the device dynamically changes the dialogue and game content based on these results.

[0903] Step 11:

[0904] The terminal collects user input data, behavioral data, and emotional data.

[0905] Step 12:

[0906] The terminal transmits the collected data to the server at regular intervals.

[0907] Server Processing

[0908] Step 1:

[0909] The server checks the ID and password sent by the user against the database.

[0910] Step 2:

[0911] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[0912] Step 3:

[0913] The server periodically receives the user's behavioral data and emotion data transmitted from the terminal.

[0914] Step 4:

[0915] The server analyzes the received data and stores it in a database.

[0916] Step 5:

[0917] The server analyzes the user's activity and emotional state based on the stored data.

[0918] Step 6:

[0919] If desired, the server will send notifications and alerts to family members about their activity and emotional state.

[0920] Sensor unit processing

[0921] Step 1:

[0922] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[0923] Step 2:

[0924] The sensor unit transmits the acquired data to the server.

[0925] Step 3:

[0926] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[0927] Step 4:

[0928] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[0929] Step 5:

[0930] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[0931] Step 6:

[0932] The sensor unit follows the received command and performs the required action (for example, locking the door).

[0933] Specific examples

[0934] Example 1: Everyday usage scenario

[0935] Step 1:

[0936] Every morning, the user turns on the tablet and an AI avatar with a photo of their grandchild appears on the screen.

[0937] Step 2:

[0938] The AI ​​avatar speaks, "Good morning, how are you feeling today?"

[0939] Step 3:

[0940] The user responds, "I'm feeling good today." At this time, the device records the user's voice and analyzes their emotional state using an emotion engine.

[0941] Step 4:

[0942] While engaging in normal dialogue, the avatar suggests a simple memory game.

[0943] Step 5:

[0944] The user plays the game and the results and emotional state are recorded on the terminal.

[0945] Step 6:

[0946] The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[0947] Example 2: Anomaly detection and response

[0948] Step 1:

[0949] The sensor unit detects when the house is unlocked and sends this information to a server.

[0950] Step 2:

[0951] The server immediately notifies the family.

[0952] Step 3:

[0953] When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, and the door is locked.

[0954] Step 4:

[0955] If the emotion engine detects anxiety in the elderly person, a notification will be sent to the family member stating that the elderly person is feeling anxious.

[0956] Example 2

[0957] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0958] There is a need to maintain the cognitive functions of the elderly, reduce feelings of loneliness, and ensure safety within the home. However, conventional systems have had difficulty adequately meeting these complex needs. In addition, there is a lack of means to properly analyze the emotional state and promptly notify family members. A new system is needed to solve these issues.

[0959] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0960] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly person to grasp their emotional state, a means for collecting activity data and emotional data of the elderly person and periodically transmitting it to the server, and a means for the server to analyze the received data and send a notification to family members if an abnormality is detected. This makes it possible to maintain the elderly person's cognitive function, reduce loneliness, and ensure safety within the home.

[0961] 1. "AI Avatar" is a virtual character used to interact with seniors, representing photographs of the user's family and friends.

[0962] 2. "Means for conducting dialogue" refers to technical devices and software that use AI avatars to engage in natural language dialogue with older people.

[0963] 3. "Cognitive maintenance strategies" are technologies that provide games and activities designed to stimulate and maintain cognitive abilities in older adults.

[0964] 4. "Means for understanding emotional states" refers to analytical technologies and devices that analyze the voices and facial expressions of elderly people to understand their emotions.

[0965] 5. "Means for collecting activity data and emotional data and periodically transmitting it to a server" refers to technology that uses terminals or sensor units to record the daily activities and emotional states of elderly people and transmits them to a server via a network.

[0966] 6. "The server analyzes the received data and notifies the family if an abnormality is detected" refers to a technology that analyzes data on the server side and immediately notifies the family if an abnormality is detected.

[0967] 7. "Means for linking with home appliances and home systems to detect and notify when locks are left locked or other abnormalities" refers to technology that uses sensor units in the home to monitor the status of home appliances and the home, and notifies if an abnormality is detected.

[0968] 8. "Means for remotely controlling home appliances and home systems" refers to technology that remotely operates and controls home appliances and various home facilities based on instructions from a server.

[0969] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[0970] System configuration

[0971] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[0972] 1. User Terminal Embodiments

[0973] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[0974] 1. Displaying AI avatars:

[0975] The device displays an AI avatar on the screen, which is set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[0976] 2. Interactive features:

[0977] The device uses natural language processing models (e.g., GPT-4) to engage in everyday conversations with the elderly, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[0978] 3. Game Features:

[0979] The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[0980] 4. Emotion recognition function:

[0981] The device analyzes the elderly person's voice and facial expressions and uses an emotion engine (e.g., Emotion API) to understand their emotional state.

[0982] 5. Data Collection:

[0983] The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically transmits them to the server.

[0984] 2. Server Implementation

[0985] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[0986] 1. Data Receipt and Storage:

[0987] The server receives the activity data and emotion data sent from the user terminal and stores them in a database (e.g., MySQL).

[0988] 2. Data Analysis:

[0989] The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[0990] 3. Notification function:

[0991] The server periodically reports the elderly person's activities to the family, and in the event of an emergency, it sends an alert to prompt immediate action.

[0992] 3. Sensor Unit Embodiment

[0993] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[0994] 1. Monitoring function:

[0995] The sensor unit monitors the house's locks, electricity, temperature, motion detection, and more.

[0996] 2. Data transmission:

[0997] The sensor unit periodically transmits the monitoring results to the server.

[0998] 3. Control function:

[0999] It receives instructions from the server and performs remote control of home appliances and locks / unlocks doors.

[1000] Specific examples

[1001] Example 1: Everyday usage scenario

[1002] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[1003] Example prompt sentence:

[1004] A user turns on the tablet, and an AI avatar with a photo of their grandchild speaks to them, saying, "Good morning, how are you feeling today?" When the user replies, "I'm feeling fine today," explain the system that performs voice analysis, records their emotional state, and sends the result to a server.

[1005] Example 2: Anomaly detection and response

[1006] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[1007] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[1008] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1009] Step 1: Start the user device

[1010] What it does: A user presses the power button on the tablet to start up the device.

[1011] Input: User presses and holds the power button.

[1012] Output: The device's home screen is displayed.

[1013] Specific behavior: The user presses and holds the tablet's power button, the boot screen appears, and then the home screen appears.

[1014] Step 2: Displaying the AI ​​Avatar

[1015] How it works: The device automatically launches the AI ​​avatar app from the home screen and displays an AI avatar on the screen, set to a photo of the elderly person's family or friends.

[1016] Enter: the app icon on your home screen.

[1017] Output: An AI avatar is displayed.

[1018] What it does: The device automatically launches the application and displays an AI avatar with a photo of your grandchild.

[1019] Step 3: Initiating everyday conversations and recognizing emotions

[1020] How it works: The device uses a natural language processing model (e.g., GPT-4) to have an AI avatar say, "Good morning, how are you feeling today?" The device then performs a voice analysis of the user's response and uses emotion recognition technology to determine their emotional state.

[1021] Input: A prompt to the generative AI model: "Good morning, how are you feeling today?" and the user's response.

[1022] Output: Emotional state (e.g., "Energetic").

[1023] Specific operation: The AI ​​avatar asks, "Good morning, how are you feeling today?", and the elderly person replies, "I'm feeling fine today." The device records the answer and uses its emotion engine to interpret it as "feeling fine."

[1024] Step 4: Propose and play the game

[1025] How it works: The device uses an AI avatar to ask the elderly, "Would you like to play a memory game today?" If the elderly agrees, the device starts the memory game.

[1026] Input: The game suggestion prompt, "Would you like to play a memory game today?" and the user's response.

[1027] Output: Game result.

[1028] Specific operation: The AI ​​avatar suggests, "Would you like to play a memory game today?" If the elderly person responds, "Sure," a simple memory game will appear on the screen.

[1029] Step 5: Collect and send data

[1030] How it works: The device collects data such as conversation content, game results, and emotion recognition results, and sends the collected data to a server via Wi-Fi.

[1031] Input: Dialogue recording, game results, emotion recognition results.

[1032] Output: Sending data to the server.

[1033] Specific operation: The device stores the conversation records, game results, and emotional state as data, and transmits them to the server via Wi-Fi.

[1034] Step 6: Server receives and stores data

[1035] Operation: The server receives data sent from the device and stores it in a database (e.g., MySQL).

[1036] Input: Data sent from the terminal.

[1037] Output: Save to database.

[1038] What happens: The server stores all received data as entries in a MySQL database.

[1039] Step 7: Data analysis and notification by the server

[1040] What it does: The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected.

[1041] Input: Record data from the database.

[1042] Output: Notification to family.

[1043] Specific operation: The server analyzes the data, and when it detects a decrease in activity, it sends a notification email to the family saying, "The elderly person's activity is decreasing."

[1044] Step 8: Anomaly detection and response

[1045] How it works: The sensor unit detects that the house doors are unlocked and sends this information to the server. The server immediately notifies the family, who then issue a command to "lock" the door via their smartphone. The sensor unit then remotely locks the door based on the received command.

[1046] Input: Anomaly detection data from the sensor unit, instructions from family members.

[1047] Output: Locks the door and notifies family members of the anxiety state.

[1048] Specific operation: The sensor detects that the door to the house is open and sends the information to the server. When a family member presses the "lock" button on their smartphone, the server sends a command to the sensor unit, which then locks the front door. If the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious."

[1049] These are the specific steps in the program processing of this system, which will help maintain cognitive function, reduce loneliness, and improve safety for the elderly.

[1050] (Application example 2)

[1051] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1052] In order to improve the quality of life of the elderly, it is important to maintain cognitive function and ensure their safety. However, because the time that family members can directly interact with the elderly is limited, a system that can remotely grasp the elderly's situation and respond appropriately is required. It is also important to analyze the elderly's emotional state and respond accordingly. Given this background, it is necessary to provide a system that can monitor the safety and emotional state of the elderly in real time and respond quickly to abnormalities or changes in emotion.

[1053] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1054] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly to grasp their emotional state, a means for monitoring the elderly's environment with a camera in the smart glasses, and a means for sending an alert to relatives when an abnormality is detected. This not only makes it possible to maintain the elderly's cognitive function and ensure their safety, but also enables immediate response to changes in their emotional state.

[1055] An "AI avatar" is a digital character equipped with artificial intelligence to interact with the elderly and display a photo of a relative or acquaintance.

[1056] "Means for displaying photographs of relatives or acquaintances" refers to technology that displays photographs of family members or friends on a digital display to help elderly people feel closer to them.

[1057] "Means for responding" is a function that uses recognition technology and voice synthesis technology to enable elderly people and AI avatars to communicate through natural conversation.

[1058] "Means for maintaining cognitive function in the elderly through entertainment" is a technology that aims to maintain and improve cognitive function in the elderly by providing activities such as simple games and puzzles.

[1059] "Means for collecting activity data and transmitting it to a central processing unit in real time" refers to a function that uses sensors and devices to record the elderly person's daily activities, voice, facial expressions, etc., and transmits the data to a server in real time.

[1060] The "means of notifying relatives of activity data" is a function that automatically analyzes collected data on elderly people and notifies relatives of abnormalities and necessary information.

[1061] The "means for detecting and notifying when someone forgets to lock their home or when there is an abnormality" is a technology that constantly monitors the elderly person's home appliances and home systems, and notifies relatives when it detects that someone has forgotten to lock their home or when there is an abnormality.

[1062] "Means for remotely controlling home appliances and home systems" refers to technology that allows relatives to remotely control home appliances and systems in the elderly person's home via devices such as smartphones.

[1063] "Means for monitoring the environment of the elderly using a camera in smart glasses" is a function that uses a camera installed in smart glasses to monitor the environment around the elderly in real time.

[1064] "Means for analyzing voice and facial expressions to grasp emotional states" refers to technology that analyzes voice and facial expression data of elderly people and uses an emotion engine to estimate and grasp their emotional states.

[1065] The "means for sending an alert to relatives when an abnormality is detected" is a system that sends an alert message to relatives when an abnormality is detected as a result of various monitoring performed on the elderly person.

[1066] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. In particular, this system has the ability to analyze the emotional state of the elderly using an emotion engine and utilize the analysis data. Specific embodiments are described below.

[1067] System configuration

[1068] The embodiment of the present invention is broadly composed of the following components.

[1069] 1. User Terminal Embodiments

[1070] The user devices are smartphones or smart glasses used by elderly people. The devices have the following main functions:

[1071] Display of AI avatar: The device displays an AI avatar on the display, which is set to a photo of the elderly person's relatives or acquaintances, giving the user a sense of familiarity.

[1072] Voice response function: The device responds naturally to the elderly through an AI avatar, helping to reduce feelings of loneliness and stimulate cognitive function.

[1073] Activity provision function: The device provides simple activities such as puzzles and games, which further maintain and improve cognitive function in the elderly.

[1074] Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[1075] Data collection and transmission: Responses, game results, and emotion recognition results are collected as data and periodically sent to the server.

[1076] 2. Server Implementation

[1077] The server is the center of the system, centrally managing elderly people's activity data and providing necessary notifications and analysis.

[1078] Data reception and storage: The server receives the activity data and emotion data sent from the user terminal and stores them in a database.

[1079] Data analysis: The server analyzes the collected data and sends notifications to relatives if any abnormalities are detected.

[1080] Notification function: The server periodically reports the elderly person's activity and emotional state to relatives, and sends immediate alerts in case of an emergency.

[1081] 3. Sensor Unit Embodiment

[1082] The sensor unit integrates various sensors in the home and monitors the status of home appliances and systems.

[1083] Monitoring functions: Sensors monitor locks, electricity, temperature, motion, and more within the home.

[1084] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1085] Control function: Receives instructions from the server and remotely controls and locks / unlocks home appliances.

[1086] Specific examples

[1087] Example 1: Everyday usage scenario

[1088] The user (elderly person) turns on the smart glasses every morning, and an AI avatar with a photo of a relative's face appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While making a normal response, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the relative of "today's activity status and emotional state."

[1089] Example 2: Anomaly detection and response

[1090] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies relatives. When a relative issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. If the emotion engine detects anxiety in the elderly person at this time, it also sends a notification to the relative that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also enables responses that take their emotional state into consideration.

[1091] Prompt Sentence Examples

[1092] "You will develop a function to detect falls among elderly people based on image data of high-risk abnormal behavior and falls, and image data of normal behavior. You will also create a system to analyze the emotional state of elderly people from images of their facial expressions, and collect and store that data."

[1093] In this way, the present invention provides a system that enables the maintenance of cognitive function and safety of elderly people, and instantly grasps their emotional state using an emotion engine.

[1094] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1095] Step 1:

[1096] The user turns on the smart glasses

[1097] Input: User action of pressing the power button on the smart glasses.

[1098] Operation: The smart glasses start up and each module of the system is initialized.

[1099] Output: The camera and microphone on the smart glasses start working, and the AI ​​avatar appears on the display.

[1100] Step 2:

[1101] The device captures the voice and facial expressions of the elderly person.

[1102] Input: Video data from the camera and audio data from the microphone of the smart glasses

[1103] How it works: The camera recognizes the elderly person's face and the microphone records their voice. This data is temporarily stored on the device.

[1104] Output: Video frames and audio clips of the elderly

[1105] Step 3:

[1106] The device analyzes your emotional state

[1107] Input: Captured video frames and audio clips

[1108] How it works: The device inputs this data into an emotion engine, which performs facial expression analysis and voice analysis. It then uses an emotion recognition model (e.g., a generative AI model using TensorFlow) to estimate the user's emotional state (e.g., happiness, sadness, anxiety, etc.).

[1109] Output: Estimated emotional state

[1110] Step 4:

[1111] The device responds to the elderly person through an AI avatar.

[1112] Input: The user's previous voice response and detected emotional state

[1113] How it works: The AI ​​avatar generates dialogue that responds to the user with the following responses, including the content of the response and speech synthesis.

[1114] Output: The following dialogue and synthesized speech

[1115] Step 5:

[1116] The device sends activity data to the server

[1117] Input: Responses, emotional state analysis results, elderly person activity data (game results, etc.)

[1118] How it works: The device packages the data and sends it over the internet to a server

[1119] Output: Elderly activity data sent to the server

[1120] Step 6:

[1121] The server receives and analyzes the data

[1122] Input: Elderly person's activity data sent from the device

[1123] How it works: The server stores the received data in a database and analyzes it for any abnormalities. If an abnormality is detected, an alert is sent to the relatives.

[1124] Output: Today's activity report or anomaly alert

[1125] Step 7:

[1126] The server monitors the data from the sensor units.

[1127] Input: Environmental data sent from sensor units in the home (temperature, lock status, etc.)

[1128] Operation: The server periodically receives this data and detects abnormal conditions.

[1129] Output: Alert notification when an abnormality occurs, log update when the status is normal

[1130] Step 8:

[1131] A relative remotely controls the device

[1132] Input: Operation command from smartphone (e.g., command to lock the door)

[1133] How it works: The server receives the command and sends appropriate control signals to the sensor units in the home.

[1134] Output: Home appliances and systems operate according to the relatives' instructions.

[1135] The above processing steps realize a system that monitors the living environment and emotional state of the elderly in real time, promptly notifies relatives when an abnormality occurs, and enables remote control.

[1136] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1137] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1138] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1139] [Third embodiment]

[1140] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1141] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1142] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1143] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1144] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1145] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1146] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1147] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1148] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1149] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1150] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1151] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1152] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Specific embodiments for implementation are described in detail below.

[1153] System configuration

[1154] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[1155] 1. User Terminal Embodiments

[1156] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[1157] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[1158] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[1159] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1160] 4. Data collection: The device collects data on interactions and game results and periodically sends it to the server.

[1161] 2. Server Implementation

[1162] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1163] 1. Receiving and storing data: The server receives the activity data sent from the user terminal and stores it in a database.

[1164] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a drop in activity over a certain period of time.

[1165] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[1166] 3. Sensor Unit Embodiment

[1167] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[1168] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[1169] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1170] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[1171] Specific examples

[1172] Example 1: Everyday usage scenario

[1173] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[1174] Example 2: Anomaly detection and response

[1175] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. This ensures the safety of the elderly and allows the family to go about their daily lives with peace of mind.

[1176] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety.

[1177] The processing flow will be explained below.

[1178] Specific processing steps of the program

[1179] User terminal processing

[1180] Step 1:

[1181] The user starts the terminal.

[1182] Step 2:

[1183] The terminal displays a login screen and prompts the user to enter their ID and password.

[1184] Step 3:

[1185] The user enters their ID and password and clicks the login button.

[1186] Step 4:

[1187] The terminal sends the entered ID and password to the server.

[1188] Step 5:

[1189] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[1190] Step 6:

[1191] The device displays an AI avatar based on the data it receives.

[1192] Step 7:

[1193] The user selects the interaction or game.

[1194] Step 8:

[1195] The device executes the selected dialogue or game logic and interacts with the user.

[1196] Step 9:

[1197] The device collects user input and behavioral data.

[1198] Step 10:

[1199] The terminal transmits the collected data to the server at regular intervals.

[1200] Server Processing

[1201] Step 1:

[1202] The server checks the ID and password sent by the user against the database.

[1203] Step 2:

[1204] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[1205] Step 3:

[1206] The server periodically receives the user's behavior data transmitted from the terminal.

[1207] Step 4:

[1208] The server analyzes the received data and stores it in a database.

[1209] Step 5:

[1210] The server analyzes the user's activity based on the stored data.

[1211] Step 6:

[1212] If necessary, the server will send activity notifications and alerts to family members.

[1213] Sensor unit processing

[1214] Step 1:

[1215] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[1216] Step 2:

[1217] The sensor unit transmits the acquired data to the server.

[1218] Step 3:

[1219] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[1220] Step 4:

[1221] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[1222] Step 5:

[1223] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[1224] Step 6:

[1225] The sensor unit follows the received command and performs the required action (for example, locking the door).

[1226] Example 1

[1227] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1228] Declining cognitive function, increasing loneliness, and safety issues significantly impair the quality of life of older adults. Effective and sustainable solutions to these problems are needed. However, existing technologies have not provided a system that comprehensively addresses the complex issues that older adults face in their daily lives. In particular, there is a need for a system that simultaneously maintains and improves cognitive function, ensures safety, and reduces loneliness.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1230] In this invention, the server includes a means for displaying an AI avatar and photos of the elderly person's family or friends, a means for interacting with the elderly person through the AI ​​avatar, a means for maintaining and improving the elderly person's cognitive function through interactions and games, a means for collecting the elderly person's activity data and periodically transmitting it to the server, a means for notifying family members of the elderly person's activity data, a means for linking with home appliances and housing systems to detect and notify if the elderly person has forgotten to lock the door or if there is an abnormality, and a means for remotely operating the elderly person's home appliances and housing systems. This makes it possible to maintain and improve the elderly person's cognitive function, reduce loneliness, and ensure their safety.

[1231] An "AI avatar" is a virtual character equipped with artificial intelligence, designed to stimulate cognitive function and reduce feelings of loneliness through dialogue and communication with the elderly.

[1232] The "photos" are facial images of the elderly person's family or friends, intended to give the elderly a sense of familiarity and security.

[1233] The "dialogue function" is a function that allows an AI avatar to have everyday conversations with elderly people and maintain and improve their cognitive function.

[1234] The "game function" allows seniors to participate in activities such as simple puzzles and memory games, with the aim of further maintaining and improving cognitive function.

[1235] "Activity data" refers to information such as the elderly person's daily behavior, conversations, and game results, and is analyzed to detect abnormalities.

[1236] The "server" is a computer system that functions as the central part of the system, centrally managing elderly people's activity data and conducting analysis and notifications.

[1237] "Data collection" is the process by which user terminals and sensor units gather information about the elderly person's daily activities and use it for later analysis.

[1238] The "notification function" is a function that reports activity data and notifies abnormalities to the elderly person's family members, and sends alerts using email, SMS, push notifications, etc.

[1239] "Home devices" refers to electrical appliances and equipment used in the homes of seniors that can be remotely controlled and monitored.

[1240] "Housing systems" refer to facilities that provide basic functions such as locks, lighting, and temperature control for elderly people's homes, and are monitored and controlled through sensors.

[1241] A "sensor unit" refers to an individual sensor device that is used to continuously monitor the status of a home appliance or home system.

[1242] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function and improve safety of the elderly through the management of AI avatars and home appliances. Specific embodiments for carrying out the invention are described in detail below.

[1243] System configuration

[1244] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[1245] 1. User Terminal Embodiments

[1246] The user devices consist of smartphones and tablets used by the elderly. The devices have the following functions:

[1247] 1. Displaying AI avatars

[1248] The device launches a built-in app and displays an AI avatar on the screen, using photos of the elderly person's family and friends to create a sense of familiarity.

[1249] 2. Interactive features

[1250] The device engages in everyday conversations with the elderly through an AI avatar, generating dialogue using a natural language processing engine (e.g., GPT-4) to reduce feelings of loneliness and stimulate cognitive function.

[1251] 3. Game Features

[1252] The device provides activities such as simple puzzles and memory games, and is implemented using Unity and React Native to maintain and improve cognitive function in the elderly.

[1253] 4. Data Collection

[1254] The device collects activity data such as conversation results and game results, and periodically sends it to the server using the HTTP protocol.

[1255] 2. Server Implementation

[1256] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1257] 1. Data Receipt and Storage

[1258] The server receives activity data sent from the user's device and stores it in a database (e.g., MySQL, MongoDB).

[1259] 2. Data Analysis

[1260] The server analyzes the collected data using machine learning algorithms (e.g., Scikit-learn, TensorFlow), and if an anomaly is detected, it sends a notification to the family.

[1261] 3. Notification function

[1262] The server periodically reports the elderly person's activities to the family, and in case of an emergency, it sends an immediate notification via email, SMS, or push notification (such as Firebase Cloud Messaging).

[1263] 3. Sensor Unit Embodiment

[1264] The sensor unit integrates various sensors in the home to monitor the status of home appliances and residential systems, and can take action if necessary.

[1265] 1. Monitoring function

[1266] Monitor locks, lights, temperature, and activity of home appliances and systems. Collect information using IoT devices (e.g., Arduino, Raspberry Pi).

[1267] 2. Data transmission

[1268] The monitoring results are sent to the server periodically using MQTT as the communication protocol.

[1269] 3. Control Function

[1270] According to instructions from the server, it remotely controls household devices and locks / unlocks doors. Instructions given by family members using their smartphones are transmitted to each sensor via the server.

[1271] Specific examples

[1272] Example 1: Everyday usage scenario

[1273] Every morning, the user (elderly person) turns on their smartphone, and an AI avatar holding a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person replies, "I'm feeling fine today." While continuing the normal dialogue, the avatar suggests, "Would you like to play a little memory game?" The user plays the game, and the results are recorded on the device and periodically sent to the server. The server analyzes the results and notifies the family of "today's activity status."

[1274] Example 2: Anomaly detection and response

[1275] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family, and when the family member issues a command to "lock" the door from their smartphone, the command is transmitted via the server to the sensor unit, which then locks the door. This ensures the safety of the elderly, allowing the family to go about their daily lives with peace of mind.

[1276] Prompt Sentence Examples

[1277] Prompt 1: Use a user terminal

[1278] "Please tell us the process steps and specific actions of how an elderly person would turn on their smartphone every morning and interact with the AI ​​avatar displayed on the screen."

[1279] Prompt 2: Anomaly detection

[1280] "Please explain in detail the specific steps and technology used by the sensor unit to notify family members if their home is left unlocked."

[1281] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1282] Step 1: Start the user device and display the AI ​​avatar

[1283] 1. The user starts up the smartphone

[1284] Input: User powers on the smartphone

[1285] Operation: Initialize the smartphone and transition to the home screen

[1286] Output: Home screen display

[1287] 2. Launch the AI ​​avatar app

[1288] Input: Tap the AI ​​Avatar app icon on the home screen

[1289] What it does: The app launches and displays an avatar showing a photo of the senior's family or friend.

[1290] Output: Display of AI avatar

[1291] Step 2: Execute the interaction

[1292] 1. Initial interaction with an AI avatar

[1293] Input: AI avatar app launch complete

[1294] Action: AI avatar says "Good morning, how are you feeling today?"

[1295] Output: Questions for the elderly

[1296] 2. Receiving and analyzing elderly people's responses

[1297] Input: Elderly person responds "I'm fine today"

[1298] How it works: The device uses a speech recognition feature (such as the Google Speech-to-Text API) to convert the voice data into text, and then uses a generative AI model (such as GPT-4) to analyze the response.

[1299] Output: Generate appropriate answers based on the analysis results

[1300] Step 3: Providing game functionality

[1301] 1. Game suggestions

[1302] Input:Continue the dialogue

[1303] How it works: The AI ​​avatar suggests, "Would you like to play a little memory game?"

[1304] Output: Game suggestions for seniors

[1305] 2. Running the game

[1306] Input: Seniors start the game

[1307] Action: The device displays a memory game using Unity and React Native and waits for user interaction.

[1308] Output: Displaying the game interface and collecting user operation data

[1309] Step 4: Collect data and send it to the server

[1310] 1. Data collection

[1311] Input: Interaction results and game results

[1312] What it does: The device temporarily stores the collected data in its internal storage.

[1313] Output: Temporarily saved activity data

[1314] 2. Data transmission

[1315] Input: Temporarily saved activity data

[1316] What happens: The device sends data to the server using the HTTP protocol.

[1317] Output: Activity data sent to the server

[1318] Step 5: Receiving and analyzing data on the server

[1319] 1. Receiving Data

[1320] Input: Activity data sent from the terminal

[1321] What happens: The server receives an HTTP request and retrieves the data.

[1322] Output: Received activity data

[1323] 2. Data storage

[1324] Input: Received activity data

[1325] What happens: The server saves the data to a database (MySQL or MongoDB).

[1326] Output: Activity data stored in the database

[1327] 3. Data Analysis

[1328] Input: Activity data stored in the database

[1329] How it works: The server analyzes the data using machine learning algorithms (Scikit-learn or TensorFlow) and detects anomalies.

[1330] Output: Analysis results and anomaly detection information

[1331] Step 6: Notify family members

[1332] 1. Generate a notification

[1333] Input: Analysis results and anomaly detection information

[1334] Action: The server generates a notification message for the family.

[1335] Output: Notification message

[1336] 2. Sending notifications

[1337] Input: Notification message

[1338] How it works: The server sends notifications via email, SMS, or push notifications (Firebase Cloud Messaging).

[1339] Output: Notification sent to family members

[1340] Step 7: Sensor-by-sensor monitoring and control

[1341] 1. Performing monitoring functions

[1342] Input: Status of home appliances and systems

[1343] Operation: Sensor units constantly monitor locks, lighting, temperature, and operation. Data is acquired using IoT devices (Arduino and Raspberry Pi).

[1344] Output: Monitoring results

[1345] 2. Data Transmission

[1346] Input: Monitoring results

[1347] Operation: The sensor unit sends the monitoring results to the server using the MQTT protocol.

[1348] Output: Monitoring data sent to the server

[1349] 3. Receiving control instructions

[1350] Input: Control instructions from family members or the server

[1351] Operation: The server sends control instructions to each sensor, and the sensor operates the home appliance.

[1352] Output: Operation results of home appliances

[1353] (Application example 1)

[1354] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1355] In addition to systems that improve the quality of life for the elderly, there is a need for means to manage worker safety. In particular, in work environments such as factories, it is necessary to ensure safety by timely monitoring the physical condition of workers and the surrounding environment. In conventional systems, the elderly management system and the worker safety management system exist independently and require separate management, making efficient integrated management difficult. Furthermore, if real-time data collection and analysis results are not immediately notified, there is a risk of delays in appropriate response. The present invention aims to solve these problems and provide an integrated system that simultaneously improves the safety and quality of life of both the elderly and workers.

[1356] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1357] In this invention, the server includes a means for collecting elderly people's activity data and transmitting it to the server in real time, a means for collecting sensor data and transmitting it to the server, and a means for analyzing the collected data and notifying the safety manager. This improves the quality of life of elderly people and makes it possible to constantly monitor the physical condition and environmental conditions of workers and ensure their safety.

[1358] An "AI avatar" is a virtual character that uses artificial intelligence technology and is a means of interacting with the elderly or workers.

[1359] "Photos of family or friends" are images used to display AI avatars to increase familiarity with the elderly.

[1360] "Means for dialogue" refers to a mechanism that allows users and AI avatars to communicate through voice or text.

[1361] "Methods for maintaining cognitive function" are methods of providing mental stimulation through dialogue, games, etc. to maintain the cognitive abilities of elderly people.

[1362] "Means for collecting activity data and transmitting it to a server in real time" refers to technology that continuously collects activity information on elderly people and workers and instantly transmits that information to a cloud server.

[1363] The "means of notifying family members of activity data" refers to a method for informing family members of the elderly person's activity status.

[1364] "Means for detecting and notifying of forgotten locks and abnormalities" refers to technology that uses sensors in the home or work environment to monitor the status of keys and abnormalities, and issues alerts as necessary.

[1365] "Means for remotely controlling home appliances and home systems" refers to technology for remotely controlling home appliances and home systems via the Internet.

[1366] "Means for monitoring physical condition and environmental conditions and issuing warnings if abnormalities are detected" refers to a method in which sensors are used to detect abnormalities in workers' health and working environment and send appropriate alerts.

[1367] "Means for analyzing collected data and notifying safety managers" refers to methods for analyzing data obtained from sensors and reporting important information to safety managers in real time.

[1368] This invention is a system that comprehensively improves the quality of life for the elderly and manages worker safety. The system is composed of an AI avatar, a user terminal, a server, a sensor unit, and a communication network. Specific embodiments are described below.

[1369] System configuration

[1370] 1. User Terminal Embodiments

[1371] The user device can be, for example, a smartphone or tablet used by the elderly, or smart glasses used by workers. The device has the following functions:

[1372] Displaying AI avatars: The device displays AI avatars on the screen. The AI ​​avatars are set with photos of the elderly's family and friends, making them feel closer to the elderly. They can also be used to assist workers.

[1373] Conversational function: The device will engage in everyday conversations with the user through an AI avatar, thereby reducing feelings of loneliness among the elderly and stimulating their cognitive functions. It will also provide work instructions to workers.

[1374] Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1375] Data collection: The terminal collects the results of conversations and games, as well as the worker's physical condition and work data, and periodically transmits them to the server.

[1376] 2. Server Implementation

[1377] The server is the center of the system, centrally managing user activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1378] Data reception and storage: The server receives the activity data sent from the user terminals and sensor units and stores it in a database.

[1379] Data Analysis: The server analyzes the collected data and sends notifications to family members or safety managers if anomalies are detected, such as a drop in activity over a certain period of time.

[1380] Notification function: The server periodically reports user activity to family members and safety managers. In the event of an emergency, an alert is sent to prompt immediate action.

[1381] 3. Sensor Unit Embodiment

[1382] The sensor unit integrates various sensors in the home or work environment to monitor their status and take action if necessary.

[1383] Monitoring function: The sensor unit monitors temperature, movement, lock status, etc. within the home or work environment.

[1384] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1385] Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking.

[1386] Specific examples

[1387] Example 1: Daily usage scenario for elderly people

[1388] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[1389] Example 2: Worker Safety Management Scenario

[1390] A worker puts on the smart glasses and begins work. An AI avatar speaks to the worker through the glasses, asking, "What's the next step?" If the worker replies, "Assembly," the avatar displays the correct work procedure. The sensor unit monitors the temperature of the work environment and the worker's physical condition, and immediately displays a warning if there is an abnormality. The data is sent to a server and analyzed as appropriate. Based on the analysis results, a notification is sent to the safety manager.

[1391] A prompt sentence from a generative AI model as an example of use

[1392] Prompt statement:

[1393] "We are considering a voice assistant that will provide safety warnings while working in a factory. Please provide example dialogue."

[1394] As described above, this system not only improves the quality of life for the elderly, but also ensures the safety of working people, providing an integrated solution that meets the needs of both.

[1395] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1396] Step 1:

[1397] The device collects the user's speech and converts it into text data using a speech recognition function. The input for this process is the user's speech, and the output is text data.

[1398] Step 2:

[1399] The device sends the collected text data to the AI ​​avatar. The AI ​​avatar generates appropriate dialogue content based on this text data. It uses a generative AI model to generate dialogue based on the prompt text. The input is text data and the output is dialogue content. Specifically, when the user says, "Tell me the next step," the AI ​​avatar uses the generative AI model to answer, "Yes, the next step is assembling the parts."

[1400] Step 3:

[1401] The device outputs the generated dialogue content to the user as voice or text. The input for this process is the dialogue content data received from the AI ​​avatar, and the output is the dialogue result presented to the user. Specifically, the text generated using speech synthesis technology is played aloud or displayed on the smart glasses display.

[1402] Step 4:

[1403] The sensor unit continuously collects environmental data and user health data. This includes various sensors such as temperature sensors and heart rate monitors. The input of this process is the data obtained from the various sensors, and the output is the collected environmental data and health data.

[1404] Step 5:

[1405] The sensor unit sends the collected data to a server. The server stores the received data in a database and analyzes it in real time. The input to this process is the data sent from the sensor unit, and the output is the stored data and the analysis results. For example, if the temperature exceeds the allowable range within a certain period of time, this information is recorded.

[1406] Step 6:

[1407] The server sends a notification if an abnormality is detected based on the analysis results. For example, if the temperature in the work environment exceeds a set standard, the server will send an alert to the safety manager. The input to this process is the analysis result data, and the output is the notification data. Specifically, the alert is sent via email or an in-application notification function.

[1408] Step 7:

[1409] Users (workers and elderly people) receive notifications and alerts and take necessary measures. The input to this process is notification data from the server, and the output is the corresponding action. For example, if an alert about an abnormal temperature is received, the user will temporarily suspend their work and take appropriate measures.

[1410] The above process will create a system that comprehensively improves the quality of life for the elderly and manages worker safety.

[1411] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1412] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[1413] System configuration

[1414] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[1415] 1. User Terminal Embodiments

[1416] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[1417] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[1418] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[1419] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1420] 4. Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[1421] 5. Data collection: The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically sends them to the server.

[1422] 2. Server Implementation

[1423] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1424] 1. Receiving and storing data: The server receives the activity data and emotion data sent from the user terminal and stores it in a database.

[1425] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[1426] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[1427] 3. Sensor Unit Embodiment

[1428] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[1429] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[1430] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1431] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[1432] Specific examples

[1433] Example 1: Everyday usage scenario

[1434] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[1435] Example 2: Anomaly detection and response

[1436] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[1437] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[1438] The processing flow will be explained below.

[1439] Specific processing steps of the program

[1440] User terminal processing

[1441] Step 1:

[1442] The user starts the terminal.

[1443] Step 2:

[1444] The terminal displays a login screen and prompts the user to enter their ID and password.

[1445] Step 3:

[1446] The user enters their ID and password and clicks the login button.

[1447] Step 4:

[1448] The terminal sends the entered ID and password to the server.

[1449] Step 5:

[1450] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[1451] Step 6:

[1452] The device displays an AI avatar based on the data it receives.

[1453] Step 7:

[1454] The user selects the interaction or game.

[1455] Step 8:

[1456] The device executes the selected dialogue or game logic and interacts with the user, capturing the user's voice and facial expressions with a camera.

[1457] Step 9:

[1458] The device transmits the acquired voice and facial expression data to an emotion engine to analyze the user's emotional state.

[1459] Step 10:

[1460] The emotion engine returns the analysis results to the device, and the device dynamically changes the dialogue and game content based on these results.

[1461] Step 11:

[1462] The terminal collects user input data, behavioral data, and emotional data.

[1463] Step 12:

[1464] The terminal transmits the collected data to the server at regular intervals.

[1465] Server Processing

[1466] Step 1:

[1467] The server checks the ID and password sent by the user against the database.

[1468] Step 2:

[1469] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[1470] Step 3:

[1471] The server periodically receives the user's behavioral data and emotion data transmitted from the terminal.

[1472] Step 4:

[1473] The server analyzes the received data and stores it in a database.

[1474] Step 5:

[1475] The server analyzes the user's activity and emotional state based on the stored data.

[1476] Step 6:

[1477] If desired, the server will send notifications and alerts to family members about their activity and emotional state.

[1478] Sensor unit processing

[1479] Step 1:

[1480] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[1481] Step 2:

[1482] The sensor unit transmits the acquired data to the server.

[1483] Step 3:

[1484] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[1485] Step 4:

[1486] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[1487] Step 5:

[1488] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[1489] Step 6:

[1490] The sensor unit follows the received command and performs the required action (for example, locking the door).

[1491] Specific examples

[1492] Example 1: Everyday usage scenario

[1493] Step 1:

[1494] Every morning, the user turns on the tablet and an AI avatar with a photo of their grandchild appears on the screen.

[1495] Step 2:

[1496] The AI ​​avatar speaks, "Good morning, how are you feeling today?"

[1497] Step 3:

[1498] The user responds, "I'm feeling good today." At this time, the device records the user's voice and analyzes their emotional state using an emotion engine.

[1499] Step 4:

[1500] While engaging in normal dialogue, the avatar suggests a simple memory game.

[1501] Step 5:

[1502] The user plays the game and the results and emotional state are recorded on the terminal.

[1503] Step 6:

[1504] The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[1505] Example 2: Anomaly detection and response

[1506] Step 1:

[1507] The sensor unit detects when the house is unlocked and sends this information to a server.

[1508] Step 2:

[1509] The server immediately notifies the family.

[1510] Step 3:

[1511] When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, and the door is locked.

[1512] Step 4:

[1513] If the emotion engine detects anxiety in the elderly person, a notification will be sent to the family member stating that the elderly person is feeling anxious.

[1514] Example 2

[1515] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1516] There is a need to maintain the cognitive functions of the elderly, reduce feelings of loneliness, and ensure safety within the home. However, conventional systems have had difficulty adequately meeting these complex needs. In addition, there is a lack of means to properly analyze the emotional state and promptly notify family members. A new system is needed to solve these issues.

[1517] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1518] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly person to grasp their emotional state, a means for collecting activity data and emotional data of the elderly person and periodically transmitting it to the server, and a means for the server to analyze the received data and send a notification to family members if an abnormality is detected. This makes it possible to maintain the elderly person's cognitive function, reduce loneliness, and ensure safety within the home.

[1519] 1. "AI Avatar" is a virtual character used to interact with seniors, representing photographs of the user's family and friends.

[1520] 2. "Means for conducting dialogue" refers to technical devices and software that use AI avatars to engage in natural language dialogue with older people.

[1521] 3. "Cognitive maintenance strategies" are technologies that provide games and activities designed to stimulate and maintain cognitive abilities in older adults.

[1522] 4. "Means for understanding emotional states" refers to analytical technologies and devices that analyze the voices and facial expressions of elderly people to understand their emotions.

[1523] 5. "Means for collecting activity data and emotional data and periodically transmitting it to a server" refers to technology that uses terminals or sensor units to record the daily activities and emotional states of elderly people and transmits them to a server via a network.

[1524] 6. "The server analyzes the received data and notifies the family if an abnormality is detected" refers to a technology that analyzes data on the server side and immediately notifies the family if an abnormality is detected.

[1525] 7. "Means for linking with home appliances and home systems to detect and notify when locks are left locked or other abnormalities" refers to technology that uses sensor units in the home to monitor the status of home appliances and the home, and notifies if an abnormality is detected.

[1526] 8. "Means for remotely controlling home appliances and home systems" refers to technology that remotely operates and controls home appliances and various home facilities based on instructions from a server.

[1527] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[1528] System configuration

[1529] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[1530] 1. User Terminal Embodiments

[1531] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[1532] 1. Displaying AI avatars:

[1533] The device displays an AI avatar on the screen, which is set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[1534] 2. Interactive features:

[1535] The device uses natural language processing models (e.g., GPT-4) to engage in everyday conversations with the elderly, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[1536] 3. Game Features:

[1537] The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1538] 4. Emotion recognition function:

[1539] The device analyzes the elderly person's voice and facial expressions and uses an emotion engine (e.g., Emotion API) to understand their emotional state.

[1540] 5. Data Collection:

[1541] The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically transmits them to the server.

[1542] 2. Server Implementation

[1543] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1544] 1. Data Receipt and Storage:

[1545] The server receives the activity data and emotion data sent from the user terminal and stores them in a database (e.g., MySQL).

[1546] 2. Data Analysis:

[1547] The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[1548] 3. Notification function:

[1549] The server periodically reports the elderly person's activities to the family, and in the event of an emergency, it sends an alert to prompt immediate action.

[1550] 3. Sensor Unit Embodiment

[1551] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[1552] 1. Monitoring function:

[1553] The sensor unit monitors the house's locks, electricity, temperature, motion detection, and more.

[1554] 2. Data transmission:

[1555] The sensor unit periodically transmits the monitoring results to the server.

[1556] 3. Control function:

[1557] It receives instructions from the server and performs remote control of home appliances and locks / unlocks doors.

[1558] Specific examples

[1559] Example 1: Everyday usage scenario

[1560] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[1561] Example prompt sentence:

[1562] A user turns on the tablet, and an AI avatar with a photo of their grandchild speaks to them, saying, "Good morning, how are you feeling today?" When the user replies, "I'm feeling fine today," explain the system that performs voice analysis, records their emotional state, and sends the result to a server.

[1563] Example 2: Anomaly detection and response

[1564] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[1565] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[1566] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1567] Step 1: Start the user device

[1568] What it does: A user presses the power button on the tablet to start up the device.

[1569] Input: User presses and holds the power button.

[1570] Output: The device's home screen is displayed.

[1571] Specific behavior: The user presses and holds the tablet's power button, the boot screen appears, and then the home screen appears.

[1572] Step 2: Displaying the AI ​​Avatar

[1573] How it works: The device automatically launches the AI ​​avatar app from the home screen and displays an AI avatar on the screen, set to a photo of the elderly person's family or friends.

[1574] Enter: the app icon on your home screen.

[1575] Output: An AI avatar is displayed.

[1576] What it does: The device automatically launches the application and displays an AI avatar with a photo of your grandchild.

[1577] Step 3: Initiating everyday conversations and recognizing emotions

[1578] How it works: The device uses a natural language processing model (e.g., GPT-4) to have an AI avatar say, "Good morning, how are you feeling today?" The device then performs a voice analysis of the user's response and uses emotion recognition technology to determine their emotional state.

[1579] Input: A prompt to the generative AI model: "Good morning, how are you feeling today?" and the user's response.

[1580] Output: Emotional state (e.g., "Energetic").

[1581] Specific operation: The AI ​​avatar asks, "Good morning, how are you feeling today?", and the elderly person replies, "I'm feeling fine today." The device records the answer and uses its emotion engine to interpret it as "feeling fine."

[1582] Step 4: Propose and play the game

[1583] How it works: The device uses an AI avatar to ask the elderly, "Would you like to play a memory game today?" If the elderly agrees, the device starts the memory game.

[1584] Input: The game suggestion prompt, "Would you like to play a memory game today?" and the user's response.

[1585] Output: Game result.

[1586] Specific operation: The AI ​​avatar suggests, "Would you like to play a memory game today?" If the elderly person responds, "Sure," a simple memory game will appear on the screen.

[1587] Step 5: Collect and send data

[1588] How it works: The device collects data such as conversation content, game results, and emotion recognition results, and sends the collected data to a server via Wi-Fi.

[1589] Input: Dialogue recording, game results, emotion recognition results.

[1590] Output: Sending data to the server.

[1591] Specific operation: The device stores the conversation records, game results, and emotional state as data, and transmits them to the server via Wi-Fi.

[1592] Step 6: Server receives and stores data

[1593] Operation: The server receives data sent from the device and stores it in a database (e.g., MySQL).

[1594] Input: Data sent from the terminal.

[1595] Output: Save to database.

[1596] What happens: The server stores all received data as entries in a MySQL database.

[1597] Step 7: Data analysis and notification by the server

[1598] What it does: The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected.

[1599] Input: Record data from the database.

[1600] Output: Notification to family.

[1601] Specific operation: The server analyzes the data, and when it detects a decrease in activity, it sends a notification email to the family saying, "The elderly person's activity is decreasing."

[1602] Step 8: Anomaly detection and response

[1603] How it works: The sensor unit detects that the house doors are unlocked and sends this information to the server. The server immediately notifies the family, who then issue a command to "lock" the door via their smartphone. The sensor unit then remotely locks the door based on the received command.

[1604] Input: Anomaly detection data from the sensor unit, instructions from family members.

[1605] Output: Locks the door and notifies family members of the anxiety state.

[1606] Specific operation: The sensor detects that the door to the house is open and sends the information to the server. When a family member presses the "lock" button on their smartphone, the server sends a command to the sensor unit, which then locks the front door. If the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious."

[1607] These are the specific steps in the program processing of this system, which will help maintain cognitive function, reduce loneliness, and improve safety for the elderly.

[1608] (Application example 2)

[1609] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1610] In order to improve the quality of life of the elderly, it is important to maintain cognitive function and ensure their safety. However, because the time that family members can directly interact with the elderly is limited, a system that can remotely grasp the elderly's situation and respond appropriately is required. It is also important to analyze the elderly's emotional state and respond accordingly. Given this background, it is necessary to provide a system that can monitor the safety and emotional state of the elderly in real time and respond quickly to abnormalities or changes in emotion.

[1611] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1612] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly to grasp their emotional state, a means for monitoring the elderly's environment with a camera in the smart glasses, and a means for sending an alert to relatives when an abnormality is detected. This not only makes it possible to maintain the elderly's cognitive function and ensure their safety, but also enables immediate response to changes in their emotional state.

[1613] An "AI avatar" is a digital character equipped with artificial intelligence to interact with the elderly and display a photo of a relative or acquaintance.

[1614] "Means for displaying photographs of relatives or acquaintances" refers to technology that displays photographs of family members or friends on a digital display to help elderly people feel closer to them.

[1615] "Means for responding" is a function that uses recognition technology and voice synthesis technology to enable elderly people and AI avatars to communicate through natural conversation.

[1616] "Means for maintaining cognitive function in the elderly through entertainment" is a technology that aims to maintain and improve cognitive function in the elderly by providing activities such as simple games and puzzles.

[1617] "Means for collecting activity data and transmitting it to a central processing unit in real time" refers to a function that uses sensors and devices to record the elderly person's daily activities, voice, facial expressions, etc., and transmits the data to a server in real time.

[1618] The "means of notifying relatives of activity data" is a function that automatically analyzes collected data on elderly people and notifies relatives of abnormalities and necessary information.

[1619] The "means for detecting and notifying when someone forgets to lock their home or when there is an abnormality" is a technology that constantly monitors the elderly person's home appliances and home systems, and notifies relatives when it detects that someone has forgotten to lock their home or when there is an abnormality.

[1620] "Means for remotely controlling home appliances and home systems" refers to technology that allows relatives to remotely control home appliances and systems in the elderly person's home via devices such as smartphones.

[1621] "Means for monitoring the environment of the elderly using a camera in smart glasses" is a function that uses a camera installed in smart glasses to monitor the environment around the elderly in real time.

[1622] "Means for analyzing voice and facial expressions to grasp emotional states" refers to technology that analyzes voice and facial expression data of elderly people and uses an emotion engine to estimate and grasp their emotional states.

[1623] The "means for sending an alert to relatives when an abnormality is detected" is a system that sends an alert message to relatives when an abnormality is detected as a result of various monitoring performed on the elderly person.

[1624] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. In particular, this system has the ability to analyze the emotional state of the elderly using an emotion engine and utilize the analysis data. Specific embodiments are described below.

[1625] System configuration

[1626] The embodiment of the present invention is broadly composed of the following components.

[1627] 1. User Terminal Embodiments

[1628] The user devices are smartphones or smart glasses used by elderly people. The devices have the following main functions:

[1629] Display of AI avatar: The device displays an AI avatar on the display, which is set to a photo of the elderly person's relatives or acquaintances, giving the user a sense of familiarity.

[1630] Voice response function: The device responds naturally to the elderly through an AI avatar, helping to reduce feelings of loneliness and stimulate cognitive function.

[1631] Activity provision function: The device provides simple activities such as puzzles and games, which further maintain and improve cognitive function in the elderly.

[1632] Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[1633] Data collection and transmission: Responses, game results, and emotion recognition results are collected as data and periodically sent to the server.

[1634] 2. Server Implementation

[1635] The server is the center of the system, centrally managing elderly people's activity data and providing necessary notifications and analysis.

[1636] Data reception and storage: The server receives the activity data and emotion data sent from the user terminal and stores them in a database.

[1637] Data analysis: The server analyzes the collected data and sends notifications to relatives if any abnormalities are detected.

[1638] Notification function: The server periodically reports the elderly person's activity and emotional state to relatives, and sends immediate alerts in case of an emergency.

[1639] 3. Sensor Unit Embodiment

[1640] The sensor unit integrates various sensors in the home and monitors the status of home appliances and systems.

[1641] Monitoring functions: Sensors monitor locks, electricity, temperature, motion, and more within the home.

[1642] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1643] Control function: Receives instructions from the server and remotely controls and locks / unlocks home appliances.

[1644] Specific examples

[1645] Example 1: Everyday usage scenario

[1646] The user (elderly person) turns on the smart glasses every morning, and an AI avatar with a photo of a relative's face appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While making a normal response, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the relative of "today's activity status and emotional state."

[1647] Example 2: Anomaly detection and response

[1648] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies relatives. When a relative issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. If the emotion engine detects anxiety in the elderly person at this time, it also sends a notification to the relative that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also enables responses that take their emotional state into consideration.

[1649] Prompt Sentence Examples

[1650] "You will develop a function to detect falls among elderly people based on image data of high-risk abnormal behavior and falls, and image data of normal behavior. You will also create a system to analyze the emotional state of elderly people from images of their facial expressions, and collect and store that data."

[1651] In this way, the present invention provides a system that enables the maintenance of cognitive function and safety of elderly people, and instantly grasps their emotional state using an emotion engine.

[1652] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1653] Step 1:

[1654] The user turns on the smart glasses

[1655] Input: User action of pressing the power button on the smart glasses.

[1656] Operation: The smart glasses start up and each module of the system is initialized.

[1657] Output: The camera and microphone on the smart glasses start working, and the AI ​​avatar appears on the display.

[1658] Step 2:

[1659] The device captures the voice and facial expressions of the elderly person.

[1660] Input: Video data from the camera and audio data from the microphone of the smart glasses

[1661] How it works: The camera recognizes the elderly person's face and the microphone records their voice. This data is temporarily stored on the device.

[1662] Output: Video frames and audio clips of the elderly

[1663] Step 3:

[1664] The device analyzes your emotional state

[1665] Input: Captured video frames and audio clips

[1666] How it works: The device inputs this data into an emotion engine, which performs facial expression analysis and voice analysis. It then uses an emotion recognition model (e.g., a generative AI model using TensorFlow) to estimate the user's emotional state (e.g., happiness, sadness, anxiety, etc.).

[1667] Output: Estimated emotional state

[1668] Step 4:

[1669] The device responds to the elderly person through an AI avatar.

[1670] Input: The user's previous voice response and detected emotional state

[1671] How it works: The AI ​​avatar generates dialogue that responds to the user with the following responses, including the content of the response and speech synthesis.

[1672] Output: The following dialogue and synthesized speech

[1673] Step 5:

[1674] The device sends activity data to the server

[1675] Input: Responses, emotional state analysis results, elderly person activity data (game results, etc.)

[1676] How it works: The device packages the data and sends it over the internet to a server

[1677] Output: Elderly activity data sent to the server

[1678] Step 6:

[1679] The server receives and analyzes the data

[1680] Input: Elderly person's activity data sent from the device

[1681] How it works: The server stores the received data in a database and analyzes it for any abnormalities. If an abnormality is detected, an alert is sent to the relatives.

[1682] Output: Today's activity report or anomaly alert

[1683] Step 7:

[1684] The server monitors the data from the sensor units.

[1685] Input: Environmental data sent from sensor units in the home (temperature, lock status, etc.)

[1686] Operation: The server periodically receives this data and detects abnormal conditions.

[1687] Output: Alert notification when an abnormality occurs, log update when the status is normal

[1688] Step 8:

[1689] A relative remotely controls the device

[1690] Input: Operation command from smartphone (e.g., command to lock the door)

[1691] How it works: The server receives the command and sends appropriate control signals to the sensor units in the home.

[1692] Output: Home appliances and systems operate according to the relatives' instructions.

[1693] The above processing steps realize a system that monitors the living environment and emotional state of the elderly in real time, promptly notifies relatives when an abnormality occurs, and enables remote control.

[1694] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1695] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1696] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1697] [Fourth embodiment]

[1698] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1699] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1700] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1701] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1702] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1703] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1704] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1705] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1706] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1707] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1708] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1709] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1710] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1711] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Specific embodiments for implementation are described in detail below.

[1712] System configuration

[1713] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[1714] 1. User Terminal Embodiments

[1715] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[1716] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[1717] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[1718] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1719] 4. Data collection: The device collects data on interactions and game results and periodically sends it to the server.

[1720] 2. Server Implementation

[1721] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1722] 1. Receiving and storing data: The server receives the activity data sent from the user terminal and stores it in a database.

[1723] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a drop in activity over a certain period of time.

[1724] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[1725] 3. Sensor Unit Embodiment

[1726] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[1727] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[1728] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1729] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[1730] Specific examples

[1731] Example 1: Everyday usage scenario

[1732] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[1733] Example 2: Anomaly detection and response

[1734] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. This ensures the safety of the elderly and allows the family to go about their daily lives with peace of mind.

[1735] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety.

[1736] The processing flow will be explained below.

[1737] Specific processing steps of the program

[1738] User terminal processing

[1739] Step 1:

[1740] The user starts the terminal.

[1741] Step 2:

[1742] The terminal displays a login screen and prompts the user to enter their ID and password.

[1743] Step 3:

[1744] The user enters their ID and password and clicks the login button.

[1745] Step 4:

[1746] The terminal sends the entered ID and password to the server.

[1747] Step 5:

[1748] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[1749] Step 6:

[1750] The device displays an AI avatar based on the data it receives.

[1751] Step 7:

[1752] The user selects the interaction or game.

[1753] Step 8:

[1754] The device executes the selected dialogue or game logic and interacts with the user.

[1755] Step 9:

[1756] The device collects user input and behavioral data.

[1757] Step 10:

[1758] The terminal transmits the collected data to the server at regular intervals.

[1759] Server Processing

[1760] Step 1:

[1761] The server checks the ID and password sent by the user against the database.

[1762] Step 2:

[1763] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[1764] Step 3:

[1765] The server periodically receives the user's behavior data transmitted from the terminal.

[1766] Step 4:

[1767] The server analyzes the received data and stores it in a database.

[1768] Step 5:

[1769] The server analyzes the user's activity based on the stored data.

[1770] Step 6:

[1771] If necessary, the server will send activity notifications and alerts to family members.

[1772] Sensor unit processing

[1773] Step 1:

[1774] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[1775] Step 2:

[1776] The sensor unit transmits the acquired data to the server.

[1777] Step 3:

[1778] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[1779] Step 4:

[1780] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[1781] Step 5:

[1782] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[1783] Step 6:

[1784] The sensor unit follows the received command and performs the required action (for example, locking the door).

[1785] Example 1

[1786] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1787] Declining cognitive function, increasing loneliness, and safety issues significantly impair the quality of life of older adults. Effective and sustainable solutions to these problems are needed. However, existing technologies have not provided a system that comprehensively addresses the complex issues that older adults face in their daily lives. In particular, there is a need for a system that simultaneously maintains and improves cognitive function, ensures safety, and reduces loneliness.

[1788] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1789] In this invention, the server includes a means for displaying an AI avatar and photos of the elderly person's family or friends, a means for interacting with the elderly person through the AI ​​avatar, a means for maintaining and improving the elderly person's cognitive function through interactions and games, a means for collecting the elderly person's activity data and periodically transmitting it to the server, a means for notifying family members of the elderly person's activity data, a means for linking with home appliances and housing systems to detect and notify if the elderly person has forgotten to lock the door or if there is an abnormality, and a means for remotely operating the elderly person's home appliances and housing systems. This makes it possible to maintain and improve the elderly person's cognitive function, reduce loneliness, and ensure their safety.

[1790] An "AI avatar" is a virtual character equipped with artificial intelligence, designed to stimulate cognitive function and reduce feelings of loneliness through dialogue and communication with the elderly.

[1791] The "photos" are facial images of the elderly person's family or friends, intended to give the elderly a sense of familiarity and security.

[1792] The "dialogue function" is a function that allows an AI avatar to have everyday conversations with elderly people and maintain and improve their cognitive function.

[1793] The "game function" allows seniors to participate in activities such as simple puzzles and memory games, with the aim of further maintaining and improving cognitive function.

[1794] "Activity data" refers to information such as the elderly person's daily behavior, conversations, and game results, and is analyzed to detect abnormalities.

[1795] The "server" is a computer system that functions as the central part of the system, centrally managing elderly people's activity data and conducting analysis and notifications.

[1796] "Data collection" is the process by which user terminals and sensor units gather information about the elderly person's daily activities and use it for later analysis.

[1797] The "notification function" is a function that reports activity data and notifies abnormalities to the elderly person's family members, and sends alerts using email, SMS, push notifications, etc.

[1798] "Home devices" refers to electrical appliances and equipment used in the homes of seniors that can be remotely controlled and monitored.

[1799] "Housing systems" refer to facilities that provide basic functions such as locks, lighting, and temperature control for elderly people's homes, and are monitored and controlled through sensors.

[1800] A "sensor unit" refers to an individual sensor device that is used to continuously monitor the status of a home appliance or home system.

[1801] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function and improve safety of the elderly through the management of AI avatars and home appliances. Specific embodiments for carrying out the invention are described in detail below.

[1802] System configuration

[1803] This system mainly consists of the following components: user terminals, servers, sensor units, and a communication network.

[1804] 1. User Terminal Embodiments

[1805] The user devices consist of smartphones and tablets used by the elderly. The devices have the following functions:

[1806] 1. Displaying AI avatars

[1807] The device launches a built-in app and displays an AI avatar on the screen, using photos of the elderly person's family and friends to create a sense of familiarity.

[1808] 2. Interactive features

[1809] The device engages in everyday conversations with the elderly through an AI avatar, generating dialogue using a natural language processing engine (e.g., GPT-4) to reduce feelings of loneliness and stimulate cognitive function.

[1810] 3. Game Features

[1811] The device provides activities such as simple puzzles and memory games, and is implemented using Unity and React Native to maintain and improve cognitive function in the elderly.

[1812] 4. Data Collection

[1813] The device collects activity data such as conversation results and game results, and periodically sends it to the server using the HTTP protocol.

[1814] 2. Server Implementation

[1815] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1816] 1. Data Receipt and Storage

[1817] The server receives activity data sent from the user's device and stores it in a database (e.g., MySQL, MongoDB).

[1818] 2. Data Analysis

[1819] The server analyzes the collected data using machine learning algorithms (e.g., Scikit-learn, TensorFlow), and if an anomaly is detected, it sends a notification to the family.

[1820] 3. Notification function

[1821] The server periodically reports the elderly person's activities to the family, and in case of an emergency, it sends an immediate notification via email, SMS, or push notification (such as Firebase Cloud Messaging).

[1822] 3. Sensor Unit Embodiment

[1823] The sensor unit integrates various sensors in the home to monitor the status of home appliances and residential systems, and can take action if necessary.

[1824] 1. Monitoring function

[1825] Monitor locks, lights, temperature, and activity of home appliances and systems. Collect information using IoT devices (e.g., Arduino, Raspberry Pi).

[1826] 2. Data transmission

[1827] The monitoring results are sent to the server periodically using MQTT as the communication protocol.

[1828] 3. Control Function

[1829] According to instructions from the server, it remotely controls household devices and locks / unlocks doors. Instructions given by family members using their smartphones are transmitted to each sensor via the server.

[1830] Specific examples

[1831] Example 1: Everyday usage scenario

[1832] Every morning, the user (elderly person) turns on their smartphone, and an AI avatar holding a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person replies, "I'm feeling fine today." While continuing the normal dialogue, the avatar suggests, "Would you like to play a little memory game?" The user plays the game, and the results are recorded on the device and periodically sent to the server. The server analyzes the results and notifies the family of "today's activity status."

[1833] Example 2: Anomaly detection and response

[1834] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately notifies the family, and when the family member issues a command to "lock" the door from their smartphone, the command is transmitted via the server to the sensor unit, which then locks the door. This ensures the safety of the elderly, allowing the family to go about their daily lives with peace of mind.

[1835] Prompt Sentence Examples

[1836] Prompt 1: Use a user terminal

[1837] "Please tell us the process steps and specific actions of how an elderly person would turn on their smartphone every morning and interact with the AI ​​avatar displayed on the screen."

[1838] Prompt 2: Anomaly detection

[1839] "Please explain in detail the specific steps and technology used by the sensor unit to notify family members if their home is left unlocked."

[1840] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1841] Step 1: Start the user device and display the AI ​​avatar

[1842] 1. The user starts up the smartphone

[1843] Input: User powers on the smartphone

[1844] Operation: Initialize the smartphone and transition to the home screen

[1845] Output: Home screen display

[1846] 2. Launch the AI ​​avatar app

[1847] Input: Tap the AI ​​Avatar app icon on the home screen

[1848] What it does: The app launches and displays an avatar showing a photo of the senior's family or friend.

[1849] Output: Display of AI avatar

[1850] Step 2: Execute the interaction

[1851] 1. Initial interaction with an AI avatar

[1852] Input: AI avatar app launch complete

[1853] Action: AI avatar says "Good morning, how are you feeling today?"

[1854] Output: Questions for the elderly

[1855] 2. Receiving and analyzing elderly people's responses

[1856] Input: Elderly person responds "I'm fine today"

[1857] How it works: The device uses a speech recognition feature (such as the Google Speech-to-Text API) to convert the voice data into text, and then uses a generative AI model (such as GPT-4) to analyze the response.

[1858] Output: Generate appropriate answers based on the analysis results

[1859] Step 3: Providing game functionality

[1860] 1. Game suggestions

[1861] Input:Continue the dialogue

[1862] How it works: The AI ​​avatar suggests, "Would you like to play a little memory game?"

[1863] Output: Game suggestions for seniors

[1864] 2. Running the game

[1865] Input: Seniors start the game

[1866] Action: The device displays a memory game using Unity and React Native and waits for user interaction.

[1867] Output: Displaying the game interface and collecting user operation data

[1868] Step 4: Collect data and send it to the server

[1869] 1. Data collection

[1870] Input: Interaction results and game results

[1871] What it does: The device temporarily stores the collected data in its internal storage.

[1872] Output: Temporarily saved activity data

[1873] 2. Data transmission

[1874] Input: Temporarily saved activity data

[1875] What happens: The device sends data to the server using the HTTP protocol.

[1876] Output: Activity data sent to the server

[1877] Step 5: Receiving and analyzing data on the server

[1878] 1. Receiving Data

[1879] Input: Activity data sent from the terminal

[1880] What happens: The server receives an HTTP request and retrieves the data.

[1881] Output: Received activity data

[1882] 2. Data storage

[1883] Input: Received activity data

[1884] What happens: The server saves the data to a database (MySQL or MongoDB).

[1885] Output: Activity data stored in the database

[1886] 3. Data Analysis

[1887] Input: Activity data stored in the database

[1888] How it works: The server analyzes the data using machine learning algorithms (Scikit-learn or TensorFlow) and detects anomalies.

[1889] Output: Analysis results and anomaly detection information

[1890] Step 6: Notify family members

[1891] 1. Generate a notification

[1892] Input: Analysis results and anomaly detection information

[1893] Action: The server generates a notification message for the family.

[1894] Output: Notification message

[1895] 2. Sending notifications

[1896] Input: Notification message

[1897] How it works: The server sends notifications via email, SMS, or push notifications (Firebase Cloud Messaging).

[1898] Output: Notification sent to family members

[1899] Step 7: Sensor-by-sensor monitoring and control

[1900] 1. Performing monitoring functions

[1901] Input: Status of home appliances and systems

[1902] Operation: Sensor units constantly monitor locks, lighting, temperature, and operation. Data is acquired using IoT devices (Arduino and Raspberry Pi).

[1903] Output: Monitoring results

[1904] 2. Data Transmission

[1905] Input: Monitoring results

[1906] Operation: The sensor unit sends the monitoring results to the server using the MQTT protocol.

[1907] Output: Monitoring data sent to the server

[1908] 3. Receiving control instructions

[1909] Input: Control instructions from family members or the server

[1910] Operation: The server sends control instructions to each sensor, and the sensor operates the home appliance.

[1911] Output: Operation results of home appliances

[1912] (Application example 1)

[1913] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1914] In addition to systems that improve the quality of life for the elderly, there is a need for means to manage worker safety. In particular, in work environments such as factories, it is necessary to ensure safety by timely monitoring the physical condition of workers and the surrounding environment. In conventional systems, the elderly management system and the worker safety management system exist independently and require separate management, making efficient integrated management difficult. Furthermore, if real-time data collection and analysis results are not immediately notified, there is a risk of delays in appropriate response. The present invention aims to solve these problems and provide an integrated system that simultaneously improves the safety and quality of life of both the elderly and workers.

[1915] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1916] In this invention, the server includes a means for collecting elderly people's activity data and transmitting it to the server in real time, a means for collecting sensor data and transmitting it to the server, and a means for analyzing the collected data and notifying the safety manager. This improves the quality of life of elderly people and makes it possible to constantly monitor the physical condition and environmental conditions of workers and ensure their safety.

[1917] An "AI avatar" is a virtual character that uses artificial intelligence technology and is a means of interacting with the elderly or workers.

[1918] "Photos of family or friends" are images used to display AI avatars to increase familiarity with the elderly.

[1919] "Means for dialogue" refers to a mechanism that allows users and AI avatars to communicate through voice or text.

[1920] "Methods for maintaining cognitive function" are methods of providing mental stimulation through dialogue, games, etc. to maintain the cognitive abilities of elderly people.

[1921] "Means for collecting activity data and transmitting it to a server in real time" refers to technology that continuously collects activity information on elderly people and workers and instantly transmits that information to a cloud server.

[1922] The "means of notifying family members of activity data" refers to a method for informing family members of the elderly person's activity status.

[1923] "Means for detecting and notifying of forgotten locks and abnormalities" refers to technology that uses sensors in the home or work environment to monitor the status of keys and abnormalities, and issues alerts as necessary.

[1924] "Means for remotely controlling home appliances and home systems" refers to technology for remotely controlling home appliances and home systems via the Internet.

[1925] "Means for monitoring physical condition and environmental conditions and issuing warnings if abnormalities are detected" refers to a method in which sensors are used to detect abnormalities in workers' health and working environment and send appropriate alerts.

[1926] "Means for analyzing collected data and notifying safety managers" refers to methods for analyzing data obtained from sensors and reporting important information to safety managers in real time.

[1927] This invention is a system that comprehensively improves the quality of life for the elderly and manages worker safety. The system is composed of an AI avatar, a user terminal, a server, a sensor unit, and a communication network. Specific embodiments are described below.

[1928] System configuration

[1929] 1. User Terminal Embodiments

[1930] The user device can be, for example, a smartphone or tablet used by the elderly, or smart glasses used by workers. The device has the following functions:

[1931] Displaying AI avatars: The device displays AI avatars on the screen. The AI ​​avatars are set with photos of the elderly's family and friends, making them feel closer to the elderly. They can also be used to assist workers.

[1932] Conversational function: The device will engage in everyday conversations with the user through an AI avatar, thereby reducing feelings of loneliness among the elderly and stimulating their cognitive functions. It will also provide work instructions to workers.

[1933] Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1934] Data collection: The terminal collects the results of conversations and games, as well as the worker's physical condition and work data, and periodically transmits them to the server.

[1935] 2. Server Implementation

[1936] The server is the center of the system, centrally managing user activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1937] Data reception and storage: The server receives the activity data sent from the user terminals and sensor units and stores it in a database.

[1938] Data Analysis: The server analyzes the collected data and sends notifications to family members or safety managers if anomalies are detected, such as a drop in activity over a certain period of time.

[1939] Notification function: The server periodically reports user activity to family members and safety managers. In the event of an emergency, an alert is sent to prompt immediate action.

[1940] 3. Sensor Unit Embodiment

[1941] The sensor unit integrates various sensors in the home or work environment to monitor their status and take action if necessary.

[1942] Monitoring function: The sensor unit monitors temperature, movement, lock status, etc. within the home or work environment.

[1943] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1944] Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking.

[1945] Specific examples

[1946] Example 1: Daily usage scenario for elderly people

[1947] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" to which the elderly person responds, "I'm feeling fine today." While engaging in normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status."

[1948] Example 2: Worker Safety Management Scenario

[1949] A worker puts on the smart glasses and begins work. An AI avatar speaks to the worker through the glasses, asking, "What's the next step?" If the worker replies, "Assembly," the avatar displays the correct work procedure. The sensor unit monitors the temperature of the work environment and the worker's physical condition, and immediately displays a warning if there is an abnormality. The data is sent to a server and analyzed as appropriate. Based on the analysis results, a notification is sent to the safety manager.

[1950] A prompt sentence from a generative AI model as an example of use

[1951] Prompt statement:

[1952] "We are considering a voice assistant that will provide safety warnings while working in a factory. Please provide example dialogue."

[1953] As described above, this system not only improves the quality of life for the elderly, but also ensures the safety of working people, providing an integrated solution that meets the needs of both.

[1954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1955] Step 1:

[1956] The device collects the user's speech and converts it into text data using a speech recognition function. The input for this process is the user's speech, and the output is text data.

[1957] Step 2:

[1958] The device sends the collected text data to the AI ​​avatar. The AI ​​avatar generates appropriate dialogue content based on this text data. It uses a generative AI model to generate dialogue based on the prompt text. The input is text data and the output is dialogue content. Specifically, when the user says, "Tell me the next step," the AI ​​avatar uses the generative AI model to answer, "Yes, the next step is assembling the parts."

[1959] Step 3:

[1960] The device outputs the generated dialogue content to the user as voice or text. The input for this process is the dialogue content data received from the AI ​​avatar, and the output is the dialogue result presented to the user. Specifically, the text generated using speech synthesis technology is played aloud or displayed on the smart glasses display.

[1961] Step 4:

[1962] The sensor unit continuously collects environmental data and user health data. This includes various sensors such as temperature sensors and heart rate monitors. The input of this process is the data obtained from the various sensors, and the output is the collected environmental data and health data.

[1963] Step 5:

[1964] The sensor unit sends the collected data to a server. The server stores the received data in a database and analyzes it in real time. The input to this process is the data sent from the sensor unit, and the output is the stored data and the analysis results. For example, if the temperature exceeds the allowable range within a certain period of time, this information is recorded.

[1965] Step 6:

[1966] The server sends a notification if an abnormality is detected based on the analysis results. For example, if the temperature in the work environment exceeds a set standard, the server will send an alert to the safety manager. The input to this process is the analysis result data, and the output is the notification data. Specifically, the alert is sent via email or an in-application notification function.

[1967] Step 7:

[1968] Users (workers and elderly people) receive notifications and alerts and take necessary measures. The input to this process is notification data from the server, and the output is the corresponding action. For example, if an alert about an abnormal temperature is received, the user will temporarily suspend their work and take appropriate measures.

[1969] The above process will create a system that comprehensively improves the quality of life for the elderly and manages worker safety.

[1970] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1971] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[1972] System configuration

[1973] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[1974] 1. User Terminal Embodiments

[1975] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[1976] 1. Displaying an AI avatar: The device will display an AI avatar on the screen, which will be set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[1977] 2. Conversation function: The device can hold daily conversations with the elderly through an AI avatar, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[1978] 3. Game function: The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[1979] 4. Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[1980] 5. Data collection: The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically sends them to the server.

[1981] 2. Server Implementation

[1982] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[1983] 1. Receiving and storing data: The server receives the activity data and emotion data sent from the user terminal and stores it in a database.

[1984] 2. Data Analysis: The server analyzes the collected data and sends notifications to family members if any anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[1985] 3. Notification function: The server periodically reports the elderly person's activities to the family, and sends an alert in case of an emergency, prompting immediate action.

[1986] 3. Sensor Unit Embodiment

[1987] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[1988] 1. Monitoring function: The sensor unit monitors house keys, electricity, temperature, motion detection, etc.

[1989] 2. Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[1990] 3. Control function: Receives instructions from the server and performs remote control of home appliances and locking / unlocking of doors.

[1991] Specific examples

[1992] Example 1: Everyday usage scenario

[1993] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[1994] Example 2: Anomaly detection and response

[1995] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[1996] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[1997] The processing flow will be explained below.

[1998] Specific processing steps of the program

[1999] User terminal processing

[2000] Step 1:

[2001] The user starts the terminal.

[2002] Step 2:

[2003] The terminal displays a login screen and prompts the user to enter their ID and password.

[2004] Step 3:

[2005] The user enters their ID and password and clicks the login button.

[2006] Step 4:

[2007] The terminal sends the entered ID and password to the server.

[2008] Step 5:

[2009] The server authenticates the received ID and password, and if authentication is successful, returns the user's data and AI avatar settings information to the device.

[2010] Step 6:

[2011] The device displays an AI avatar based on the data it receives.

[2012] Step 7:

[2013] The user selects the interaction or game.

[2014] Step 8:

[2015] The device executes the selected dialogue or game logic and interacts with the user, capturing the user's voice and facial expressions with a camera.

[2016] Step 9:

[2017] The device transmits the acquired voice and facial expression data to an emotion engine to analyze the user's emotional state.

[2018] Step 10:

[2019] The emotion engine returns the analysis results to the device, and the device dynamically changes the dialogue and game content based on these results.

[2020] Step 11:

[2021] The terminal collects user input data, behavioral data, and emotional data.

[2022] Step 12:

[2023] The terminal transmits the collected data to the server at regular intervals.

[2024] Server Processing

[2025] Step 1:

[2026] The server checks the ID and password sent by the user against the database.

[2027] Step 2:

[2028] If authentication is successful, the server returns the user's data and AI avatar configuration information to the device.

[2029] Step 3:

[2030] The server periodically receives the user's behavioral data and emotion data transmitted from the terminal.

[2031] Step 4:

[2032] The server analyzes the received data and stores it in a database.

[2033] Step 5:

[2034] The server analyzes the user's activity and emotional state based on the stored data.

[2035] Step 6:

[2036] If desired, the server will send notifications and alerts to family members about their activity and emotional state.

[2037] Sensor unit processing

[2038] Step 1:

[2039] The sensor unit checks sensors such as house keys, electricity, temperature, and motion detection at regular intervals.

[2040] Step 2:

[2041] The sensor unit transmits the acquired data to the server.

[2042] Step 3:

[2043] If the sensor unit detects an abnormality, it immediately sends the information to the server.

[2044] Step 4:

[2045] The server analyzes the received sensor data and notifies the family if any abnormalities are detected.

[2046] Step 5:

[2047] When a family member requests remote control, the server receives the instruction and sends a command to the sensor unit.

[2048] Step 6:

[2049] The sensor unit follows the received command and performs the required action (for example, locking the door).

[2050] Specific examples

[2051] Example 1: Everyday usage scenario

[2052] Step 1:

[2053] Every morning, the user turns on the tablet and an AI avatar with a photo of their grandchild appears on the screen.

[2054] Step 2:

[2055] The AI ​​avatar speaks, "Good morning, how are you feeling today?"

[2056] Step 3:

[2057] The user responds, "I'm feeling good today." At this time, the device records the user's voice and analyzes their emotional state using an emotion engine.

[2058] Step 4:

[2059] While engaging in normal dialogue, the avatar suggests a simple memory game.

[2060] Step 5:

[2061] The user plays the game and the results and emotional state are recorded on the terminal.

[2062] Step 6:

[2063] The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[2064] Example 2: Anomaly detection and response

[2065] Step 1:

[2066] The sensor unit detects when the house is unlocked and sends this information to a server.

[2067] Step 2:

[2068] The server immediately notifies the family.

[2069] Step 3:

[2070] When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, and the door is locked.

[2071] Step 4:

[2072] If the emotion engine detects anxiety in the elderly person, a notification will be sent to the family member stating that the elderly person is feeling anxious.

[2073] Example 2

[2074] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2075] There is a need to maintain the cognitive functions of the elderly, reduce feelings of loneliness, and ensure safety within the home. However, conventional systems have had difficulty adequately meeting these complex needs. In addition, there is a lack of means to properly analyze the emotional state and promptly notify family members. A new system is needed to solve these issues.

[2076] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2077] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly person to grasp their emotional state, a means for collecting activity data and emotional data of the elderly person and periodically transmitting it to the server, and a means for the server to analyze the received data and send a notification to family members if an abnormality is detected. This makes it possible to maintain the elderly person's cognitive function, reduce loneliness, and ensure safety within the home.

[2078] 1. "AI Avatar" is a virtual character used to interact with seniors, representing photographs of the user's family and friends.

[2079] 2. "Means for conducting dialogue" refers to technical devices and software that use AI avatars to engage in natural language dialogue with older people.

[2080] 3. "Cognitive maintenance strategies" are technologies that provide games and activities designed to stimulate and maintain cognitive abilities in older adults.

[2081] 4. "Means for understanding emotional states" refers to analytical technologies and devices that analyze the voices and facial expressions of elderly people to understand their emotions.

[2082] 5. "Means for collecting activity data and emotional data and periodically transmitting it to a server" refers to technology that uses terminals or sensor units to record the daily activities and emotional states of elderly people and transmits them to a server via a network.

[2083] 6. "The server analyzes the received data and notifies the family if an abnormality is detected" refers to a technology that analyzes data on the server side and immediately notifies the family if an abnormality is detected.

[2084] 7. "Means for linking with home appliances and home systems to detect and notify when locks are left locked or other abnormalities" refers to technology that uses sensor units in the home to monitor the status of home appliances and the home, and notifies if an abnormality is detected.

[2085] 8. "Means for remotely controlling home appliances and home systems" refers to technology that remotely operates and controls home appliances and various home facilities based on instructions from a server.

[2086] The present invention is a system for improving the quality of life of the elderly. It aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. Furthermore, by combining it with an emotion engine, the emotional state of the elderly can be analyzed and utilized as data. Specific embodiments for implementation are described in detail below.

[2087] System configuration

[2088] This system mainly consists of the following components: user terminal, server, sensor unit, emotion engine, and communication network.

[2089] 1. User Terminal Embodiments

[2090] The user device is, for example, a smartphone or tablet used by the elderly. The device has the following functions:

[2091] 1. Displaying AI avatars:

[2092] The device displays an AI avatar on the screen, which is set to the face of the elderly person's family or friends, making them feel closer to the elderly.

[2093] 2. Interactive features:

[2094] The device uses natural language processing models (e.g., GPT-4) to engage in everyday conversations with the elderly, thereby reducing their sense of loneliness and stimulating their cognitive functions.

[2095] 3. Game Features:

[2096] The device provides activities such as simple puzzles and memory games, which further maintain and improve the cognitive function of the elderly.

[2097] 4. Emotion recognition function:

[2098] The device analyzes the elderly person's voice and facial expressions and uses an emotion engine (e.g., Emotion API) to understand their emotional state.

[2099] 5. Data Collection:

[2100] The device collects data on the results of conversations and games, as well as emotion recognition results, and periodically transmits them to the server.

[2101] 2. Server Implementation

[2102] The server is the center of the system, centrally managing the elderly's activity data and providing necessary notifications and analysis. Its specific functions are as follows:

[2103] 1. Data Receipt and Storage:

[2104] The server receives the activity data and emotion data sent from the user terminal and stores them in a database (e.g., MySQL).

[2105] 2. Data Analysis:

[2106] The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected, such as a decrease in activity over a certain period of time or a change in emotional state.

[2107] 3. Notification function:

[2108] The server periodically reports the elderly person's activities to the family, and in the event of an emergency, it sends an alert to prompt immediate action.

[2109] 3. Sensor Unit Embodiment

[2110] The sensor unit integrates various sensors in the home to monitor the status of appliances and home systems, and can even take action if necessary.

[2111] 1. Monitoring function:

[2112] The sensor unit monitors the house's locks, electricity, temperature, motion detection, and more.

[2113] 2. Data transmission:

[2114] The sensor unit periodically transmits the monitoring results to the server.

[2115] 3. Control function:

[2116] It receives instructions from the server and performs remote control of home appliances and locks / unlocks doors.

[2117] Specific examples

[2118] Example 1: Everyday usage scenario

[2119] Every morning, the user (elderly person) turns on the tablet, and an AI avatar with a photo of their grandchild appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While engaging in a normal conversation, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which analyzes the data and notifies the family of "today's activity status and emotional state."

[2120] Example prompt sentence:

[2121] A user turns on the tablet, and an AI avatar with a photo of their grandchild speaks to them, saying, "Good morning, how are you feeling today?" When the user replies, "I'm feeling fine today," explain the system that performs voice analysis, records their emotional state, and sends the result to a server.

[2122] Example 2: Anomaly detection and response

[2123] The sensor unit detects that the house is unlocked and sends this information to the server. The server immediately sends a notification to the family. When a family member issues a command to "lock" from their smartphone, the server sends this command to the sensor unit, which then locks the door. At this time, if the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious." This not only ensures the safety of the elderly, but also makes it possible to respond in a way that takes their emotional state into consideration.

[2124] As described above, this system maintains the cognitive function of the elderly, reduces feelings of loneliness, and provides a concrete method for family members to remotely check on the safety of the elderly and manage their safety. Furthermore, by combining it with an emotion engine, it is possible to grasp the emotional state of the elderly and respond more appropriately.

[2125] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2126] Step 1: Start the user device

[2127] What it does: A user presses the power button on the tablet to start up the device.

[2128] Input: User presses and holds the power button.

[2129] Output: The device's home screen is displayed.

[2130] Specific behavior: The user presses and holds the tablet's power button, the boot screen appears, and then the home screen appears.

[2131] Step 2: Displaying the AI ​​Avatar

[2132] How it works: The device automatically launches the AI ​​avatar app from the home screen and displays an AI avatar on the screen, set to a photo of the elderly person's family or friends.

[2133] Enter: the app icon on your home screen.

[2134] Output: An AI avatar is displayed.

[2135] What it does: The device automatically launches the application and displays an AI avatar with a photo of your grandchild.

[2136] Step 3: Initiating everyday conversations and recognizing emotions

[2137] How it works: The device uses a natural language processing model (e.g., GPT-4) to have an AI avatar say, "Good morning, how are you feeling today?" The device then performs a voice analysis of the user's response and uses emotion recognition technology to determine their emotional state.

[2138] Input: A prompt to the generative AI model: "Good morning, how are you feeling today?" and the user's response.

[2139] Output: Emotional state (e.g., "Energetic").

[2140] Specific operation: The AI ​​avatar asks, "Good morning, how are you feeling today?", and the elderly person replies, "I'm feeling fine today." The device records the answer and uses its emotion engine to interpret it as "feeling fine."

[2141] Step 4: Propose and play the game

[2142] How it works: The device uses an AI avatar to ask the elderly, "Would you like to play a memory game today?" If the elderly agrees, the device starts the memory game.

[2143] Input: The game suggestion prompt, "Would you like to play a memory game today?" and the user's response.

[2144] Output: Game result.

[2145] Specific operation: The AI ​​avatar suggests, "Would you like to play a memory game today?" If the elderly person responds, "Sure," a simple memory game will appear on the screen.

[2146] Step 5: Collect and send data

[2147] How it works: The device collects data such as conversation content, game results, and emotion recognition results, and sends the collected data to a server via Wi-Fi.

[2148] Input: Dialogue recording, game results, emotion recognition results.

[2149] Output: Sending data to the server.

[2150] Specific operation: The device stores the conversation records, game results, and emotional state as data, and transmits them to the server via Wi-Fi.

[2151] Step 6: Server receives and stores data

[2152] Operation: The server receives data sent from the device and stores it in a database (e.g., MySQL).

[2153] Input: Data sent from the terminal.

[2154] Output: Save to database.

[2155] What happens: The server stores all received data as entries in a MySQL database.

[2156] Step 7: Data analysis and notification by the server

[2157] What it does: The server analyzes the collected data using machine learning models (e.g., scikit-learn) and sends notifications to family members if anomalies are detected.

[2158] Input: Record data from the database.

[2159] Output: Notification to family.

[2160] Specific operation: The server analyzes the data, and when it detects a decrease in activity, it sends a notification email to the family saying, "The elderly person's activity is decreasing."

[2161] Step 8: Anomaly detection and response

[2162] How it works: The sensor unit detects that the house doors are unlocked and sends this information to the server. The server immediately notifies the family, who then issue a command to "lock" the door via their smartphone. The sensor unit then remotely locks the door based on the received command.

[2163] Input: Anomaly detection data from the sensor unit, instructions from family members.

[2164] Output: Locks the door and notifies family members of the anxiety state.

[2165] Specific operation: The sensor detects that the door to the house is open and sends the information to the server. When a family member presses the "lock" button on their smartphone, the server sends a command to the sensor unit, which then locks the front door. If the emotion engine detects anxiety in the elderly person, it also sends a notification to the family that "the elderly person is feeling anxious."

[2166] These are the specific steps in the program processing of this system, which will help maintain cognitive function, reduce loneliness, and improve safety for the elderly.

[2167] (Application example 2)

[2168] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2169] In order to improve the quality of life of the elderly, it is important to maintain cognitive function and ensure their safety. However, because the time that family members can directly interact with the elderly is limited, a system that can remotely grasp the elderly's situation and respond appropriately is required. It is also important to analyze the elderly's emotional state and respond accordingly. Given this background, it is necessary to provide a system that can monitor the safety and emotional state of the elderly in real time and respond quickly to abnormalities or changes in emotion.

[2170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2171] In this invention, the server includes a means for analyzing the voice and facial expressions of the elderly to grasp their emotional state, a means for monitoring the elderly's environment with a camera in the smart glasses, and a means for sending an alert to relatives when an abnormality is detected. This not only makes it possible to maintain the elderly's cognitive function and ensure their safety, but also enables immediate response to changes in their emotional state.

[2172] An "AI avatar" is a digital character equipped with artificial intelligence to interact with the elderly and display a photo of a relative or acquaintance.

[2173] "Means for displaying photographs of relatives or acquaintances" refers to technology that displays photographs of family members or friends on a digital display to help elderly people feel closer to them.

[2174] "Means for responding" is a function that uses recognition technology and voice synthesis technology to enable elderly people and AI avatars to communicate through natural conversation.

[2175] "Means for maintaining cognitive function in the elderly through entertainment" is a technology that aims to maintain and improve cognitive function in the elderly by providing activities such as simple games and puzzles.

[2176] "Means for collecting activity data and transmitting it to a central processing unit in real time" refers to a function that uses sensors and devices to record the elderly person's daily activities, voice, facial expressions, etc., and transmits the data to a server in real time.

[2177] The "means of notifying relatives of activity data" is a function that automatically analyzes collected data on elderly people and notifies relatives of abnormalities and necessary information.

[2178] The "means for detecting and notifying when someone forgets to lock their home or when there is an abnormality" is a technology that constantly monitors the elderly person's home appliances and home systems, and notifies relatives when it detects that someone has forgotten to lock their home or when there is an abnormality.

[2179] "Means for remotely controlling home appliances and home systems" refers to technology that allows relatives to remotely control home appliances and systems in the elderly person's home via devices such as smartphones.

[2180] "Means for monitoring the environment of the elderly using a camera in smart glasses" is a function that uses a camera installed in smart glasses to monitor the environment around the elderly in real time.

[2181] "Means for analyzing voice and facial expressions to grasp emotional states" refers to technology that analyzes voice and facial expression data of elderly people and uses an emotion engine to estimate and grasp their emotional states.

[2182] The "means for sending an alert to relatives when an abnormality is detected" is a system that sends an alert message to relatives when an abnormality is detected as a result of various monitoring performed on the elderly person.

[2183] The present invention is a system for improving the quality of life of the elderly, and aims to maintain the cognitive function of the elderly and improve their safety through the management of AI avatars and home appliances. In particular, this system has the ability to analyze the emotional state of the elderly using an emotion engine and utilize the analysis data. Specific embodiments are described below.

[2184] System configuration

[2185] The embodiment of the present invention is broadly composed of the following components.

[2186] 1. User Terminal Embodiments

[2187] The user devices are smartphones or smart glasses used by elderly people. The devices have the following main functions:

[2188] Display of AI avatar: The device displays an AI avatar on the display, which is set to a photo of the elderly person's relatives or acquaintances, giving the user a sense of familiarity.

[2189] Voice response function: The device responds naturally to the elderly through an AI avatar, helping to reduce feelings of loneliness and stimulate cognitive function.

[2190] Activity provision function: The device provides simple activities such as puzzles and games, which further maintain and improve cognitive function in the elderly.

[2191] Emotion recognition function: The device analyzes the elderly person's voice and facial expressions and uses an emotion engine to understand their emotional state.

[2192] Data collection and transmission: Responses, game results, and emotion recognition results are collected as data and periodically sent to the server.

[2193] 2. Server Implementation

[2194] The server is the center of the system, centrally managing elderly people's activity data and providing necessary notifications and analysis.

[2195] Data reception and storage: The server receives the activity data and emotion data sent from the user terminal and stores them in a database.

[2196] Data analysis: The server analyzes the collected data and sends notifications to relatives if any abnormalities are detected.

[2197] Notification function: The server periodically reports the elderly person's activity and emotional state to relatives, and sends immediate alerts in case of an emergency.

[2198] 3. Sensor Unit Embodiment

[2199] The sensor unit integrates various sensors in the home and monitors the status of home appliances and systems.

[2200] Monitoring functions: Sensors monitor locks, electricity, temperature, motion, and more within the home.

[2201] Data transmission: The sensor unit periodically transmits the monitoring results to the server.

[2202] Control function: Receives instructions from the server and remotely controls and locks / unlocks home appliances.

[2203] Specific examples

[2204] Example 1: Everyday usage scenario

[2205] The user (elderly person) turns on the smart glasses every morning, and an AI avatar with a photo of a relative's face appears on the screen. The AI ​​avatar asks, "Good morning, how are you feeling today?" The elderly person responds, "I'm feeling fine today." During this interaction, the device records the elderly person's voice and analyzes their emotional state using an emotion engine. While making a normal response, the avatar suggests a simple memory game. The elderly person plays the game, and the results and emotional state are recorded on the device. The device sends this data to a server, which ana...

Claims

1. Display an AI avatar and a photo of the elderly person's family or friends Means and To communicate with the elderly through AI avatars Means and Maintaining cognitive function in the elderly through dialogue and games Means and Collects elderly people's activity data and sends it to a server in real time Means and Notifying family members of elderly people's activity data Means and It works in conjunction with home appliances and home systems to detect and notify you if you forget to lock your door or if there are any abnormalities. Means and Remote control of elderly people's home appliances and home systems Means and A system including:

2. Analyzes conversations and game results, and sends alerts to family members if anything unusual occurs The system of claim 1 , further comprising:

3. The sensor unit constantly monitors the status of home appliances and home systems, and sends the data to a server. The system of claim 1 , further comprising:

Images