System

Abstract

A system is provided.SOLUTION: This system is equipped with generation artificial intelligence, and includes a means for supporting the memory of the daily life of the aged person, a means for managing the schedule of the aged person, a means for detecting the abnormal operation of the aged person, and a means for analyzing and visualizing the daily behavior of the aged person.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In an aging society, elderly people often experience memory loss, difficulty managing schedules, and anxiety about changes in their health. Furthermore, it takes time and effort for family members and caregivers to monitor and appropriately support the daily lives of elderly people. This reduces the quality of life of elderly people and increases the burden on their families. Current technology is insufficient to provide comprehensive support for the daily activities and health status of elderly people, and this issue needs to be addressed. [Means for solving the problem]

[0005] This invention provides comprehensive support for the daily lives of the elderly through a system equipped with generative AI. Specifically, it provides the following means:

[0006] A method for supporting memory in the daily life of the elderly: Supports the memory of the elderly by displaying specific questions and collecting the user's answers to them.

[0007] Schedule management means: Saves the schedule entered by the user and notifies at the specified time.

[0008] Abnormal behavior detection: Recognizes daily activity patterns and sends an alert to set contacts if an abnormality is detected.

[0009] Behavioral analysis tools: Collect and analyze users' daily behavioral data and visualize it using graphs and charts.

[0010] This will help reduce memory loss and difficulty managing schedules that elderly people face in their daily lives, and allow them to take prompt action if an abnormality is detected.It will also make it easier for family members and caregivers to understand the health status and daily activities of the elderly, providing peace of mind.

[0011] "Generative AI" is an AI system that has the ability to analyze data, learn, and evolve on its own, and provides services tailored to the needs of each individual user.

[0012] The "memory support means" is a function that evaluates and supports the user's memory and cognitive function by displaying specific questions and collecting and analyzing the user's answers to those questions.

[0013] The "schedule management means" is a function that saves the schedule entered by the user and notifies the user at a specified time.

[0014] The "abnormal behavior detection means" is a function that recognizes daily activity patterns, detects abnormal behavior that deviates from standard behavior patterns, and sends a warning to the specified contacts.

[0015] The "behavioral analysis means" is a function that collects the user's daily behavioral data, analyzes it to understand behavioral patterns and health conditions, and visualizes them as graphs and charts. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system for the elderly equipped with productive artificial intelligence (generative AI) that comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[0038] Cognitive exercise features

[0039] 1. At a specific time each day, the device asks the elderly questions to assess their memory and cognitive function, such as "What did you have for breakfast today?"

[0040] 2. The user answers the displayed question by typing, for example, "I ate bread and coffee."

[0041] 3. The device sends the user's answer to the server.

[0042] 4. The server quantifies the received data and records it in a database. This data is used to track and analyze changes in the user's cognitive function.

[0043] Schedule reminder function

[0044] 1. The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM."

[0045] 2. The terminal sends this schedule data to the server.

[0046] 3. The server sets a reminder based on the received schedule data. When the specified time approaches, for example, a reminder notification is set 15 minutes before.

[0047] 4. When the reminder time arrives, the device will display a notification to the user, allowing the user to take the necessary action without forgetting the appointment.

[0048] Non-response and abnormality warning function

[0049] 1. The device monitors the elderly person's daily activity patterns, such as meal times, walking distance, and response status.

[0050] 2. The server analyzes the data collected daily and compares it with standard patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[0051] 3. If the server detects an abnormality, it will send a warning message to pre-defined contacts, detailing the abnormality and recommending a course of action.

[0052] Behavioral analysis and visualization features

[0053] 1. The device collects real-time data on the elderly person's daily activities, including walking distance, dietary habits, and sleep duration.

[0054] 2. The device periodically sends the collected data to the server.

[0055] 3. The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status.

[0056] 4. The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance.

[0057] 5. The device provides visualized data to the user and their family through a dashboard and notification function.

[0058] Specific examples

[0059] The user answers the question on the device by saying, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[0060] In this way, the present invention comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0061] The processing flow will be explained below.

[0062] Cognitive exercise features

[0063] Step 1:

[0064] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess the user's memory.

[0065] Step 2:

[0066] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[0067] Step 3:

[0068] The device sends the user's response data to the server in real time.

[0069] Step 4:

[0070] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0071] Step 5:

[0072] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0073] Schedule reminder function

[0074] Step 1:

[0075] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[0076] Step 2:

[0077] The terminal transmits the input schedule data to the server.

[0078] Step 3:

[0079] The server stores the received schedule data and sets reminders, such as notifying you 15 minutes before the specified time.

[0080] Step 4:

[0081] When the set reminder time arrives, the server sends a reminder notification to the device.

[0082] Step 5:

[0083] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[0084] Non-response and abnormality warning function

[0085] Step 1:

[0086] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0087] Step 2:

[0088] The device sends collected activity data, including sensor data and user input data, to a server.

[0089] Step 3:

[0090] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[0091] Step 4:

[0092] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check."

[0093] Step 5:

[0094] The device will notify contacts of the received warning message, allowing for a prompt response.

[0095] Behavioral analysis and visualization features

[0096] Step 1:

[0097] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[0098] Step 2:

[0099] The device periodically transmits the collected data to the server in real time.

[0100] Step 3:

[0101] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0102] Step 4:

[0103] The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by users and their families.

[0104] Step 5:

[0105] The device will then communicate the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[0106] Through the above steps, this system provides comprehensive support for the lives of elderly people, enabling them to live their daily lives with peace of mind.

[0107] Example 1

[0108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0109] Maintaining memory, managing schedules, and monitoring health are important in the daily lives of elderly people. However, these tasks are difficult to perform alone and can be a burden on family members and caregivers. Furthermore, a system is needed to respond quickly when elderly people exhibit abnormal behavior in their daily lives. This invention provides a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0111] In this invention, the server includes means for generating specific questions, collecting and transmitting answers from users to the server, means for transmitting schedule data entered by the user to the server and for the server to set reminders, and means for collecting behavioral data in real time and providing visualized data to the user and their family through an interface, thereby enabling memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization in the daily lives of elderly people.

[0112] "Generative artificial intelligence" is an automated intelligent system that responds and analyzes based on user data input and interactions.

[0113] The "memory support means" is a function that asks specific questions to the user to help the user's cognitive functions.

[0114] The "schedule management means" is a function that manages the schedule entered by the user and notifies the user at a specific time.

[0115] The "abnormal behavior detection means" is a function that monitors the user's daily activities and detects abnormal behavior or patterns.

[0116] "Behavioral analysis" is the process of collecting and analyzing users' daily behavioral data.

[0117] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[0118] "Means for generating specific questions" refers to a function that uses a generative AI model to automatically create appropriate questions for the user.

[0119] The "means for collecting user responses" is a function that collects the responses entered by the user as data and transmits them to the server.

[0120] The "means for setting a reminder" is a function for managing a schedule so as to notify the user at a specific time based on the schedule input by the user.

[0121] "Means for collecting behavioral data in real time" refers to a function that uses sensors and devices to detect and record users' daily life behavior in real time.

[0122] "Means for providing visualized data to users and their families through an interface" refers to a function that visually displays collected and analyzed data and provides it to users and their families in the form of a dashboard or notification.

[0123] The present invention is a system equipped with generative artificial intelligence (generative AI) that provides comprehensive support for the daily lives of elderly people. This system provides integrated functions for memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization. Detailed embodiments of each function are described below.

[0124] Hardware Configuration

[0125] The system uses the following hardware components:

[0126] Terminal: A handheld device, such as a tablet or smartphone, that acts as an interface for user input.

[0127] Server: Cloud server or corporate server that analyzes, stores, and manages data.

[0128] Sensor devices: Wearable devices (e.g., smart watches) and home sensors that collect daily activity data of older adults.

[0129] Software Configuration

[0130] Generative AI models (e.g., GPT-4): Used to generate specific questions, providing users with the ability to ask appropriate questions in natural language.

[0131] Database management system: Stores and manages user response data and behavioral data.

[0132] Analysis algorithm: An AI algorithm that analyzes collected data and evaluates the user's health status and behavioral patterns.

[0133] Notification system: A system for notifying users or stakeholders of reminders and anomaly detection messages.

[0134] Memory support function

[0135] The device generates questions to assess memory and cognitive function at specific times each day. For example, a generative AI model is used to create a question such as "What did you have for breakfast today?" The user responds by typing "I had bread and coffee." The device sends this response to a server, which quantifies the data and records it in a database. This allows changes in cognitive function to be tracked and analyzed.

[0136] Schedule management function

[0137] The user inputs their schedule into the device. For example, they input an appointment to "go to the hospital tomorrow at 10:00 AM." The device sends this schedule data to the server, and the server sets a reminder for the specified time. When the specified time approaches, the device displays a notification to the user. This allows the user to remember their appointment.

[0138] Abnormal behavior detection function

[0139] The terminal monitors the elderly person's daily activity patterns through sensor devices, including meal times, walking distance, and response status. The server analyzes the collected data and compares it with standard patterns to detect abnormalities. For example, a lack of response during lunchtime is deemed abnormal. If an abnormality is detected, the server sends a warning message to family members or caregivers.

[0140] Behavioral analysis and visualization features

[0141] The device collects daily behavioral data of the elderly in real time and periodically transmits it to a server. The server then analyzes the received data using AI algorithms to evaluate behavioral patterns and health conditions. The analysis results are visualized as graphs and charts and provided to the user and their family via the device's dashboard.

[0142] Examples and prompts

[0143] Examples:

[0144] The user answers the question displayed on the device by saying, "I had bread and coffee for breakfast today." The device then sends this answer to the server, which then digitizes the data and records it.

[0145] When a user enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[0146] If daily activity is abnormal, the server will send an alert to the family.

[0147] Ultimately, behavioral patterns are visualized and the data is always available for reference.

[0148] Example prompt sentence:

[0149] "What did you have for breakfast today?"

[0150] "Please enter an appointment to go to the hospital tomorrow at 10 AM."

[0151] Check your walking distance.

[0152] In this way, the present invention realizes a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Cognitive exercise features

[0155] Step 1:

[0156] At a specific time each day, the device uses a generative AI model (e.g., GPT-4) to generate questions for the user to assess their memory and cognitive function. For example, a question might be generated: "What did you have for breakfast today?" Input: Current time, generative AI model. Output: Specific question.

[0157] Step 2:

[0158] The terminal displays the generated question to the user. The user inputs the answer using a keyboard or voice input. For example, the user answers "I ate bread and coffee." Input: The generated question. Output: The user's answer data.

[0159] Step 3:

[0160] The terminal collects the user's response data and sends it to the server using a secure protocol (e.g. HTTPS). Input: User's response data. Output: Data sent to the server.

[0161] Step 4:

[0162] The server analyzes the received response data and organizes it as numerical data. It analyzes specific keywords (e.g., bread, coffee) and tags them. Input: User response data. Output: Tagged numerical data.

[0163] Step 5:

[0164] The server records the analyzed and quantified data in a database, which is later used to track changes in the user's cognitive function. Input: Analyzed and quantified data. Output: Data recorded in the database.

[0165] Schedule reminder function

[0166] Step 1:

[0167] The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM." Input: User's schedule data. Output: Input schedule data.

[0168] Step 2:

[0169] The terminal sends this schedule data to the server. Input: User's schedule data. Output: Schedule data sent to the server.

[0170] Step 3:

[0171] The server sets a reminder based on the received schedule data. The reminder is set to notify 15 minutes before the scheduled time. Input: Schedule data. Output: Set reminder.

[0172] Step 4:

[0173] When the reminder time approaches, the device will remind the user with a pop-up notification or a voice notification. For example, it may notify the user that it is almost time to go to the hospital. Input: The set reminder. Output: The reminder notification.

[0174] Non-response and abnormality warning function

[0175] Step 1:

[0176] The device uses sensors and user input to monitor the elderly person's daily activity patterns, including meal times, walking distance, and responsiveness. Input: Activity data from sensors and user input. Output: Collected activity data.

[0177] Step 2:

[0178] The server analyzes the activity data collected periodically and compares it with standard patterns to detect anomalies. For example, if there is no response during lunch time, it will be judged as an anomaly. Input: Collected activity data. Output: Detected anomalous data.

[0179] Step 3:

[0180] When an abnormality is detected, the server sends a warning message to pre-defined contacts. This message describes the abnormality and the required action. Input: Detected abnormal data. Output: Warning message.

[0181] Behavioral analysis and visualization features

[0182] Step 1:

[0183] The device collects daily behavioral data of elderly people in real time, including walking distance, dietary habits, and sleep duration. Input: Behavioral data collected in real time. Output: Collected behavioral data.

[0184] Step 2:

[0185] The device periodically sends the collected behavioral data to the server. Input: Collected behavioral data. Output: Data sent to the server.

[0186] Step 3:

[0187] The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status. Input: Data sent to the server. Output: Analysis results.

[0188] Step 4:

[0189] The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance. Input: Analysis results. Output: Visualized data (graphs and charts).

[0190] Step 5:

[0191] The device provides visualized data in the form of a dashboard to the user and their family. Periodic reports are also delivered via the notification function. Input: Visualized data. Output: Reports via dashboards and notifications.

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] Current support systems for the elderly are limited to use at home and lack support when out and about, especially in physical stores. This can lead to elderly people getting lost in stores or forgetting important shopping lists. Furthermore, there is also the problem of not being able to respond immediately when abnormal behavior occurs in stores. These issues need to be resolved.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the elderly's daily memory, means for managing the elderly's schedule, means for detecting abnormal behavior of the elderly, means for analyzing and visualizing the elderly's daily behavior, means for displaying product information and prices when the elderly approaches a product shelf, means for detecting and notifying an abnormality when the elderly does not move within a certain period of time in the store, and means for collecting, analyzing, and visualizing data on the elderly's behavior in the store. This enables the elderly to shop safely and efficiently in physical stores, and allows for quick response even if abnormal behavior occurs.

[0197] "Generative AI" refers to AI that has the ability to automatically generate new information and judgments using data.

[0198] "Memory support means" refers to a device or system that provides a function to support the memory of elderly people by displaying specific questions and collecting and analyzing the user's answers to those questions.

[0199] "Schedule management means" refers to a device or system that saves the schedule entered by the user and notifies the user at the specified time.

[0200] "Abnormal behavior detection means" refers to a device or system that has the function of monitoring the behavior of an elderly person and detecting abnormal behavior that deviates from normal patterns.

[0201] The term "behavioral analysis means" refers to a device or system that has the function of collecting and analyzing daily behavioral data of elderly people.

[0202] "Visualization means" refers to a device or system that has the function of displaying analyzed behavioral data in a visual format such as a graph or chart, and providing it in a form that is easy for users to understand.

[0203] "Product information display means" refers to a device or system that has the function of displaying product information and prices when an elderly person approaches a product shelf.

[0204] "Abnormality detection and notification means" refers to a device or system that has the function of detecting an abnormality and issuing a notification if an elderly person does not move within a certain period of time inside the store.

[0205] "Behavioral data collection means" refers to a device or system that has the function of continuously collecting data on the behavior of elderly people in a store.

[0206] The term "analysis result visualization means" refers to a device or system that has the function of analyzing collected behavioral data and visually expressing the results to provide to the user.

[0207] The present invention aims to provide support for elderly people in brick-and-mortar stores, particularly by providing a system that maintains cognitive abilities, monitors health status, and assists in efficient shopping. To achieve this, the following hardware and software are used:

[0208] Hardware and Software

[0209] Hardware:

[0210] Smartphone

[0211] Smart Glasses

[0212] server

[0213] software:

[0214] Programming language: Python

[0215] Machine learning library: TensorFlow

[0216] Web framework: Flask

[0217] Database: SQLite

[0218] Notification Service: Twilio API

[0219] System flow

[0220] 1. Memory Support Tools:

[0221] The server periodically generates questions for the elderly, which are generated based on a generative AI model.

[0222] The device (smartphone or smart glasses) displays this question to the elderly person and collects their answer.

[0223] The server stores the received response data and analyzes it to evaluate the cognitive status of the elderly person.

[0224] 2. Scheduling Management Methods:

[0225] The user inputs the schedule into the terminal.

[0226] The server stores this schedule data and uses the Twilio API to send reminder notifications when the specified time approaches.

[0227] 3. Abnormal Operation Detection Method:

[0228] The device uses sensors in smartphones and smart glasses to monitor the movements of elderly people.

[0229] The server analyzes the received operational data and sends a warning message to the specified contacts if an abnormality is detected.

[0230] 4. Behavioral analysis and visualization tools:

[0231] The terminal collects data on the elderly's behavior within the store and sends it to a server.

[0232] The server analyzes the collected data using TensorFlow to evaluate behavioral patterns and health status.

[0233] The analysis results will be visualized as graphs and charts using Flask, making them accessible to the elderly and their families.

[0234] 5. Product information display means:

[0235] When an elderly person approaches a store shelf, the device uses RFID or NFC to obtain product information and displays it on the smart glasses or smartphone.

[0236] 6. Anomaly detection notification method:

[0237] The device detects this abnormality if the elderly person does not move within a certain period of time.

[0238] The server analyzes the abnormal condition and sends a notification using the Twilio API.

[0239] Specific examples

[0240] For example, when an elderly person approaches the bread shelf, the smart glasses will display a message saying, "This product is bread. The price is 300 yen." If the elderly person sets a reminder to "not forget to take today's medicine," a notification will be sent at the scheduled time. If an abnormality is detected, a notification will be sent to store staff saying, "The customer has been stationary for a long time."

[0241] Prompt Sentence Examples

[0242] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] server:

[0246] A generative AI model is used to generate specific questions to assess the elderly person's cognition and memory, such as "What did you have for breakfast today?", and the generated questions are sent to the device.

[0247] Input: User data, past response data

[0248] Output: Generated question text

[0249] Step 2:

[0250] Device:

[0251] The generated question is displayed to the user. The device uses a smartphone or smart glasses to visually present the question to the elderly.

[0252] Input: Generated question text

[0253] Output: Screen showing the question

[0254] Step 3:

[0255] User:

[0256] Enter your answer to the question. For example, you might enter, "I had bread and coffee for breakfast today." After you enter the answer, the data is recorded on the device.

[0257] Input: The text of the user's answer to the question

[0258] Output: User's answer text

[0259] Step 4:

[0260] Device:

[0261] The system receives the user's answers and sends them to the server, where the answers are stored in a database for later analysis.

[0262] Input: User's answer text

[0263] Output: Data sent to the server

[0264] Step 5:

[0265] server:

[0266] The received response data is analyzed to assess the user's cognitive function. This includes comparing it with past data and quantifying it to track memory fluctuations. The analysis results are stored in a database.

[0267] Input: User response data, past data

[0268] Output: Analysis result data

[0269] Step 6:

[0270] User:

[0271] Enter your schedule. For example, enter "I'll go to the hospital tomorrow at 10:00 AM." The entered data will be saved on your device.

[0272] Input: Schedule data (date, time, content)

[0273] Output: Schedule registration data to the terminal

[0274] Step 7:

[0275] Device:

[0276] The entered schedule data is sent to the server.

[0277] Input: Schedule data

[0278] Output: Data sent to the server

[0279] Step 8:

[0280] server:

[0281] It saves the received schedule data, generates reminder notifications when the specified time approaches, and sends notifications to the user using the Twilio API.

[0282] Input: Schedule data

[0283] Output: Reminder notification

[0284] Step 9:

[0285] Device:

[0286] The movement of elderly people is monitored using sensors on smartphones and smart glasses, and the collected data is sent to a server.

[0287] Input: Motion data (sensor data)

[0288] Output: Data sent to the server

[0289] Step 10:

[0290] server:

[0291] The received operational data is analyzed, and if an abnormality is detected, a warning message is sent to the specified contacts. Notifications are sent using the Twilio API.

[0292] Input: Operation data

[0293] Output: Warning notice

[0294] Step 11:

[0295] Device:

[0296] When an elderly person approaches a product shelf, product information is obtained using RFID or NFC and displayed on the smart glasses or smartphone.

[0297] Input: Product data (RFID / NFC tag information)

[0298] Output: Product information display

[0299] Step 12:

[0300] User:

[0301] The user checks the product information and purchases it if necessary. The user's behavior data and purchase data are recorded on the device.

[0302] Input: Product information, purchase action

[0303] Output: Purchase data

[0304] Step 13:

[0305] Device:

[0306] Data on the behavior of elderly people in the store is collected and sent to a server.

[0307] Input: Behavioral data (location information, movement history)

[0308] Output: Data sent to the server

[0309] Step 14:

[0310] server:

[0311] The collected behavioral data is analyzed, and the results are visualized to evaluate behavioral patterns and health conditions, generating reports that can be viewed by seniors and their families.

[0312] Input: Behavioral data

[0313] Output: Visualized data of analysis results (graphs, charts)

[0314] Prompt Sentence Examples

[0315] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[0316] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0317] The present invention is a system for the elderly equipped with generative artificial intelligence (generative AI) and an emotion engine, which comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[0318] Cognitive exercise features

[0319] 1. The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.). The questions evaluate the user's memory, such as "What did you have for breakfast today?"

[0320] 2. The user enters an answer to the question displayed on the terminal. For example, the user enters "I ate bread and coffee."

[0321] 3. The device sends the user's response data to the server in real time.

[0322] 4. The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0323] 5. The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0324] Schedule reminder function

[0325] 1. The user enters their schedule into the device. For example, they might enter, "I'm going to the hospital tomorrow at 10:00 AM."

[0326] 2. The terminal sends the entered schedule data to the server.

[0327] 3. The server saves the received schedule data and sets a reminder, such as notifying you 15 minutes before the specified time.

[0328] 4. When the set reminder time arrives, the server sends a reminder notification to the device.

[0329] 5. The device displays a reminder notification to the user, for example, a pop-up notification saying, "You have an appointment to go to the doctor in 10 minutes."

[0330] Non-response and abnormality warning function

[0331] 1. The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0332] 2. The device sends the collected activity data, including sensor data and user input data, to the server.

[0333] 3. The server analyzes the received data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[0334] 4. If the server detects an abnormality, it will send a warning message to pre-defined contacts, for example, "Mom was unresponsive at lunchtime. Please check on her."

[0335] 5. The device will notify the contacts of the received warning message, allowing for a prompt response.

[0336] Behavioral analysis and visualization features

[0337] 1. The device collects daily user behavior data, including walking distance, dietary intake, and sleep duration.

[0338] 2. The device periodically transmits the collected data to the server in real time.

[0339] 3. The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0340] 4. The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by the user and their family.

[0341] 5. The device notifies the user and their family of the visualized data, for example, by displaying a graph of the change in walking distance each week.

[0342] Emotion Engine Functions

[0343] 1. The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[0344] 2. The device sends emotion analysis data to the server, including voice data and facial expression data.

[0345] 3. The server generates an appropriate response based on the received emotion analysis data. For example, if the user is feeling down, it generates an encouraging message.

[0346] 4. The server sends the generated dialogue content to the terminal.

[0347] 5. The device displays the generated dialogue to the user, continues the conversation, and, if necessary, sends messages of encouragement or confirmation of the user's safety based on the user's emotional state.

[0348] Specific examples

[0349] The user answers the device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, and a reminder notification is displayed on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[0350] Furthermore, the device analyzes the user's emotions and, for example, if the user is feeling down, displays an encouraging message such as, "Are you OK? Is there anything I can help you with?" In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[0351] The processing flow will be explained below.

[0352] Cognitive exercise features

[0353] Step 1:

[0354] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess memory ability.

[0355] Step 2:

[0356] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[0357] Step 3:

[0358] The device sends the user's response data to the server in real time.

[0359] Step 4:

[0360] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0361] Step 5:

[0362] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0363] Schedule reminder function

[0364] Step 1:

[0365] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[0366] Step 2:

[0367] The terminal transmits the input schedule data to the server.

[0368] Step 3:

[0369] The server stores the received schedule data and sets a reminder, for example, to notify you 15 minutes before the specified time.

[0370] Step 4:

[0371] When the set reminder time arrives, the server sends a reminder notification to the device.

[0372] Step 5:

[0373] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[0374] Non-response and abnormality warning function

[0375] Step 1:

[0376] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0377] Step 2:

[0378] The device sends collected activity data, including sensor data and user input data, to a server.

[0379] Step 3:

[0380] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[0381] Step 4:

[0382] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check on her."

[0383] Step 5:

[0384] The device will notify contacts of the received warning message, allowing for a prompt response.

[0385] Behavioral analysis and visualization features

[0386] Step 1:

[0387] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[0388] Step 2:

[0389] The device periodically transmits the collected data to the server in real time.

[0390] Step 3:

[0391] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0392] Step 4:

[0393] The server visualizes the analysis results as graphs and charts, and the visualized data is displayed on a dashboard or similar.

[0394] Step 5:

[0395] The device then communicates the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[0396] Emotion Engine Functions

[0397] Step 1:

[0398] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[0399] Step 2:

[0400] The device sends emotion analysis data, including voice data and facial expression data, to the server.

[0401] Step 3:

[0402] The server generates appropriate responses based on the received emotion analysis data, for example, an encouraging message if the user is feeling down.

[0403] Step 4:

[0404] The server transmits the generated dialogue content to the terminal.

[0405] Step 5:

[0406] The device displays the generated dialogue content to the user, continues the dialogue appropriately, and, if necessary, sends messages of encouragement or confirmation of safety based on the user's emotional state.

[0407] As a specific example, if a user says, "I'm feeling a little down today," the device analyzes the user's voice and facial expressions to recognize the emotion. The device sends this data to a server, which then generates a dialogue such as, "Are you OK? Is there anything I can help you with?" and sends it back to the device. The device then displays this message to the user and the dialogue continues. In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] In the lives of elderly people, there is a growing need for early detection of memory decline, difficulty in schedule management, and abnormal behavior. Furthermore, analyzing and visualizing daily behavior is important for elderly people and their families. However, systems that comprehensively and effectively support these issues are still insufficient. Furthermore, there is a need for systems that can grasp the emotional state of elderly people and generate appropriate responses.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server includes a means for supporting the elderly's memory in daily life, a means for managing the elderly's schedule, a means for detecting abnormal behavior of the elderly, a means for analyzing and visualizing the elderly's daily behavior, and a means for analyzing the elderly's emotional state and generating appropriate responses. This enables not only memory support and schedule management, but also early detection of abnormal behavior, visualization of behavioral analysis, emotion analysis, and appropriate responses.

[0413] "Generative AI" is AI that generates sentences and dialogues using natural language processing and machine learning.

[0414] The "memory support means" is a function that evaluates and supports the user's memory by displaying specific questions to the user and collecting, recording, and analyzing the user's answers.

[0415] The "schedule management means" is a function that saves the schedule entered by the user and sends a reminder notification at a specified time.

[0416] The "abnormal behavior detection means" is a function that monitors the behavioral data of elderly people and issues an alert if any unusual behavior or reaction is detected.

[0417] The "behavioral analysis means" is a function that evaluates the user's behavioral patterns and health condition by collecting and analyzing the user's daily behavioral data.

[0418] "Visualization means" is a function that visually displays the results of behavioral analysis as graphs or charts.

[0419] The "emotional state analysis means" is a function that analyzes data such as voice and facial expression to determine the user's emotional state.

[0420] The "response generation means" is a function that generates appropriate dialogue content and messages based on the results of the emotional state analysis.

[0421] A "server" is a centralized computer system that stores, analyzes, and processes data.

[0422] A "terminal" is a computer device that is directly operated by a user and that communicates with a server to provide various functions.

[0423] This invention is a system for supporting the daily lives of elderly people, utilizing generative artificial intelligence to support memory, manage schedules, detect abnormal behavior, analyze and visualize behavior, and analyze emotional states. This system mainly consists of a server and a terminal, and monitors user input data and daily activity data in real time, analyzing and responding appropriately.

[0424] Hardware and Software

[0425] A server is a centralized computer system that stores, analyzes, and processes data. It requires powerful processors, memory, and large amounts of storage. The software required includes a database management system (DBMS), machine learning libraries (e.g., TensorFlow, PyTorch), data analysis tools, and visualization tools (e.g., Tableau).

[0426] A terminal is a computing device that is directly operated by a user, and is typically a compact tablet or smartphone. Terminals are equipped with various sensors (e.g., camera, microphone, accelerometer) and use these sensors to collect data. Terminal software includes a user interface, a data collection application, and a communication module.

[0427] Program processing

[0428] At a specific time each day, the device displays a question to the user to assess their memory. For example, "What did you have for breakfast today?" The user responds by answering "I had bread and coffee." The device then sends this response data to the server in real time. The server then converts the received data into a numerical value and stores it in a database. For example, "bread" is converted into a numerical value of 1, and "coffee" into a numerical value of 1. The server then analyzes the state of the user's cognitive function based on the saved data, comparing it with past data, and evaluates progress and changes.

[0429] When a user inputs their schedule into a device, for example, "I'm going to the hospital tomorrow at 10 AM," the device sends this schedule data to the server, which saves the received data and sets a reminder. A reminder is sent 15 minutes before the specified time. When the time for this reminder arrives, the server sends a reminder to the device, and the device displays a pop-up notification saying, "You have an appointment to go to the hospital in 10 minutes."

[0430] Furthermore, the device monitors the user's daily activity patterns. For example, it monitors meal times, walking distance, and response status in real time. The device sends the collected activity data to a server, which analyzes the received data and compares it with standard behavioral patterns to determine whether there are any abnormalities. If an abnormality is detected, for example, if there is no response during lunch, the server determines that this is an abnormality and sends a warning message to the specified contacts saying, "Mom did not respond during lunch. Please check." The device then notifies the contacts of the received warning message.

[0431] The server analyzes the behavioral data using AI algorithms to evaluate the user's behavioral patterns and health status. The analysis results are visualized as graphs and charts and displayed to the user and their family. For example, a graph showing changes in walking distance each week is displayed.

[0432] The device analyzes the user's voice and facial expressions in real time to determine their emotional state. For example, if the user is feeling down, the device sends this analysis data to the server, which then generates an encouraging message. The server then sends the generated dialogue content to the device, which then displays a message such as, "Are you OK? Is there anything I can help you with?"

[0433] Specific examples

[0434] For example, a user might answer to their device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital tomorrow at 10 a.m.", the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If their daily activities are abnormal, the server sends an alert to their family, and ultimately, their behavioral patterns are visualized and the data can be constantly referenced. Furthermore, the device analyzes the user's emotions. For example, if the user is feeling down, it displays an encouraging message such as, "Are you okay? Is there anything I can help you with?"

[0435] Prompt Sentence Examples

[0436] To generate a message of encouragement for a user who is feeling down, an example of a prompt to input to the AI ​​model is as follows:

[0437] "When the user is feeling down, generate an encouraging message. The user's name is Takahashi."

[0438] This allows the generative AI model to generate dialogue such as, "Takahashi-san, are you okay? Is there anything I can help you with?"

[0439] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0440] Cognitive exercise features

[0441] Step 1:

[0442] The device displays a question to assess memory ability every morning at 9:00, such as "What did you have for breakfast today?" The input is a specific time (e.g., 9:00), and the output is the display of the question.

[0443] Step 2:

[0444] The user inputs "I ate bread and coffee" into the terminal. The input is the user's answer, and the output is text data.

[0445] Step 3:

[0446] The terminal transmits the user's answers to the server in real time. The input is the user's answer text data, and the output is the transmitted data. This is done using a communication module.

[0447] Step 4:

[0448] The server digitizes the received text data and stores it in a database. The input is the received text data, and the output is the digitized data and its storage. For example, "bread" is digitized as 1 and "coffee" is digitized as 1.

[0449] Step 5:

[0450] The server analyzes the user's cognitive function status based on the stored data and compares it with past data. The input is the stored data, and the output is the analysis results. Machine learning algorithms are used for these analyses.

[0451] Schedule reminder function

[0452] Step 1:

[0453] The user inputs an appointment into the terminal, such as "I will go to the hospital tomorrow at 10:00 AM." The input is the user's schedule data, and the output is the input data.

[0454] Step 2:

[0455] The terminal transmits the input schedule data to the server. The input is the schedule data, and the output is the transmission data. The data is transmitted via the communication module.

[0456] Step 3:

[0457] The server saves the received data and sets the reminder time. The input is the received data and the output is the reminder setting. For example, set it to notify 15 minutes before the specified time.

[0458] Step 4:

[0459] The server sends a notification to the device when the specified reminder time arrives. The input is the reminder time and the output is the notification data.

[0460] Step 5:

[0461] The device displays a pop-up notification saying, "I have an appointment to go to the hospital in 10 minutes." The input is the notification data, and the output is the pop-up display.

[0462] Non-response and abnormality warning function

[0463] Step 1:

[0464] The device monitors daily user activity data such as meal times, walking distance, and response status. The input is sensor data and user operation data, and the output is collected data.

[0465] Step 2:

[0466] The terminal transmits the collected activity data to the server in real time. The input is the collected data and the output is the transmitted data.

[0467] Step 3:

[0468] The server analyzes the received data and compares it with standard behavioral patterns to detect anomalies. The input is the received data, and the output is the analysis results. For example, if there is no response during lunch time, an anomaly is detected.

[0469] Step 4:

[0470] If an abnormality is detected, the server sends a warning message to pre-defined contacts. The input is the analysis result, and the output is the warning message. For example, "Mom was unresponsive at lunchtime. Please check."

[0471] Step 5:

[0472] The terminal notifies the contact of the received warning message. The input is the warning message and the output is the notification display.

[0473] Behavioral analysis and visualization features

[0474] Step 1:

[0475] The device collects daily user behavior data, including walking distance, dietary habits, sleep duration, etc. The input is sensor data and user operation data, and the output is the collected data.

[0476] Step 2:

[0477] The terminal transmits the collected data to the server in real time. The input is the collected data and the output is the transmitted data.

[0478] Step 3:

[0479] The server uses AI algorithms to analyze the received data and evaluate the user's behavioral patterns and health status. The input is the received data, and the output is the analysis results.

[0480] Step 4:

[0481] The server visualizes the analysis results as graphs and charts and displays them on a dashboard. The input is the analysis results, and the output is the visualized data. For example, a graph showing changes in walking distance each week is displayed.

[0482] Step 5:

[0483] The terminal notifies the user and their family of the visualized data. The input is the visualized data, and the output is a notification display.

[0484] Emotion Engine Functions

[0485] Step 1:

[0486] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state. The input is voice data and facial expression data, and the output is emotion analysis data.

[0487] Step 2:

[0488] The device transmits the emotion analysis data to the server in real time. The input is the emotion analysis data, and the output is the transmitted data.

[0489] Step 3:

[0490] The server generates appropriate dialogue content based on the received data. The input is emotion analysis data, and the output is the generated dialogue content. For example, if the user is feeling down, it generates an encouraging message.

[0491] Step 4:

[0492] The server sends the generated dialogue content to the terminal. The input is the generated dialogue content, and the output is the transmitted data.

[0493] Step 5:

[0494] The terminal displays the generated dialogue to the user and continues the conversation. The input is the transmitted data, and the output is the displayed dialogue. For example, it displays a message such as "Are you OK? Is there anything I can help you with?"

[0495] (Application example 2)

[0496] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0497] There is a need for support for the daily lives of the elderly, as well as cognitive function support, schedule management, abnormal behavior detection, and emotion monitoring for factory workers. Current systems have difficulty providing these functions comprehensively, resulting in increased human resources and management costs. Furthermore, it is difficult to detect abnormal behavior and emotional states of elderly people and workers in real time and respond quickly. For this reason, a comprehensive support system is needed to ensure that elderly people and workers can live their daily lives and work with peace of mind.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the memory of elderly people in their daily lives, means for managing the schedule of elderly people, means for detecting abnormal movements of elderly people, means for analyzing and visualizing the daily behavior of elderly people, means for supporting the cognitive function of workers, means for notifying factory workers of schedules and meetings, means for detecting and notifying non-response and abnormal behavior of workers, and means for monitoring the emotional state of workers and providing support. This enables comprehensive support for elderly people and workers.

[0499] "Generative AI" is AI that has the ability to generate new information and solutions based on data.

[0500] "Elderly people" is a broad term referring to users who provide support for the daily lives of the elderly.

[0501] "Memory Support" is a method of presenting questions and tasks to assess, maintain, and improve a user's memory.

[0502] "Schedule management" is a means of saving the schedule entered by the user and notifying them at the appropriate time.

[0503] "Abnormal behavior detection" is a method for monitoring and detecting in real time any behavior that deviates from a user's normal behavior pattern.

[0504] "Behavioral analysis and visualization" is a method of collecting and analyzing users' daily behavioral data and displaying the results in graphs and charts.

[0505] "Workers" is a broad term referring to workers in factories and other places.

[0506] "Cognitive support" is a method of presenting questions and tasks to assess workers' cognitive abilities and to maintain and improve them.

[0507] "Schedule and meeting notifications" are a means of notifying workers of their shifts and scheduled meetings at the appropriate time.

[0508] "Detection of non-responsiveness and abnormal behavior" refers to the means of detecting when a worker does not behave or respond normally or when the worker behaves abnormally.

[0509] "Emotional state monitoring" is a means of monitoring and analyzing workers' emotional states in real time.

[0510] "Providing support" is a means of providing encouragement and appropriate responses that take into account the worker's emotional state.

[0511] System Program

[0512] The system for this application consists of a number of hardware and software components. The system has the following functions:

[0513] 1. Generative AI: Has the ability to generate new information and solutions based on data.

[0514] 2. Memory support for the elderly: Ask questions and collect answers to support the memory of the elderly in their daily lives.

[0515] 3. Managing schedules for seniors: Save schedules entered by seniors and notify them at the appropriate time.

[0516] 4. Detection of abnormal behavior of elderly people: Detects and notifies abnormal behavior of elderly people in real time.

[0517] 5. Analysis and visualization of elderly people's behavior: Collect and analyze data on the daily behavior of elderly people and visualize the results.

[0518] 6. Supporting workers' cognitive function: Presents questions and tasks to assess, maintain, and improve the cognitive function of factory workers.

[0519] 7. Worker Schedule and Meeting Notification: Providing timely notification of factory workers' shift and meeting schedules.

[0520] 8. Detecting unresponsiveness or abnormal behavior of workers: Detecting unresponsiveness or abnormal behavior of factory workers and notifying management.

[0521] 9. Monitoring workers' emotional states: Monitor the emotional states of factory workers in real time and provide encouragement and support.

[0522] Hardware and Software

[0523] 1. Hardware:

[0524] The company will use devices designed for use by the elderly and factory robots that work in collaboration with workers within the factory.

[0525] 2. Software:

[0526] Real-time data processing server equipped with generative AI models

[0527] Analysis software using AI algorithms

[0528] Processing logic written in Python scripts

[0529] Process Overview

[0530] 1. Memory support

[0531] The server generates specific questions and sends them to the terminal, which displays the questions to the user and collects the user's answers and sends them to the server, which records and analyzes the answer data.

[0532] 2. Schedule Management

[0533] The server receives and stores the schedule data entered by the user, and when the designated time approaches, the server sends a corresponding reminder notification to the terminal, which then displays the notification to the user.

[0534] 3. Abnormal behavior detection

[0535] The device uses sensors to monitor the daily activities of elderly people and workers, and if an abnormality is detected, the data is sent to a server, which then sends a warning message to pre-set contacts.

[0536] 4. Behavioral analysis and visualization

[0537] The device collects daily activity data (e.g., walking distance, meal contents, sleep time, etc.) and periodically sends it to a server. The server analyzes the received data and visualizes the results as graphs and charts. This visualized data is displayed on a dashboard that can be viewed by the user and their family.

[0538] 5. Monitoring your emotional state

[0539] The device analyzes the user's voice and facial expressions to determine the user's emotional state in real time. The emotional analysis data is sent to the server, which then generates an appropriate response and sends it to the device. The device then displays the generated dialogue to the user and, if necessary, sends a message of safety confirmation or encouragement based on the user's emotional state.

[0540] Examples and prompts

[0541] 1. When an elderly person answers the device, "I had bread and coffee for breakfast today," the device sends this answer to the server, which then converts the data into numerical values ​​and records them.

[0542] 2. When a factory worker enters "I have a meeting tomorrow at 10:00 AM" into their schedule, the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[0543] Example prompt:

[0544] Ask "What did you have for lunch yesterday?" and get the user's answer. Then send the answer data to the server and store it along with the analysis results.

[0545] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0546] Step 1:

[0547] The device displays a specific question to the user every morning. For example, "What did you have for lunch yesterday?" This question is obtained from the server. The server generates question data using a generative AI model and sends it to the device.

[0548] Step 2:

[0549] The user inputs an answer to the question displayed on the terminal. For example, the user answers "I ate a sandwich." This input data is used as data for memory support for the user.

[0550] Step 3:

[0551] The device sends the user's response data to the server, which then processes the data by converting it into a number or categorizing it. For example, it converts "sandwich" into a specific number or category.

[0552] Step 4:

[0553] The server evaluates the user's cognitive function based on the saved response data, compares it with past responses stored in the database, calculates progress and changes, and generates analysis results using a generative AI model.

[0554] Step 5:

[0555] The server visualizes the analysis results as graphs and charts and sends them to the device, where users and their families can view the results on a dashboard.

[0556] Step 6:

[0557] A user inputs schedule data into a terminal. For example, the user inputs "I have a meeting tomorrow at 10:00 AM." This input data is used as schedule management data.

[0558] Step 7:

[0559] The terminal sends the input schedule data to the server, which receives the data and sets it to send a reminder notification at the specified time.

[0560] Step 8:

[0561] When the specified reminder time approaches, the server generates a reminder notification and sends it to the device. For example, it creates a notification saying "I have a meeting in 10 minutes."

[0562] Step 9:

[0563] The device will display a reminder notification to the user, who can then review and modify the action based on the notification.

[0564] Step 10:

[0565] The device monitors the user's daily activity data in real time and detects abnormal behavior or unresponsiveness, such as when the user is unresponsive after their usual lunch break.

[0566] Step 11:

[0567] If any abnormal behavior is detected, the device will send the data to the server, which will then send a warning message to pre-defined contacts if it determines that something is wrong.

[0568] Step 12:

[0569] The device analyzes the user's voice and facial expressions to determine their emotional state. The emotion analysis data is sent to the server, which then generates an appropriate response. If the device detects that the user is depressed, it generates an encouraging message.

[0570] Step 13:

[0571] The server then sends the generated dialogue to the device, which displays it to the user and continues the conversation as needed. It may also send messages of safety confirmation or encouragement.

[0572] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0573] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search<url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0574] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0575] [Second embodiment]

[0576] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0577] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0578] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0579] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0580] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0581] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0582] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0583] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0584] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0585] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0586] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0587] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0588] The present invention is a system for the elderly equipped with productive artificial intelligence (generative AI) that comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[0589] Cognitive exercise features

[0590] 1. At a specific time each day, the device asks the elderly questions to assess their memory and cognitive function, such as "What did you have for breakfast today?"

[0591] 2. The user answers the displayed question by typing, for example, "I ate bread and coffee."

[0592] 3. The device sends the user's answer to the server.

[0593] 4. The server quantifies the received data and records it in a database. This data is used to track and analyze changes in the user's cognitive function.

[0594] Schedule reminder function

[0595] 1. The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM."

[0596] 2. The terminal sends this schedule data to the server.

[0597] 3. The server sets a reminder based on the received schedule data. When the specified time approaches, for example, a reminder notification is set 15 minutes before.

[0598] 4. When the reminder time arrives, the device will display a notification to the user, allowing the user to take the necessary action without forgetting the appointment.

[0599] Non-response and abnormality warning function

[0600] 1. The device monitors the elderly person's daily activity patterns, such as meal times, walking distance, and response status.

[0601] 2. The server analyzes the data collected daily and compares it with standard patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[0602] 3. If the server detects an abnormality, it will send a warning message to pre-defined contacts, detailing the abnormality and recommending a course of action.

[0603] Behavioral analysis and visualization features

[0604] 1. The device collects real-time data on the elderly person's daily activities, including walking distance, dietary habits, and sleep duration.

[0605] 2. The device periodically sends the collected data to the server.

[0606] 3. The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status.

[0607] 4. The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance.

[0608] 5. The device provides visualized data to the user and their family through a dashboard and notification function.

[0609] Specific examples

[0610] The user answers the question on the device by saying, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[0611] In this way, the present invention comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0612] The processing flow will be explained below.

[0613] Cognitive exercise features

[0614] Step 1:

[0615] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess the user's memory.

[0616] Step 2:

[0617] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[0618] Step 3:

[0619] The device sends the user's response data to the server in real time.

[0620] Step 4:

[0621] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0622] Step 5:

[0623] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0624] Schedule reminder function

[0625] Step 1:

[0626] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[0627] Step 2:

[0628] The terminal transmits the input schedule data to the server.

[0629] Step 3:

[0630] The server stores the received schedule data and sets reminders, such as notifying you 15 minutes before the specified time.

[0631] Step 4:

[0632] When the set reminder time arrives, the server sends a reminder notification to the device.

[0633] Step 5:

[0634] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[0635] Non-response and abnormality warning function

[0636] Step 1:

[0637] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0638] Step 2:

[0639] The device sends collected activity data, including sensor data and user input data, to a server.

[0640] Step 3:

[0641] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[0642] Step 4:

[0643] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check."

[0644] Step 5:

[0645] The device will notify contacts of the received warning message, allowing for a prompt response.

[0646] Behavioral analysis and visualization features

[0647] Step 1:

[0648] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[0649] Step 2:

[0650] The device periodically transmits the collected data to the server in real time.

[0651] Step 3:

[0652] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0653] Step 4:

[0654] The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by users and their families.

[0655] Step 5:

[0656] The device will then communicate the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[0657] Through the above steps, this system provides comprehensive support for the lives of elderly people, enabling them to live their daily lives with peace of mind.

[0658] Example 1

[0659] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0660] Maintaining memory, managing schedules, and monitoring health are important in the daily lives of elderly people. However, these tasks are difficult to perform alone and can be a burden on family members and caregivers. Furthermore, a system is needed to respond quickly when elderly people exhibit abnormal behavior in their daily lives. This invention provides a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0661] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0662] In this invention, the server includes means for generating specific questions, collecting and transmitting answers from users to the server, means for transmitting schedule data entered by the user to the server and for the server to set reminders, and means for collecting behavioral data in real time and providing visualized data to the user and their family through an interface, thereby enabling memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization in the daily lives of elderly people.

[0663] "Generative artificial intelligence" is an automated intelligent system that responds and analyzes based on user data input and interactions.

[0664] The "memory support means" is a function that asks specific questions to the user to help the user's cognitive functions.

[0665] The "schedule management means" is a function that manages the schedule entered by the user and notifies the user at a specific time.

[0666] The "abnormal behavior detection means" is a function that monitors the user's daily activities and detects abnormal behavior or patterns.

[0667] "Behavioral analysis" is the process of collecting and analyzing users' daily behavioral data.

[0668] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[0669] "Means for generating specific questions" refers to a function that uses a generative AI model to automatically create appropriate questions for the user.

[0670] The "means for collecting user responses" is a function that collects the responses entered by the user as data and transmits them to the server.

[0671] The "means for setting a reminder" is a function for managing a schedule so as to notify the user at a specific time based on the schedule input by the user.

[0672] "Means for collecting behavioral data in real time" refers to a function that uses sensors and devices to detect and record users' daily life behavior in real time.

[0673] "Means for providing visualized data to users and their families through an interface" refers to a function that visually displays collected and analyzed data and provides it to users and their families in the form of a dashboard or notification.

[0674] The present invention is a system equipped with generative artificial intelligence (generative AI) that provides comprehensive support for the daily lives of elderly people. This system provides integrated functions for memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization. Detailed embodiments of each function are described below.

[0675] Hardware Configuration

[0676] The system uses the following hardware components:

[0677] Terminal: A handheld device, such as a tablet or smartphone, that acts as an interface for user input.

[0678] Server: Cloud server or corporate server that analyzes, stores, and manages data.

[0679] Sensor devices: Wearable devices (e.g., smart watches) and home sensors that collect daily activity data of older adults.

[0680] Software Configuration

[0681] Generative AI models (e.g., GPT-4): Used to generate specific questions, providing users with the ability to ask appropriate questions in natural language.

[0682] Database management system: Stores and manages user response data and behavioral data.

[0683] Analysis algorithm: An AI algorithm that analyzes collected data and evaluates the user's health status and behavioral patterns.

[0684] Notification system: A system for notifying users or stakeholders of reminders and anomaly detection messages.

[0685] Memory support function

[0686] The device generates questions to assess memory and cognitive function at specific times each day. For example, a generative AI model is used to create a question such as "What did you have for breakfast today?" The user responds by typing "I had bread and coffee." The device sends this response to a server, which quantifies the data and records it in a database. This allows changes in cognitive function to be tracked and analyzed.

[0687] Schedule management function

[0688] The user inputs their schedule into the device. For example, they input an appointment to "go to the hospital tomorrow at 10:00 AM." The device sends this schedule data to the server, and the server sets a reminder for the specified time. When the specified time approaches, the device displays a notification to the user. This allows the user to remember their appointment.

[0689] Abnormal behavior detection function

[0690] The terminal monitors the elderly person's daily activity patterns through sensor devices, including meal times, walking distance, and response status. The server analyzes the collected data and compares it with standard patterns to detect abnormalities. For example, a lack of response during lunchtime is deemed abnormal. If an abnormality is detected, the server sends a warning message to family members or caregivers.

[0691] Behavioral analysis and visualization features

[0692] The device collects daily behavioral data of the elderly in real time and periodically transmits it to a server. The server then analyzes the received data using AI algorithms to evaluate behavioral patterns and health conditions. The analysis results are visualized as graphs and charts and provided to the user and their family via the device's dashboard.

[0693] Examples and prompts

[0694] Examples:

[0695] The user answers the question displayed on the device by saying, "I had bread and coffee for breakfast today." The device then sends this answer to the server, which then digitizes the data and records it.

[0696] When a user enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[0697] If daily activity is abnormal, the server will send an alert to the family.

[0698] Ultimately, behavioral patterns are visualized and the data is always available for reference.

[0699] Example prompt sentence:

[0700] "What did you have for breakfast today?"

[0701] "Please enter an appointment to go to the hospital tomorrow at 10 AM."

[0702] Check your walking distance.

[0703] In this way, the present invention realizes a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[0704] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0705] Cognitive exercise features

[0706] Step 1:

[0707] At a specific time each day, the device uses a generative AI model (e.g., GPT-4) to generate questions for the user to assess their memory and cognitive function. For example, a question might be generated: "What did you have for breakfast today?" Input: Current time, generative AI model. Output: Specific question.

[0708] Step 2:

[0709] The terminal displays the generated question to the user. The user inputs the answer using a keyboard or voice input. For example, the user answers "I ate bread and coffee." Input: The generated question. Output: The user's answer data.

[0710] Step 3:

[0711] The terminal collects the user's response data and sends it to the server using a secure protocol (e.g. HTTPS). Input: User's response data. Output: Data sent to the server.

[0712] Step 4:

[0713] The server analyzes the received response data and organizes it as numerical data. It analyzes specific keywords (e.g., bread, coffee) and tags them. Input: User response data. Output: Tagged numerical data.

[0714] Step 5:

[0715] The server records the analyzed and quantified data in a database, which is later used to track changes in the user's cognitive function. Input: Analyzed and quantified data. Output: Data recorded in the database.

[0716] Schedule reminder function

[0717] Step 1:

[0718] The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM." Input: User's schedule data. Output: Input schedule data.

[0719] Step 2:

[0720] The terminal sends this schedule data to the server. Input: User's schedule data. Output: Schedule data sent to the server.

[0721] Step 3:

[0722] The server sets a reminder based on the received schedule data. The reminder is set to notify 15 minutes before the scheduled time. Input: Schedule data. Output: Set reminder.

[0723] Step 4:

[0724] When the reminder time approaches, the device will remind the user with a pop-up notification or a voice notification. For example, it may notify the user that it is almost time to go to the hospital. Input: The set reminder. Output: The reminder notification.

[0725] Non-response and abnormality warning function

[0726] Step 1:

[0727] The device uses sensors and user input to monitor the elderly person's daily activity patterns, including meal times, walking distance, and responsiveness. Input: Activity data from sensors and user input. Output: Collected activity data.

[0728] Step 2:

[0729] The server analyzes the activity data collected periodically and compares it with standard patterns to detect anomalies. For example, if there is no response during lunch time, it will be judged as an anomaly. Input: Collected activity data. Output: Detected anomalous data.

[0730] Step 3:

[0731] When an abnormality is detected, the server sends a warning message to pre-defined contacts. This message describes the abnormality and the required action. Input: Detected abnormal data. Output: Warning message.

[0732] Behavioral analysis and visualization features

[0733] Step 1:

[0734] The device collects daily behavioral data of elderly people in real time, including walking distance, dietary habits, and sleep duration. Input: Behavioral data collected in real time. Output: Collected behavioral data.

[0735] Step 2:

[0736] The device periodically sends the collected behavioral data to the server. Input: Collected behavioral data. Output: Data sent to the server.

[0737] Step 3:

[0738] The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status. Input: Data sent to the server. Output: Analysis results.

[0739] Step 4:

[0740] The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance. Input: Analysis results. Output: Visualized data (graphs and charts).

[0741] Step 5:

[0742] The device provides visualized data in the form of a dashboard to the user and their family. Periodic reports are also delivered via the notification function. Input: Visualized data. Output: Reports via dashboards and notifications.

[0743] (Application example 1)

[0744] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0745] Current support systems for the elderly are limited to use at home and lack support when out and about, especially in physical stores. This can lead to elderly people getting lost in stores or forgetting important shopping lists. Furthermore, there is also the problem of not being able to respond immediately when abnormal behavior occurs in stores. These issues need to be resolved.

[0746] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0747] In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the elderly's daily memory, means for managing the elderly's schedule, means for detecting abnormal behavior of the elderly, means for analyzing and visualizing the elderly's daily behavior, means for displaying product information and prices when the elderly approaches a product shelf, means for detecting and notifying an abnormality when the elderly does not move within a certain period of time in the store, and means for collecting, analyzing, and visualizing data on the elderly's behavior in the store. This enables the elderly to shop safely and efficiently in physical stores, and allows for quick response even if abnormal behavior occurs.

[0748] "Generative AI" refers to AI that has the ability to automatically generate new information and judgments using data.

[0749] "Memory support means" refers to a device or system that provides a function to support the memory of elderly people by displaying specific questions and collecting and analyzing the user's answers to those questions.

[0750] "Schedule management means" refers to a device or system that saves the schedule entered by the user and notifies the user at the specified time.

[0751] "Abnormal behavior detection means" refers to a device or system that has the function of monitoring the behavior of an elderly person and detecting abnormal behavior that deviates from normal patterns.

[0752] The term "behavioral analysis means" refers to a device or system that has the function of collecting and analyzing daily behavioral data of elderly people.

[0753] "Visualization means" refers to a device or system that has the function of displaying analyzed behavioral data in a visual format such as a graph or chart, and providing it in a form that is easy for users to understand.

[0754] "Product information display means" refers to a device or system that has the function of displaying product information and prices when an elderly person approaches a product shelf.

[0755] "Abnormality detection and notification means" refers to a device or system that has the function of detecting an abnormality and issuing a notification if an elderly person does not move within a certain period of time inside the store.

[0756] "Behavioral data collection means" refers to a device or system that has the function of continuously collecting data on the behavior of elderly people in a store.

[0757] The term "analysis result visualization means" refers to a device or system that has the function of analyzing collected behavioral data and visually expressing the results to provide to the user.

[0758] The present invention aims to provide support for elderly people in brick-and-mortar stores, particularly by providing a system that maintains cognitive abilities, monitors health status, and assists in efficient shopping. To achieve this, the following hardware and software are used:

[0759] Hardware and Software

[0760] Hardware:

[0761] Smartphone

[0762] Smart Glasses

[0763] server

[0764] software:

[0765] Programming language: Python

[0766] Machine learning library: TensorFlow

[0767] Web framework: Flask

[0768] Database: SQLite

[0769] Notification Service: Twilio API

[0770] System flow

[0771] 1. Memory Support Tools:

[0772] The server periodically generates questions for the elderly, which are generated based on a generative AI model.

[0773] The device (smartphone or smart glasses) displays this question to the elderly person and collects their answer.

[0774] The server stores the received response data and analyzes it to evaluate the cognitive status of the elderly person.

[0775] 2. Scheduling Management Methods:

[0776] The user inputs the schedule into the terminal.

[0777] The server stores this schedule data and uses the Twilio API to send reminder notifications when the specified time approaches.

[0778] 3. Abnormal Operation Detection Method:

[0779] The device uses sensors in smartphones and smart glasses to monitor the movements of elderly people.

[0780] The server analyzes the received operational data and sends a warning message to the specified contacts if an abnormality is detected.

[0781] 4. Behavioral analysis and visualization tools:

[0782] The terminal collects data on the elderly's behavior within the store and sends it to a server.

[0783] The server analyzes the collected data using TensorFlow to evaluate behavioral patterns and health status.

[0784] The analysis results will be visualized as graphs and charts using Flask, making them accessible to the elderly and their families.

[0785] 5. Product information display means:

[0786] When an elderly person approaches a store shelf, the device uses RFID or NFC to obtain product information and displays it on the smart glasses or smartphone.

[0787] 6. Anomaly detection notification method:

[0788] The device detects this abnormality if the elderly person does not move within a certain period of time.

[0789] The server analyzes the abnormal condition and sends a notification using the Twilio API.

[0790] Specific examples

[0791] For example, when an elderly person approaches the bread shelf, the smart glasses will display a message saying, "This product is bread. The price is 300 yen." If the elderly person sets a reminder to "not forget to take today's medicine," a notification will be sent at the scheduled time. If an abnormality is detected, a notification will be sent to store staff saying, "The customer has been stationary for a long time."

[0792] Prompt Sentence Examples

[0793] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[0794] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0795] Step 1:

[0796] server:

[0797] A generative AI model is used to generate specific questions to assess the elderly person's cognition and memory, such as "What did you have for breakfast today?", and the generated questions are sent to the device.

[0798] Input: User data, past response data

[0799] Output: Generated question text

[0800] Step 2:

[0801] Device:

[0802] The generated question is displayed to the user. The device uses a smartphone or smart glasses to visually present the question to the elderly.

[0803] Input: Generated question text

[0804] Output: Screen showing the question

[0805] Step 3:

[0806] User:

[0807] Enter your answer to the question. For example, you might enter, "I had bread and coffee for breakfast today." After you enter the answer, the data is recorded on the device.

[0808] Input: The text of the user's answer to the question

[0809] Output: User's answer text

[0810] Step 4:

[0811] Device:

[0812] The system receives the user's answers and sends them to the server, where the answers are stored in a database for later analysis.

[0813] Input: User's answer text

[0814] Output: Data sent to the server

[0815] Step 5:

[0816] server:

[0817] The received response data is analyzed to assess the user's cognitive function. This includes comparing it with past data and quantifying it to track memory fluctuations. The analysis results are stored in a database.

[0818] Input: User response data, past data

[0819] Output: Analysis result data

[0820] Step 6:

[0821] User:

[0822] Enter your schedule. For example, enter "I'll go to the hospital tomorrow at 10:00 AM." The entered data will be saved on your device.

[0823] Input: Schedule data (date, time, content)

[0824] Output: Schedule registration data to the terminal

[0825] Step 7:

[0826] Device:

[0827] The entered schedule data is sent to the server.

[0828] Input: Schedule data

[0829] Output: Data sent to the server

[0830] Step 8:

[0831] server:

[0832] It saves the received schedule data, generates reminder notifications when the specified time approaches, and sends notifications to the user using the Twilio API.

[0833] Input: Schedule data

[0834] Output: Reminder notification

[0835] Step 9:

[0836] Device:

[0837] The movement of elderly people is monitored using sensors on smartphones and smart glasses, and the collected data is sent to a server.

[0838] Input: Motion data (sensor data)

[0839] Output: Data sent to the server

[0840] Step 10:

[0841] server:

[0842] The received operational data is analyzed, and if an abnormality is detected, a warning message is sent to the specified contacts. Notifications are sent using the Twilio API.

[0843] Input: Operation data

[0844] Output: Warning notice

[0845] Step 11:

[0846] Device:

[0847] When an elderly person approaches a product shelf, product information is obtained using RFID or NFC and displayed on the smart glasses or smartphone.

[0848] Input: Product data (RFID / NFC tag information)

[0849] Output: Product information display

[0850] Step 12:

[0851] User:

[0852] The user checks the product information and purchases it if necessary. The user's behavior data and purchase data are recorded on the device.

[0853] Input: Product information, purchase action

[0854] Output: Purchase data

[0855] Step 13:

[0856] Device:

[0857] Data on the behavior of elderly people in the store is collected and sent to a server.

[0858] Input: Behavioral data (location information, movement history)

[0859] Output: Data sent to the server

[0860] Step 14:

[0861] server:

[0862] The collected behavioral data is analyzed, and the results are visualized to evaluate behavioral patterns and health conditions, generating reports that can be viewed by seniors and their families.

[0863] Input: Behavioral data

[0864] Output: Visualized data of analysis results (graphs, charts)

[0865] Prompt Sentence Examples

[0866] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[0867] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0868] The present invention is a system for the elderly equipped with generative artificial intelligence (generative AI) and an emotion engine, which comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[0869] Cognitive exercise features

[0870] 1. The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.). The questions evaluate the user's memory, such as "What did you have for breakfast today?"

[0871] 2. The user enters an answer to the question displayed on the terminal. For example, the user enters "I ate bread and coffee."

[0872] 3. The device sends the user's response data to the server in real time.

[0873] 4. The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0874] 5. The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0875] Schedule reminder function

[0876] 1. The user enters their schedule into the device. For example, they might enter, "I'm going to the hospital tomorrow at 10:00 AM."

[0877] 2. The terminal sends the entered schedule data to the server.

[0878] 3. The server saves the received schedule data and sets a reminder, such as notifying you 15 minutes before the specified time.

[0879] 4. When the set reminder time arrives, the server sends a reminder notification to the device.

[0880] 5. The device displays a reminder notification to the user, for example, a pop-up notification saying, "You have an appointment to go to the doctor in 10 minutes."

[0881] Non-response and abnormality warning function

[0882] 1. The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0883] 2. The device sends the collected activity data, including sensor data and user input data, to the server.

[0884] 3. The server analyzes the received data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[0885] 4. If the server detects an abnormality, it will send a warning message to pre-defined contacts, for example, "Mom was unresponsive at lunchtime. Please check on her."

[0886] 5. The device will notify the contacts of the received warning message, allowing for a prompt response.

[0887] Behavioral analysis and visualization features

[0888] 1. The device collects daily user behavior data, including walking distance, dietary intake, and sleep duration.

[0889] 2. The device periodically transmits the collected data to the server in real time.

[0890] 3. The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0891] 4. The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by the user and their family.

[0892] 5. The device notifies the user and their family of the visualized data, for example, by displaying a graph of the change in walking distance each week.

[0893] Emotion Engine Functions

[0894] 1. The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[0895] 2. The device sends emotion analysis data to the server, including voice data and facial expression data.

[0896] 3. The server generates an appropriate response based on the received emotion analysis data. For example, if the user is feeling down, it generates an encouraging message.

[0897] 4. The server sends the generated dialogue content to the terminal.

[0898] 5. The device displays the generated dialogue to the user, continues the conversation, and, if necessary, sends messages of encouragement or confirmation of the user's safety based on the user's emotional state.

[0899] Specific examples

[0900] The user answers the device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, and a reminder notification is displayed on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[0901] Furthermore, the device analyzes the user's emotions and, for example, if the user is feeling down, displays an encouraging message such as, "Are you OK? Is there anything I can help you with?" In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[0902] The processing flow will be explained below.

[0903] Cognitive exercise features

[0904] Step 1:

[0905] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess memory ability.

[0906] Step 2:

[0907] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[0908] Step 3:

[0909] The device sends the user's response data to the server in real time.

[0910] Step 4:

[0911] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[0912] Step 5:

[0913] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[0914] Schedule reminder function

[0915] Step 1:

[0916] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[0917] Step 2:

[0918] The terminal transmits the input schedule data to the server.

[0919] Step 3:

[0920] The server stores the received schedule data and sets a reminder, for example, to notify you 15 minutes before the specified time.

[0921] Step 4:

[0922] When the set reminder time arrives, the server sends a reminder notification to the device.

[0923] Step 5:

[0924] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[0925] Non-response and abnormality warning function

[0926] Step 1:

[0927] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[0928] Step 2:

[0929] The device sends collected activity data, including sensor data and user input data, to a server.

[0930] Step 3:

[0931] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[0932] Step 4:

[0933] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check on her."

[0934] Step 5:

[0935] The device will notify contacts of the received warning message, allowing for a prompt response.

[0936] Behavioral analysis and visualization features

[0937] Step 1:

[0938] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[0939] Step 2:

[0940] The device periodically transmits the collected data to the server in real time.

[0941] Step 3:

[0942] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[0943] Step 4:

[0944] The server visualizes the analysis results as graphs and charts, and the visualized data is displayed on a dashboard or similar.

[0945] Step 5:

[0946] The device then communicates the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[0947] Emotion Engine Functions

[0948] Step 1:

[0949] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[0950] Step 2:

[0951] The device sends emotion analysis data, including voice data and facial expression data, to the server.

[0952] Step 3:

[0953] The server generates appropriate responses based on the received emotion analysis data, for example, an encouraging message if the user is feeling down.

[0954] Step 4:

[0955] The server transmits the generated dialogue content to the terminal.

[0956] Step 5:

[0957] The device displays the generated dialogue content to the user, continues the dialogue appropriately, and, if necessary, sends messages of encouragement or confirmation of safety based on the user's emotional state.

[0958] As a specific example, if a user says, "I'm feeling a little down today," the device analyzes the user's voice and facial expressions to recognize the emotion. The device sends this data to a server, which then generates a dialogue such as, "Are you OK? Is there anything I can help you with?" and sends it back to the device. The device then displays this message to the user and the dialogue continues. In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[0959] Example 2

[0960] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0961] In the lives of elderly people, there is a growing need for early detection of memory decline, difficulty in schedule management, and abnormal behavior. Furthermore, analyzing and visualizing daily behavior is important for elderly people and their families. However, systems that comprehensively and effectively support these issues are still insufficient. Furthermore, there is a need for systems that can grasp the emotional state of elderly people and generate appropriate responses.

[0962] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0963] In this invention, the server includes a means for supporting the elderly's memory in daily life, a means for managing the elderly's schedule, a means for detecting abnormal behavior of the elderly, a means for analyzing and visualizing the elderly's daily behavior, and a means for analyzing the elderly's emotional state and generating appropriate responses. This enables not only memory support and schedule management, but also early detection of abnormal behavior, visualization of behavioral analysis, emotion analysis, and appropriate responses.

[0964] "Generative AI" is AI that generates sentences and dialogues using natural language processing and machine learning.

[0965] The "memory support means" is a function that evaluates and supports the user's memory by displaying specific questions to the user and collecting, recording, and analyzing the user's answers.

[0966] The "schedule management means" is a function that saves the schedule entered by the user and sends a reminder notification at a specified time.

[0967] The "abnormal behavior detection means" is a function that monitors the behavioral data of elderly people and issues an alert if any unusual behavior or reaction is detected.

[0968] The "behavioral analysis means" is a function that evaluates the user's behavioral patterns and health condition by collecting and analyzing the user's daily behavioral data.

[0969] "Visualization means" is a function that visually displays the results of behavioral analysis as graphs or charts.

[0970] The "emotional state analysis means" is a function that analyzes data such as voice and facial expression to determine the user's emotional state.

[0971] The "response generation means" is a function that generates appropriate dialogue content and messages based on the results of the emotional state analysis.

[0972] A "server" is a centralized computer system that stores, analyzes, and processes data.

[0973] A "terminal" is a computer device that is directly operated by a user and that communicates with a server to provide various functions.

[0974] This invention is a system for supporting the daily lives of elderly people, utilizing generative artificial intelligence to support memory, manage schedules, detect abnormal behavior, analyze and visualize behavior, and analyze emotional states. This system mainly consists of a server and a terminal, and monitors user input data and daily activity data in real time, analyzing and responding appropriately.

[0975] Hardware and Software

[0976] A server is a centralized computer system that stores, analyzes, and processes data. It requires powerful processors, memory, and large amounts of storage. The software required includes a database management system (DBMS), machine learning libraries (e.g., TensorFlow, PyTorch), data analysis tools, and visualization tools (e.g., Tableau).

[0977] A terminal is a computing device that is directly operated by a user, and is typically a compact tablet or smartphone. Terminals are equipped with various sensors (e.g., camera, microphone, accelerometer) and use these sensors to collect data. Terminal software includes a user interface, a data collection application, and a communication module.

[0978] Program processing

[0979] At a specific time each day, the device displays a question to the user to assess their memory. For example, "What did you have for breakfast today?" The user responds by answering "I had bread and coffee." The device then sends this response data to the server in real time. The server then converts the received data into a numerical value and stores it in a database. For example, "bread" is converted into a numerical value of 1, and "coffee" into a numerical value of 1. The server then analyzes the state of the user's cognitive function based on the saved data, comparing it with past data, and evaluates progress and changes.

[0980] When a user inputs their schedule into a device, for example, "I'm going to the hospital tomorrow at 10 AM," the device sends this schedule data to the server, which saves the received data and sets a reminder. A reminder is sent 15 minutes before the specified time. When the time for this reminder arrives, the server sends a reminder to the device, and the device displays a pop-up notification saying, "You have an appointment to go to the hospital in 10 minutes."

[0981] Furthermore, the device monitors the user's daily activity patterns. For example, it monitors meal times, walking distance, and response status in real time. The device sends the collected activity data to a server, which analyzes the received data and compares it with standard behavioral patterns to determine whether there are any abnormalities. If an abnormality is detected, for example, if there is no response during lunch, the server determines that this is an abnormality and sends a warning message to the specified contacts saying, "Mom did not respond during lunch. Please check." The device then notifies the contacts of the received warning message.

[0982] The server analyzes the behavioral data using AI algorithms to evaluate the user's behavioral patterns and health status. The analysis results are visualized as graphs and charts and displayed to the user and their family. For example, a graph showing changes in walking distance each week is displayed.

[0983] The device analyzes the user's voice and facial expressions in real time to determine their emotional state. For example, if the user is feeling down, the device sends this analysis data to the server, which then generates an encouraging message. The server then sends the generated dialogue content to the device, which then displays a message such as, "Are you OK? Is there anything I can help you with?"

[0984] Specific examples

[0985] For example, a user might answer to their device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital tomorrow at 10 a.m.", the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If their daily activities are abnormal, the server sends an alert to their family, and ultimately, their behavioral patterns are visualized and the data can be constantly referenced. Furthermore, the device analyzes the user's emotions. For example, if the user is feeling down, it displays an encouraging message such as, "Are you okay? Is there anything I can help you with?"

[0986] Prompt Sentence Examples

[0987] To generate a message of encouragement for a user who is feeling down, an example of a prompt to input to the AI ​​model is as follows:

[0988] "When the user is feeling down, generate an encouraging message. The user's name is Takahashi."

[0989] This allows the generative AI model to generate dialogue such as, "Takahashi-san, are you okay? Is there anything I can help you with?"

[0990] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0991] Cognitive exercise features

[0992] Step 1:

[0993] The device displays a question to assess memory ability every morning at 9:00, such as "What did you have for breakfast today?" The input is a specific time (e.g., 9:00), and the output is the display of the question.

[0994] Step 2:

[0995] The user inputs "I ate bread and coffee" into the terminal. The input is the user's answer, and the output is text data.

[0996] Step 3:

[0997] The terminal transmits the user's answers to the server in real time. The input is the user's answer text data, and the output is the transmitted data. This is done using a communication module.

[0998] Step 4:

[0999] The server digitizes the received text data and stores it in a database. The input is the received text data, and the output is the digitized data and its storage. For example, "bread" is digitized as 1 and "coffee" is digitized as 1.

[1000] Step 5:

[1001] The server analyzes the user's cognitive function status based on the stored data and compares it with past data. The input is the stored data, and the output is the analysis results. Machine learning algorithms are used for these analyses.

[1002] Schedule reminder function

[1003] Step 1:

[1004] The user inputs an appointment into the terminal, such as "I will go to the hospital tomorrow at 10:00 AM." The input is the user's schedule data, and the output is the input data.

[1005] Step 2:

[1006] The terminal transmits the input schedule data to the server. The input is the schedule data, and the output is the transmission data. The data is transmitted via the communication module.

[1007] Step 3:

[1008] The server saves the received data and sets the reminder time. The input is the received data and the output is the reminder setting. For example, set it to notify 15 minutes before the specified time.

[1009] Step 4:

[1010] The server sends a notification to the device when the specified reminder time arrives. The input is the reminder time and the output is the notification data.

[1011] Step 5:

[1012] The device displays a pop-up notification saying, "I have an appointment to go to the hospital in 10 minutes." The input is the notification data, and the output is the pop-up display.

[1013] Non-response and abnormality warning function

[1014] Step 1:

[1015] The device monitors daily user activity data such as meal times, walking distance, and response status. The input is sensor data and user operation data, and the output is collected data.

[1016] Step 2:

[1017] The terminal transmits the collected activity data to the server in real time. The input is the collected data and the output is the transmitted data.

[1018] Step 3:

[1019] The server analyzes the received data and compares it with standard behavioral patterns to detect anomalies. The input is the received data, and the output is the analysis results. For example, if there is no response during lunch time, an anomaly is detected.

[1020] Step 4:

[1021] If an abnormality is detected, the server sends a warning message to pre-defined contacts. The input is the analysis result, and the output is the warning message. For example, "Mom was unresponsive at lunchtime. Please check."

[1022] Step 5:

[1023] The terminal notifies the contact of the received warning message. The input is the warning message and the output is the notification display.

[1024] Behavioral analysis and visualization features

[1025] Step 1:

[1026] The device collects daily user behavior data, including walking distance, dietary habits, sleep duration, etc. The input is sensor data and user operation data, and the output is the collected data.

[1027] Step 2:

[1028] The terminal transmits the collected data to the server in real time. The input is the collected data and the output is the transmitted data.

[1029] Step 3:

[1030] The server uses AI algorithms to analyze the received data and evaluate the user's behavioral patterns and health status. The input is the received data, and the output is the analysis results.

[1031] Step 4:

[1032] The server visualizes the analysis results as graphs and charts and displays them on a dashboard. The input is the analysis results, and the output is the visualized data. For example, a graph showing changes in walking distance each week is displayed.

[1033] Step 5:

[1034] The terminal notifies the user and their family of the visualized data. The input is the visualized data, and the output is a notification display.

[1035] Emotion Engine Functions

[1036] Step 1:

[1037] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state. The input is voice data and facial expression data, and the output is emotion analysis data.

[1038] Step 2:

[1039] The device transmits the emotion analysis data to the server in real time. The input is the emotion analysis data, and the output is the transmitted data.

[1040] Step 3:

[1041] The server generates appropriate dialogue content based on the received data. The input is emotion analysis data, and the output is the generated dialogue content. For example, if the user is feeling down, it generates an encouraging message.

[1042] Step 4:

[1043] The server sends the generated dialogue content to the terminal. The input is the generated dialogue content, and the output is the transmitted data.

[1044] Step 5:

[1045] The terminal displays the generated dialogue to the user and continues the conversation. The input is the transmitted data, and the output is the displayed dialogue. For example, it displays a message such as "Are you OK? Is there anything I can help you with?"

[1046] (Application example 2)

[1047] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1048] There is a need for support for the daily lives of the elderly, as well as cognitive function support, schedule management, abnormal behavior detection, and emotion monitoring for factory workers. Current systems have difficulty providing these functions comprehensively, resulting in increased human resources and management costs. Furthermore, it is difficult to detect abnormal behavior and emotional states of elderly people and workers in real time and respond quickly. For this reason, a comprehensive support system is needed to ensure that elderly people and workers can live their daily lives and work with peace of mind.

[1049] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the memory of elderly people in their daily lives, means for managing the schedule of elderly people, means for detecting abnormal movements of elderly people, means for analyzing and visualizing the daily behavior of elderly people, means for supporting the cognitive function of workers, means for notifying factory workers of schedules and meetings, means for detecting and notifying non-response and abnormal behavior of workers, and means for monitoring the emotional state of workers and providing support. This enables comprehensive support for elderly people and workers.

[1050] "Generative AI" is AI that has the ability to generate new information and solutions based on data.

[1051] "Elderly people" is a broad term referring to users who provide support for the daily lives of the elderly.

[1052] "Memory Support" is a method of presenting questions and tasks to assess, maintain, and improve a user's memory.

[1053] "Schedule management" is a means of saving the schedule entered by the user and notifying them at the appropriate time.

[1054] "Abnormal behavior detection" is a method for monitoring and detecting in real time any behavior that deviates from a user's normal behavior pattern.

[1055] "Behavioral analysis and visualization" is a method of collecting and analyzing users' daily behavioral data and displaying the results in graphs and charts.

[1056] "Workers" is a broad term referring to workers in factories and other places.

[1057] "Cognitive support" is a method of presenting questions and tasks to assess workers' cognitive abilities and to maintain and improve them.

[1058] "Schedule and meeting notifications" are a means of notifying workers of their shifts and scheduled meetings at the appropriate time.

[1059] "Detection of non-responsiveness and abnormal behavior" refers to the means of detecting when a worker does not behave or respond normally or when the worker behaves abnormally.

[1060] "Emotional state monitoring" is a means of monitoring and analyzing workers' emotional states in real time.

[1061] "Providing support" is a means of providing encouragement and appropriate responses that take into account the worker's emotional state.

[1062] System Program

[1063] The system for this application consists of a number of hardware and software components. The system has the following functions:

[1064] 1. Generative AI: Has the ability to generate new information and solutions based on data.

[1065] 2. Memory support for the elderly: Ask questions and collect answers to support the memory of the elderly in their daily lives.

[1066] 3. Managing schedules for seniors: Save schedules entered by seniors and notify them at the appropriate time.

[1067] 4. Detection of abnormal behavior of elderly people: Detects and notifies abnormal behavior of elderly people in real time.

[1068] 5. Analysis and visualization of elderly people's behavior: Collect and analyze data on the daily behavior of elderly people and visualize the results.

[1069] 6. Supporting workers' cognitive function: Presents questions and tasks to assess, maintain, and improve the cognitive function of factory workers.

[1070] 7. Worker Schedule and Meeting Notification: Providing timely notification of factory workers' shift and meeting schedules.

[1071] 8. Detecting unresponsiveness or abnormal behavior of workers: Detecting unresponsiveness or abnormal behavior of factory workers and notifying management.

[1072] 9. Monitoring workers' emotional states: Monitor the emotional states of factory workers in real time and provide encouragement and support.

[1073] Hardware and Software

[1074] 1. Hardware:

[1075] The company will use devices designed for use by the elderly and factory robots that work in collaboration with workers within the factory.

[1076] 2. Software:

[1077] Real-time data processing server equipped with generative AI models

[1078] Analysis software using AI algorithms

[1079] Processing logic written in Python scripts

[1080] Process Overview

[1081] 1. Memory support

[1082] The server generates specific questions and sends them to the terminal, which displays the questions to the user and collects the user's answers and sends them to the server, which records and analyzes the answer data.

[1083] 2. Schedule Management

[1084] The server receives and stores the schedule data entered by the user, and when the designated time approaches, the server sends a corresponding reminder notification to the terminal, which then displays the notification to the user.

[1085] 3. Abnormal behavior detection

[1086] The device uses sensors to monitor the daily activities of elderly people and workers, and if an abnormality is detected, the data is sent to a server, which then sends a warning message to pre-set contacts.

[1087] 4. Behavioral analysis and visualization

[1088] The device collects daily activity data (e.g., walking distance, meal contents, sleep time, etc.) and periodically sends it to a server. The server analyzes the received data and visualizes the results as graphs and charts. This visualized data is displayed on a dashboard that can be viewed by the user and their family.

[1089] 5. Monitoring your emotional state

[1090] The device analyzes the user's voice and facial expressions to determine the user's emotional state in real time. The emotional analysis data is sent to the server, which then generates an appropriate response and sends it to the device. The device then displays the generated dialogue to the user and, if necessary, sends a message of safety confirmation or encouragement based on the user's emotional state.

[1091] Examples and prompts

[1092] 1. When an elderly person answers the device, "I had bread and coffee for breakfast today," the device sends this answer to the server, which then converts the data into numerical values ​​and records them.

[1093] 2. When a factory worker enters "I have a meeting tomorrow at 10:00 AM" into their schedule, the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[1094] Example prompt:

[1095] Ask "What did you have for lunch yesterday?" and get the user's answer. Then send the answer data to the server and store it along with the analysis results.

[1096] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1097] Step 1:

[1098] The device displays a specific question to the user every morning. For example, "What did you have for lunch yesterday?" This question is obtained from the server. The server generates question data using a generative AI model and sends it to the device.

[1099] Step 2:

[1100] The user inputs an answer to the question displayed on the terminal. For example, the user answers "I ate a sandwich." This input data is used as data for memory support for the user.

[1101] Step 3:

[1102] The device sends the user's response data to the server, which then processes the data by converting it into a number or categorizing it. For example, it converts "sandwich" into a specific number or category.

[1103] Step 4:

[1104] The server evaluates the user's cognitive function based on the saved response data, compares it with past responses stored in the database, calculates progress and changes, and generates analysis results using a generative AI model.

[1105] Step 5:

[1106] The server visualizes the analysis results as graphs and charts and sends them to the device, where users and their families can view the results on a dashboard.

[1107] Step 6:

[1108] A user inputs schedule data into a terminal. For example, the user inputs "I have a meeting tomorrow at 10:00 AM." This input data is used as schedule management data.

[1109] Step 7:

[1110] The terminal sends the input schedule data to the server, which receives the data and sets it to send a reminder notification at the specified time.

[1111] Step 8:

[1112] When the specified reminder time approaches, the server generates a reminder notification and sends it to the device. For example, it creates a notification saying "I have a meeting in 10 minutes."

[1113] Step 9:

[1114] The device will display a reminder notification to the user, who can then review and modify the action based on the notification.

[1115] Step 10:

[1116] The device monitors the user's daily activity data in real time and detects abnormal behavior or unresponsiveness, such as when the user is unresponsive after their usual lunch break.

[1117] Step 11:

[1118] If any abnormal behavior is detected, the device will send the data to the server, which will then send a warning message to pre-defined contacts if it determines that something is wrong.

[1119] Step 12:

[1120] The device analyzes the user's voice and facial expressions to determine their emotional state. The emotion analysis data is sent to the server, which then generates an appropriate response. If the device detects that the user is depressed, it generates an encouraging message.

[1121] Step 13:

[1122] The server then sends the generated dialogue to the device, which displays it to the user and continues the conversation as needed. It may also send messages of safety confirmation or encouragement.

[1123] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1124] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1125] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1126] [Third embodiment]

[1127] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1128] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1129] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1130] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1131] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1132] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1133] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1134] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1135] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1136] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1137] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1138] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1139] The present invention is a system for the elderly equipped with productive artificial intelligence (generative AI) that comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[1140] Cognitive exercise features

[1141] 1. At a specific time each day, the device asks the elderly questions to assess their memory and cognitive function, such as "What did you have for breakfast today?"

[1142] 2. The user answers the displayed question by typing, for example, "I ate bread and coffee."

[1143] 3. The device sends the user's answer to the server.

[1144] 4. The server quantifies the received data and records it in a database. This data is used to track and analyze changes in the user's cognitive function.

[1145] Schedule reminder function

[1146] 1. The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM."

[1147] 2. The terminal sends this schedule data to the server.

[1148] 3. The server sets a reminder based on the received schedule data. When the specified time approaches, for example, a reminder notification is set 15 minutes before.

[1149] 4. When the reminder time arrives, the device will display a notification to the user, allowing the user to take the necessary action without forgetting the appointment.

[1150] Non-response and abnormality warning function

[1151] 1. The device monitors the elderly person's daily activity patterns, such as meal times, walking distance, and response status.

[1152] 2. The server analyzes the data collected daily and compares it with standard patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[1153] 3. If the server detects an abnormality, it will send a warning message to pre-defined contacts, detailing the abnormality and recommending a course of action.

[1154] Behavioral analysis and visualization features

[1155] 1. The device collects real-time data on the elderly person's daily activities, including walking distance, dietary habits, and sleep duration.

[1156] 2. The device periodically sends the collected data to the server.

[1157] 3. The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status.

[1158] 4. The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance.

[1159] 5. The device provides visualized data to the user and their family through a dashboard and notification function.

[1160] Specific examples

[1161] The user answers the question on the device by saying, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[1162] In this way, the present invention comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1163] The processing flow will be explained below.

[1164] Cognitive exercise features

[1165] Step 1:

[1166] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess the user's memory.

[1167] Step 2:

[1168] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[1169] Step 3:

[1170] The device sends the user's response data to the server in real time.

[1171] Step 4:

[1172] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[1173] Step 5:

[1174] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[1175] Schedule reminder function

[1176] Step 1:

[1177] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[1178] Step 2:

[1179] The terminal transmits the input schedule data to the server.

[1180] Step 3:

[1181] The server stores the received schedule data and sets reminders, such as notifying you 15 minutes before the specified time.

[1182] Step 4:

[1183] When the set reminder time arrives, the server sends a reminder notification to the device.

[1184] Step 5:

[1185] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[1186] Non-response and abnormality warning function

[1187] Step 1:

[1188] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[1189] Step 2:

[1190] The device sends collected activity data, including sensor data and user input data, to a server.

[1191] Step 3:

[1192] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[1193] Step 4:

[1194] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check."

[1195] Step 5:

[1196] The device will notify contacts of the received warning message, allowing for a prompt response.

[1197] Behavioral analysis and visualization features

[1198] Step 1:

[1199] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[1200] Step 2:

[1201] The device periodically transmits the collected data to the server in real time.

[1202] Step 3:

[1203] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[1204] Step 4:

[1205] The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by users and their families.

[1206] Step 5:

[1207] The device will then communicate the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[1208] Through the above steps, this system provides comprehensive support for the lives of elderly people, enabling them to live their daily lives with peace of mind.

[1209] Example 1

[1210] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1211] Maintaining memory, managing schedules, and monitoring health are important in the daily lives of elderly people. However, these tasks are difficult to perform alone and can be a burden on family members and caregivers. Furthermore, a system is needed to respond quickly when elderly people exhibit abnormal behavior in their daily lives. This invention provides a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1212] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1213] In this invention, the server includes means for generating specific questions, collecting and transmitting answers from users to the server, means for transmitting schedule data entered by the user to the server and for the server to set reminders, and means for collecting behavioral data in real time and providing visualized data to the user and their family through an interface, thereby enabling memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization in the daily lives of elderly people.

[1214] "Generative artificial intelligence" is an automated intelligent system that responds and analyzes based on user data input and interactions.

[1215] The "memory support means" is a function that asks specific questions to the user to help the user's cognitive functions.

[1216] The "schedule management means" is a function that manages the schedule entered by the user and notifies the user at a specific time.

[1217] The "abnormal behavior detection means" is a function that monitors the user's daily activities and detects abnormal behavior or patterns.

[1218] "Behavioral analysis" is the process of collecting and analyzing users' daily behavioral data.

[1219] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[1220] "Means for generating specific questions" refers to a function that uses a generative AI model to automatically create appropriate questions for the user.

[1221] The "means for collecting user responses" is a function that collects the responses entered by the user as data and transmits them to the server.

[1222] The "means for setting a reminder" is a function for managing a schedule so as to notify the user at a specific time based on the schedule input by the user.

[1223] "Means for collecting behavioral data in real time" refers to a function that uses sensors and devices to detect and record users' daily life behavior in real time.

[1224] "Means for providing visualized data to users and their families through an interface" refers to a function that visually displays collected and analyzed data and provides it to users and their families in the form of a dashboard or notification.

[1225] The present invention is a system equipped with generative artificial intelligence (generative AI) that provides comprehensive support for the daily lives of elderly people. This system provides integrated functions for memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization. Detailed embodiments of each function are described below.

[1226] Hardware Configuration

[1227] The system uses the following hardware components:

[1228] Terminal: A handheld device, such as a tablet or smartphone, that acts as an interface for user input.

[1229] Server: Cloud server or corporate server that analyzes, stores, and manages data.

[1230] Sensor devices: Wearable devices (e.g., smart watches) and home sensors that collect daily activity data of older adults.

[1231] Software Configuration

[1232] Generative AI models (e.g., GPT-4): Used to generate specific questions, providing users with the ability to ask appropriate questions in natural language.

[1233] Database management system: Stores and manages user response data and behavioral data.

[1234] Analysis algorithm: An AI algorithm that analyzes collected data and evaluates the user's health status and behavioral patterns.

[1235] Notification system: A system for notifying users or stakeholders of reminders and anomaly detection messages.

[1236] Memory support function

[1237] The device generates questions to assess memory and cognitive function at specific times each day. For example, a generative AI model is used to create a question such as "What did you have for breakfast today?" The user responds by typing "I had bread and coffee." The device sends this response to a server, which quantifies the data and records it in a database. This allows changes in cognitive function to be tracked and analyzed.

[1238] Schedule management function

[1239] The user inputs their schedule into the device. For example, they input an appointment to "go to the hospital tomorrow at 10:00 AM." The device sends this schedule data to the server, and the server sets a reminder for the specified time. When the specified time approaches, the device displays a notification to the user. This allows the user to remember their appointment.

[1240] Abnormal behavior detection function

[1241] The terminal monitors the elderly person's daily activity patterns through sensor devices, including meal times, walking distance, and response status. The server analyzes the collected data and compares it with standard patterns to detect abnormalities. For example, a lack of response during lunchtime is deemed abnormal. If an abnormality is detected, the server sends a warning message to family members or caregivers.

[1242] Behavioral analysis and visualization features

[1243] The device collects daily behavioral data of the elderly in real time and periodically transmits it to a server. The server then analyzes the received data using AI algorithms to evaluate behavioral patterns and health conditions. The analysis results are visualized as graphs and charts and provided to the user and their family via the device's dashboard.

[1244] Examples and prompts

[1245] Examples:

[1246] The user answers the question displayed on the device by saying, "I had bread and coffee for breakfast today." The device then sends this answer to the server, which then digitizes the data and records it.

[1247] When a user enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[1248] If daily activity is abnormal, the server will send an alert to the family.

[1249] Ultimately, behavioral patterns are visualized and the data is always available for reference.

[1250] Example prompt sentence:

[1251] "What did you have for breakfast today?"

[1252] "Please enter an appointment to go to the hospital tomorrow at 10 AM."

[1253] Check your walking distance.

[1254] In this way, the present invention realizes a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1255] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1256] Cognitive exercise features

[1257] Step 1:

[1258] At a specific time each day, the device uses a generative AI model (e.g., GPT-4) to generate questions for the user to assess their memory and cognitive function. For example, a question might be generated: "What did you have for breakfast today?" Input: Current time, generative AI model. Output: Specific question.

[1259] Step 2:

[1260] The terminal displays the generated question to the user. The user inputs the answer using a keyboard or voice input. For example, the user answers "I ate bread and coffee." Input: The generated question. Output: The user's answer data.

[1261] Step 3:

[1262] The terminal collects the user's response data and sends it to the server using a secure protocol (e.g. HTTPS). Input: User's response data. Output: Data sent to the server.

[1263] Step 4:

[1264] The server analyzes the received response data and organizes it as numerical data. It analyzes specific keywords (e.g., bread, coffee) and tags them. Input: User response data. Output: Tagged numerical data.

[1265] Step 5:

[1266] The server records the analyzed and quantified data in a database, which is later used to track changes in the user's cognitive function. Input: Analyzed and quantified data. Output: Data recorded in the database.

[1267] Schedule reminder function

[1268] Step 1:

[1269] The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM." Input: User's schedule data. Output: Input schedule data.

[1270] Step 2:

[1271] The terminal sends this schedule data to the server. Input: User's schedule data. Output: Schedule data sent to the server.

[1272] Step 3:

[1273] The server sets a reminder based on the received schedule data. The reminder is set to notify 15 minutes before the scheduled time. Input: Schedule data. Output: Set reminder.

[1274] Step 4:

[1275] When the reminder time approaches, the device will remind the user with a pop-up notification or a voice notification. For example, it may notify the user that it is almost time to go to the hospital. Input: The set reminder. Output: The reminder notification.

[1276] Non-response and abnormality warning function

[1277] Step 1:

[1278] The device uses sensors and user input to monitor the elderly person's daily activity patterns, including meal times, walking distance, and responsiveness. Input: Activity data from sensors and user input. Output: Collected activity data.

[1279] Step 2:

[1280] The server analyzes the activity data collected periodically and compares it with standard patterns to detect anomalies. For example, if there is no response during lunch time, it will be judged as an anomaly. Input: Collected activity data. Output: Detected anomalous data.

[1281] Step 3:

[1282] When an abnormality is detected, the server sends a warning message to pre-defined contacts. This message describes the abnormality and the required action. Input: Detected abnormal data. Output: Warning message.

[1283] Behavioral analysis and visualization features

[1284] Step 1:

[1285] The device collects daily behavioral data of elderly people in real time, including walking distance, dietary habits, and sleep duration. Input: Behavioral data collected in real time. Output: Collected behavioral data.

[1286] Step 2:

[1287] The device periodically sends the collected behavioral data to the server. Input: Collected behavioral data. Output: Data sent to the server.

[1288] Step 3:

[1289] The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status. Input: Data sent to the server. Output: Analysis results.

[1290] Step 4:

[1291] The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance. Input: Analysis results. Output: Visualized data (graphs and charts).

[1292] Step 5:

[1293] The device provides visualized data in the form of a dashboard to the user and their family. Periodic reports are also delivered via the notification function. Input: Visualized data. Output: Reports via dashboards and notifications.

[1294] (Application example 1)

[1295] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1296] Current support systems for the elderly are limited to use at home and lack support when out and about, especially in physical stores. This can lead to elderly people getting lost in stores or forgetting important shopping lists. Furthermore, there is also the problem of not being able to respond immediately when abnormal behavior occurs in stores. These issues need to be resolved.

[1297] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1298] In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the elderly's daily memory, means for managing the elderly's schedule, means for detecting abnormal behavior of the elderly, means for analyzing and visualizing the elderly's daily behavior, means for displaying product information and prices when the elderly approaches a product shelf, means for detecting and notifying an abnormality when the elderly does not move within a certain period of time in the store, and means for collecting, analyzing, and visualizing data on the elderly's behavior in the store. This enables the elderly to shop safely and efficiently in physical stores, and allows for quick response even if abnormal behavior occurs.

[1299] "Generative AI" refers to AI that has the ability to automatically generate new information and judgments using data.

[1300] "Memory support means" refers to a device or system that provides a function to support the memory of elderly people by displaying specific questions and collecting and analyzing the user's answers to those questions.

[1301] "Schedule management means" refers to a device or system that saves the schedule entered by the user and notifies the user at the specified time.

[1302] "Abnormal behavior detection means" refers to a device or system that has the function of monitoring the behavior of an elderly person and detecting abnormal behavior that deviates from normal patterns.

[1303] The term "behavioral analysis means" refers to a device or system that has the function of collecting and analyzing daily behavioral data of elderly people.

[1304] "Visualization means" refers to a device or system that has the function of displaying analyzed behavioral data in a visual format such as a graph or chart, and providing it in a form that is easy for users to understand.

[1305] "Product information display means" refers to a device or system that has the function of displaying product information and prices when an elderly person approaches a product shelf.

[1306] "Abnormality detection and notification means" refers to a device or system that has the function of detecting an abnormality and issuing a notification if an elderly person does not move within a certain period of time inside the store.

[1307] "Behavioral data collection means" refers to a device or system that has the function of continuously collecting data on the behavior of elderly people in a store.

[1308] The term "analysis result visualization means" refers to a device or system that has the function of analyzing collected behavioral data and visually expressing the results to provide to the user.

[1309] The present invention aims to provide support for elderly people in brick-and-mortar stores, particularly by providing a system that maintains cognitive abilities, monitors health status, and assists in efficient shopping. To achieve this, the following hardware and software are used:

[1310] Hardware and Software

[1311] Hardware:

[1312] Smartphone

[1313] Smart Glasses

[1314] server

[1315] software:

[1316] Programming language: Python

[1317] Machine learning library: TensorFlow

[1318] Web framework: Flask

[1319] Database: SQLite

[1320] Notification Service: Twilio API

[1321] System flow

[1322] 1. Memory Support Tools:

[1323] The server periodically generates questions for the elderly, which are generated based on a generative AI model.

[1324] The device (smartphone or smart glasses) displays this question to the elderly person and collects their answer.

[1325] The server stores the received response data and analyzes it to evaluate the cognitive status of the elderly person.

[1326] 2. Scheduling Management Methods:

[1327] The user inputs the schedule into the terminal.

[1328] The server stores this schedule data and uses the Twilio API to send reminder notifications when the specified time approaches.

[1329] 3. Abnormal Operation Detection Method:

[1330] The device uses sensors in smartphones and smart glasses to monitor the movements of elderly people.

[1331] The server analyzes the received operational data and sends a warning message to the specified contacts if an abnormality is detected.

[1332] 4. Behavioral analysis and visualization tools:

[1333] The terminal collects data on the elderly's behavior within the store and sends it to a server.

[1334] The server analyzes the collected data using TensorFlow to evaluate behavioral patterns and health status.

[1335] The analysis results will be visualized as graphs and charts using Flask, making them accessible to the elderly and their families.

[1336] 5. Product information display means:

[1337] When an elderly person approaches a store shelf, the device uses RFID or NFC to obtain product information and displays it on the smart glasses or smartphone.

[1338] 6. Anomaly detection notification method:

[1339] The device detects this abnormality if the elderly person does not move within a certain period of time.

[1340] The server analyzes the abnormal condition and sends a notification using the Twilio API.

[1341] Specific examples

[1342] For example, when an elderly person approaches the bread shelf, the smart glasses will display a message saying, "This product is bread. The price is 300 yen." If the elderly person sets a reminder to "not forget to take today's medicine," a notification will be sent at the scheduled time. If an abnormality is detected, a notification will be sent to store staff saying, "The customer has been stationary for a long time."

[1343] Prompt Sentence Examples

[1344] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[1345] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1346] Step 1:

[1347] server:

[1348] A generative AI model is used to generate specific questions to assess the elderly person's cognition and memory, such as "What did you have for breakfast today?", and the generated questions are sent to the device.

[1349] Input: User data, past response data

[1350] Output: Generated question text

[1351] Step 2:

[1352] Device:

[1353] The generated question is displayed to the user. The device uses a smartphone or smart glasses to visually present the question to the elderly.

[1354] Input: Generated question text

[1355] Output: Screen showing the question

[1356] Step 3:

[1357] User:

[1358] Enter your answer to the question. For example, you might enter, "I had bread and coffee for breakfast today." After you enter the answer, the data is recorded on the device.

[1359] Input: The text of the user's answer to the question

[1360] Output: User's answer text

[1361] Step 4:

[1362] Device:

[1363] The system receives the user's answers and sends them to the server, where the answers are stored in a database for later analysis.

[1364] Input: User's answer text

[1365] Output: Data sent to the server

[1366] Step 5:

[1367] server:

[1368] The received response data is analyzed to assess the user's cognitive function. This includes comparing it with past data and quantifying it to track memory fluctuations. The analysis results are stored in a database.

[1369] Input: User response data, past data

[1370] Output: Analysis result data

[1371] Step 6:

[1372] User:

[1373] Enter your schedule. For example, enter "I'll go to the hospital tomorrow at 10:00 AM." The entered data will be saved on your device.

[1374] Input: Schedule data (date, time, content)

[1375] Output: Schedule registration data to the terminal

[1376] Step 7:

[1377] Device:

[1378] The entered schedule data is sent to the server.

[1379] Input: Schedule data

[1380] Output: Data sent to the server

[1381] Step 8:

[1382] server:

[1383] It saves the received schedule data, generates reminder notifications when the specified time approaches, and sends notifications to the user using the Twilio API.

[1384] Input: Schedule data

[1385] Output: Reminder notification

[1386] Step 9:

[1387] Device:

[1388] The movement of elderly people is monitored using sensors on smartphones and smart glasses, and the collected data is sent to a server.

[1389] Input: Motion data (sensor data)

[1390] Output: Data sent to the server

[1391] Step 10:

[1392] server:

[1393] The received operational data is analyzed, and if an abnormality is detected, a warning message is sent to the specified contacts. Notifications are sent using the Twilio API.

[1394] Input: Operation data

[1395] Output: Warning notice

[1396] Step 11:

[1397] Device:

[1398] When an elderly person approaches a product shelf, product information is obtained using RFID or NFC and displayed on the smart glasses or smartphone.

[1399] Input: Product data (RFID / NFC tag information)

[1400] Output: Product information display

[1401] Step 12:

[1402] User:

[1403] The user checks the product information and purchases it if necessary. The user's behavior data and purchase data are recorded on the device.

[1404] Input: Product information, purchase action

[1405] Output: Purchase data

[1406] Step 13:

[1407] Device:

[1408] Data on the behavior of elderly people in the store is collected and sent to a server.

[1409] Input: Behavioral data (location information, movement history)

[1410] Output: Data sent to the server

[1411] Step 14:

[1412] server:

[1413] The collected behavioral data is analyzed, and the results are visualized to evaluate behavioral patterns and health conditions, generating reports that can be viewed by seniors and their families.

[1414] Input: Behavioral data

[1415] Output: Visualized data of analysis results (graphs, charts)

[1416] Prompt Sentence Examples

[1417] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[1418] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1419] The present invention is a system for the elderly equipped with generative artificial intelligence (generative AI) and an emotion engine, which comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[1420] Cognitive exercise features

[1421] 1. The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.). The questions evaluate the user's memory, such as "What did you have for breakfast today?"

[1422] 2. The user enters an answer to the question displayed on the terminal. For example, the user enters "I ate bread and coffee."

[1423] 3. The device sends the user's response data to the server in real time.

[1424] 4. The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[1425] 5. The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[1426] Schedule reminder function

[1427] 1. The user enters their schedule into the device. For example, they might enter, "I'm going to the hospital tomorrow at 10:00 AM."

[1428] 2. The terminal sends the entered schedule data to the server.

[1429] 3. The server saves the received schedule data and sets a reminder, such as notifying you 15 minutes before the specified time.

[1430] 4. When the set reminder time arrives, the server sends a reminder notification to the device.

[1431] 5. The device displays a reminder notification to the user, for example, a pop-up notification saying, "You have an appointment to go to the doctor in 10 minutes."

[1432] Non-response and abnormality warning function

[1433] 1. The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[1434] 2. The device sends the collected activity data, including sensor data and user input data, to the server.

[1435] 3. The server analyzes the received data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[1436] 4. If the server detects an abnormality, it will send a warning message to pre-defined contacts, for example, "Mom was unresponsive at lunchtime. Please check on her."

[1437] 5. The device will notify the contacts of the received warning message, allowing for a prompt response.

[1438] Behavioral analysis and visualization features

[1439] 1. The device collects daily user behavior data, including walking distance, dietary intake, and sleep duration.

[1440] 2. The device periodically transmits the collected data to the server in real time.

[1441] 3. The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[1442] 4. The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by the user and their family.

[1443] 5. The device notifies the user and their family of the visualized data, for example, by displaying a graph of the change in walking distance each week.

[1444] Emotion Engine Functions

[1445] 1. The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[1446] 2. The device sends emotion analysis data to the server, including voice data and facial expression data.

[1447] 3. The server generates an appropriate response based on the received emotion analysis data. For example, if the user is feeling down, it generates an encouraging message.

[1448] 4. The server sends the generated dialogue content to the terminal.

[1449] 5. The device displays the generated dialogue to the user, continues the conversation, and, if necessary, sends messages of encouragement or confirmation of the user's safety based on the user's emotional state.

[1450] Specific examples

[1451] The user answers the device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, and a reminder notification is displayed on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[1452] Furthermore, the device analyzes the user's emotions and, for example, if the user is feeling down, displays an encouraging message such as, "Are you OK? Is there anything I can help you with?" In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[1453] The processing flow will be explained below.

[1454] Cognitive exercise features

[1455] Step 1:

[1456] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess memory ability.

[1457] Step 2:

[1458] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[1459] Step 3:

[1460] The device sends the user's response data to the server in real time.

[1461] Step 4:

[1462] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[1463] Step 5:

[1464] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[1465] Schedule reminder function

[1466] Step 1:

[1467] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[1468] Step 2:

[1469] The terminal transmits the input schedule data to the server.

[1470] Step 3:

[1471] The server stores the received schedule data and sets a reminder, for example, to notify you 15 minutes before the specified time.

[1472] Step 4:

[1473] When the set reminder time arrives, the server sends a reminder notification to the device.

[1474] Step 5:

[1475] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[1476] Non-response and abnormality warning function

[1477] Step 1:

[1478] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[1479] Step 2:

[1480] The device sends collected activity data, including sensor data and user input data, to a server.

[1481] Step 3:

[1482] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[1483] Step 4:

[1484] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check on her."

[1485] Step 5:

[1486] The device will notify contacts of the received warning message, allowing for a prompt response.

[1487] Behavioral analysis and visualization features

[1488] Step 1:

[1489] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[1490] Step 2:

[1491] The device periodically transmits the collected data to the server in real time.

[1492] Step 3:

[1493] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[1494] Step 4:

[1495] The server visualizes the analysis results as graphs and charts, and the visualized data is displayed on a dashboard or similar.

[1496] Step 5:

[1497] The device then communicates the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[1498] Emotion Engine Functions

[1499] Step 1:

[1500] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[1501] Step 2:

[1502] The device sends emotion analysis data, including voice data and facial expression data, to the server.

[1503] Step 3:

[1504] The server generates appropriate responses based on the received emotion analysis data, for example, an encouraging message if the user is feeling down.

[1505] Step 4:

[1506] The server transmits the generated dialogue content to the terminal.

[1507] Step 5:

[1508] The device displays the generated dialogue content to the user, continues the dialogue appropriately, and, if necessary, sends messages of encouragement or confirmation of safety based on the user's emotional state.

[1509] As a specific example, if a user says, "I'm feeling a little down today," the device analyzes the user's voice and facial expressions to recognize the emotion. The device sends this data to a server, which then generates a dialogue such as, "Are you OK? Is there anything I can help you with?" and sends it back to the device. The device then displays this message to the user and the dialogue continues. In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[1510] Example 2

[1511] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1512] In the lives of elderly people, there is a growing need for early detection of memory decline, difficulty in schedule management, and abnormal behavior. Furthermore, analyzing and visualizing daily behavior is important for elderly people and their families. However, systems that comprehensively and effectively support these issues are still insufficient. Furthermore, there is a need for systems that can grasp the emotional state of elderly people and generate appropriate responses.

[1513] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1514] In this invention, the server includes a means for supporting the elderly's memory in daily life, a means for managing the elderly's schedule, a means for detecting abnormal behavior of the elderly, a means for analyzing and visualizing the elderly's daily behavior, and a means for analyzing the elderly's emotional state and generating appropriate responses. This enables not only memory support and schedule management, but also early detection of abnormal behavior, visualization of behavioral analysis, emotion analysis, and appropriate responses.

[1515] "Generative AI" is AI that generates sentences and dialogues using natural language processing and machine learning.

[1516] The "memory support means" is a function that evaluates and supports the user's memory by displaying specific questions to the user and collecting, recording, and analyzing the user's answers.

[1517] The "schedule management means" is a function that saves the schedule entered by the user and sends a reminder notification at a specified time.

[1518] The "abnormal behavior detection means" is a function that monitors the behavioral data of elderly people and issues an alert if any unusual behavior or reaction is detected.

[1519] The "behavioral analysis means" is a function that evaluates the user's behavioral patterns and health condition by collecting and analyzing the user's daily behavioral data.

[1520] "Visualization means" is a function that visually displays the results of behavioral analysis as graphs or charts.

[1521] The "emotional state analysis means" is a function that analyzes data such as voice and facial expression to determine the user's emotional state.

[1522] The "response generation means" is a function that generates appropriate dialogue content and messages based on the results of the emotional state analysis.

[1523] A "server" is a centralized computer system that stores, analyzes, and processes data.

[1524] A "terminal" is a computer device that is directly operated by a user and that communicates with a server to provide various functions.

[1525] This invention is a system for supporting the daily lives of elderly people, utilizing generative artificial intelligence to support memory, manage schedules, detect abnormal behavior, analyze and visualize behavior, and analyze emotional states. This system mainly consists of a server and a terminal, and monitors user input data and daily activity data in real time, analyzing and responding appropriately.

[1526] Hardware and Software

[1527] A server is a centralized computer system that stores, analyzes, and processes data. It requires powerful processors, memory, and large amounts of storage. The software required includes a database management system (DBMS), machine learning libraries (e.g., TensorFlow, PyTorch), data analysis tools, and visualization tools (e.g., Tableau).

[1528] A terminal is a computing device that is directly operated by a user, and is typically a compact tablet or smartphone. Terminals are equipped with various sensors (e.g., camera, microphone, accelerometer) and use these sensors to collect data. Terminal software includes a user interface, a data collection application, and a communication module.

[1529] Program processing

[1530] At a specific time each day, the device displays a question to the user to assess their memory. For example, "What did you have for breakfast today?" The user responds by answering "I had bread and coffee." The device then sends this response data to the server in real time. The server then converts the received data into a numerical value and stores it in a database. For example, "bread" is converted into a numerical value of 1, and "coffee" into a numerical value of 1. The server then analyzes the state of the user's cognitive function based on the saved data, comparing it with past data, and evaluates progress and changes.

[1531] When a user inputs their schedule into a device, for example, "I'm going to the hospital tomorrow at 10 AM," the device sends this schedule data to the server, which saves the received data and sets a reminder. A reminder is sent 15 minutes before the specified time. When the time for this reminder arrives, the server sends a reminder to the device, and the device displays a pop-up notification saying, "You have an appointment to go to the hospital in 10 minutes."

[1532] Furthermore, the device monitors the user's daily activity patterns. For example, it monitors meal times, walking distance, and response status in real time. The device sends the collected activity data to a server, which analyzes the received data and compares it with standard behavioral patterns to determine whether there are any abnormalities. If an abnormality is detected, for example, if there is no response during lunch, the server determines that this is an abnormality and sends a warning message to the specified contacts saying, "Mom did not respond during lunch. Please check." The device then notifies the contacts of the received warning message.

[1533] The server analyzes the behavioral data using AI algorithms to evaluate the user's behavioral patterns and health status. The analysis results are visualized as graphs and charts and displayed to the user and their family. For example, a graph showing changes in walking distance each week is displayed.

[1534] The device analyzes the user's voice and facial expressions in real time to determine their emotional state. For example, if the user is feeling down, the device sends this analysis data to the server, which then generates an encouraging message. The server then sends the generated dialogue content to the device, which then displays a message such as, "Are you OK? Is there anything I can help you with?"

[1535] Specific examples

[1536] For example, a user might answer to their device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital tomorrow at 10 a.m.", the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If their daily activities are abnormal, the server sends an alert to their family, and ultimately, their behavioral patterns are visualized and the data can be constantly referenced. Furthermore, the device analyzes the user's emotions. For example, if the user is feeling down, it displays an encouraging message such as, "Are you okay? Is there anything I can help you with?"

[1537] Prompt Sentence Examples

[1538] To generate a message of encouragement for a user who is feeling down, an example of a prompt to input to the AI ​​model is as follows:

[1539] "When the user is feeling down, generate an encouraging message. The user's name is Takahashi."

[1540] This allows the generative AI model to generate dialogue such as, "Takahashi-san, are you okay? Is there anything I can help you with?"

[1541] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1542] Cognitive exercise features

[1543] Step 1:

[1544] The device displays a question to assess memory ability every morning at 9:00, such as "What did you have for breakfast today?" The input is a specific time (e.g., 9:00), and the output is the display of the question.

[1545] Step 2:

[1546] The user inputs "I ate bread and coffee" into the terminal. The input is the user's answer, and the output is text data.

[1547] Step 3:

[1548] The terminal transmits the user's answers to the server in real time. The input is the user's answer text data, and the output is the transmitted data. This is done using a communication module.

[1549] Step 4:

[1550] The server digitizes the received text data and stores it in a database. The input is the received text data, and the output is the digitized data and its storage. For example, "bread" is digitized as 1 and "coffee" is digitized as 1.

[1551] Step 5:

[1552] The server analyzes the user's cognitive function status based on the stored data and compares it with past data. The input is the stored data, and the output is the analysis results. Machine learning algorithms are used for these analyses.

[1553] Schedule reminder function

[1554] Step 1:

[1555] The user inputs an appointment into the terminal, such as "I will go to the hospital tomorrow at 10:00 AM." The input is the user's schedule data, and the output is the input data.

[1556] Step 2:

[1557] The terminal transmits the input schedule data to the server. The input is the schedule data, and the output is the transmission data. The data is transmitted via the communication module.

[1558] Step 3:

[1559] The server saves the received data and sets the reminder time. The input is the received data and the output is the reminder setting. For example, set it to notify 15 minutes before the specified time.

[1560] Step 4:

[1561] The server sends a notification to the device when the specified reminder time arrives. The input is the reminder time and the output is the notification data.

[1562] Step 5:

[1563] The device displays a pop-up notification saying, "I have an appointment to go to the hospital in 10 minutes." The input is the notification data, and the output is the pop-up display.

[1564] Non-response and abnormality warning function

[1565] Step 1:

[1566] The device monitors daily user activity data such as meal times, walking distance, and response status. The input is sensor data and user operation data, and the output is collected data.

[1567] Step 2:

[1568] The terminal transmits the collected activity data to the server in real time. The input is the collected data and the output is the transmitted data.

[1569] Step 3:

[1570] The server analyzes the received data and compares it with standard behavioral patterns to detect anomalies. The input is the received data, and the output is the analysis results. For example, if there is no response during lunch time, an anomaly is detected.

[1571] Step 4:

[1572] If an abnormality is detected, the server sends a warning message to pre-defined contacts. The input is the analysis result, and the output is the warning message. For example, "Mom was unresponsive at lunchtime. Please check."

[1573] Step 5:

[1574] The terminal notifies the contact of the received warning message. The input is the warning message and the output is the notification display.

[1575] Behavioral analysis and visualization features

[1576] Step 1:

[1577] The device collects daily user behavior data, including walking distance, dietary habits, sleep duration, etc. The input is sensor data and user operation data, and the output is the collected data.

[1578] Step 2:

[1579] The terminal transmits the collected data to the server in real time. The input is the collected data and the output is the transmitted data.

[1580] Step 3:

[1581] The server uses AI algorithms to analyze the received data and evaluate the user's behavioral patterns and health status. The input is the received data, and the output is the analysis results.

[1582] Step 4:

[1583] The server visualizes the analysis results as graphs and charts and displays them on a dashboard. The input is the analysis results, and the output is the visualized data. For example, a graph showing changes in walking distance each week is displayed.

[1584] Step 5:

[1585] The terminal notifies the user and their family of the visualized data. The input is the visualized data, and the output is a notification display.

[1586] Emotion Engine Functions

[1587] Step 1:

[1588] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state. The input is voice data and facial expression data, and the output is emotion analysis data.

[1589] Step 2:

[1590] The device transmits the emotion analysis data to the server in real time. The input is the emotion analysis data, and the output is the transmitted data.

[1591] Step 3:

[1592] The server generates appropriate dialogue content based on the received data. The input is emotion analysis data, and the output is the generated dialogue content. For example, if the user is feeling down, it generates an encouraging message.

[1593] Step 4:

[1594] The server sends the generated dialogue content to the terminal. The input is the generated dialogue content, and the output is the transmitted data.

[1595] Step 5:

[1596] The terminal displays the generated dialogue to the user and continues the conversation. The input is the transmitted data, and the output is the displayed dialogue. For example, it displays a message such as "Are you OK? Is there anything I can help you with?"

[1597] (Application example 2)

[1598] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1599] There is a need for support for the daily lives of the elderly, as well as cognitive function support, schedule management, abnormal behavior detection, and emotion monitoring for factory workers. Current systems have difficulty providing these functions comprehensively, resulting in increased human resources and management costs. Furthermore, it is difficult to detect abnormal behavior and emotional states of elderly people and workers in real time and respond quickly. For this reason, a comprehensive support system is needed to ensure that elderly people and workers can live their daily lives and work with peace of mind.

[1600] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the memory of elderly people in their daily lives, means for managing the schedule of elderly people, means for detecting abnormal movements of elderly people, means for analyzing and visualizing the daily behavior of elderly people, means for supporting the cognitive function of workers, means for notifying factory workers of schedules and meetings, means for detecting and notifying non-response and abnormal behavior of workers, and means for monitoring the emotional state of workers and providing support. This enables comprehensive support for elderly people and workers.

[1601] "Generative AI" is AI that has the ability to generate new information and solutions based on data.

[1602] "Elderly people" is a broad term referring to users who provide support for the daily lives of the elderly.

[1603] "Memory Support" is a method of presenting questions and tasks to assess, maintain, and improve a user's memory.

[1604] "Schedule management" is a means of saving the schedule entered by the user and notifying them at the appropriate time.

[1605] "Abnormal behavior detection" is a method for monitoring and detecting in real time any behavior that deviates from a user's normal behavior pattern.

[1606] "Behavioral analysis and visualization" is a method of collecting and analyzing users' daily behavioral data and displaying the results in graphs and charts.

[1607] "Workers" is a broad term referring to workers in factories and other places.

[1608] "Cognitive support" is a method of presenting questions and tasks to assess workers' cognitive abilities and to maintain and improve them.

[1609] "Schedule and meeting notifications" are a means of notifying workers of their shifts and scheduled meetings at the appropriate time.

[1610] "Detection of non-responsiveness and abnormal behavior" refers to the means of detecting when a worker does not behave or respond normally or when the worker behaves abnormally.

[1611] "Emotional state monitoring" is a means of monitoring and analyzing workers' emotional states in real time.

[1612] "Providing support" is a means of providing encouragement and appropriate responses that take into account the worker's emotional state.

[1613] System Program

[1614] The system for this application consists of a number of hardware and software components. The system has the following functions:

[1615] 1. Generative AI: Has the ability to generate new information and solutions based on data.

[1616] 2. Memory support for the elderly: Ask questions and collect answers to support the memory of the elderly in their daily lives.

[1617] 3. Managing schedules for seniors: Save schedules entered by seniors and notify them at the appropriate time.

[1618] 4. Detection of abnormal behavior of elderly people: Detects and notifies abnormal behavior of elderly people in real time.

[1619] 5. Analysis and visualization of elderly people's behavior: Collect and analyze data on the daily behavior of elderly people and visualize the results.

[1620] 6. Supporting workers' cognitive function: Presents questions and tasks to assess, maintain, and improve the cognitive function of factory workers.

[1621] 7. Worker Schedule and Meeting Notification: Providing timely notification of factory workers' shift and meeting schedules.

[1622] 8. Detecting unresponsiveness or abnormal behavior of workers: Detecting unresponsiveness or abnormal behavior of factory workers and notifying management.

[1623] 9. Monitoring workers' emotional states: Monitor the emotional states of factory workers in real time and provide encouragement and support.

[1624] Hardware and Software

[1625] 1. Hardware:

[1626] The company will use devices designed for use by the elderly and factory robots that work in collaboration with workers within the factory.

[1627] 2. Software:

[1628] Real-time data processing server equipped with generative AI models

[1629] Analysis software using AI algorithms

[1630] Processing logic written in Python scripts

[1631] Process Overview

[1632] 1. Memory support

[1633] The server generates specific questions and sends them to the terminal, which displays the questions to the user and collects the user's answers and sends them to the server, which records and analyzes the answer data.

[1634] 2. Schedule Management

[1635] The server receives and stores the schedule data entered by the user, and when the designated time approaches, the server sends a corresponding reminder notification to the terminal, which then displays the notification to the user.

[1636] 3. Abnormal behavior detection

[1637] The device uses sensors to monitor the daily activities of elderly people and workers, and if an abnormality is detected, the data is sent to a server, which then sends a warning message to pre-set contacts.

[1638] 4. Behavioral analysis and visualization

[1639] The device collects daily activity data (e.g., walking distance, meal contents, sleep time, etc.) and periodically sends it to a server. The server analyzes the received data and visualizes the results as graphs and charts. This visualized data is displayed on a dashboard that can be viewed by the user and their family.

[1640] 5. Monitoring your emotional state

[1641] The device analyzes the user's voice and facial expressions to determine the user's emotional state in real time. The emotional analysis data is sent to the server, which then generates an appropriate response and sends it to the device. The device then displays the generated dialogue to the user and, if necessary, sends a message of safety confirmation or encouragement based on the user's emotional state.

[1642] Examples and prompts

[1643] 1. When an elderly person answers the device, "I had bread and coffee for breakfast today," the device sends this answer to the server, which then converts the data into numerical values ​​and records them.

[1644] 2. When a factory worker enters "I have a meeting tomorrow at 10:00 AM" into their schedule, the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[1645] Example prompt:

[1646] Ask "What did you have for lunch yesterday?" and get the user's answer. Then send the answer data to the server and store it along with the analysis results.

[1647] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1648] Step 1:

[1649] The device displays a specific question to the user every morning. For example, "What did you have for lunch yesterday?" This question is obtained from the server. The server generates question data using a generative AI model and sends it to the device.

[1650] Step 2:

[1651] The user inputs an answer to the question displayed on the terminal. For example, the user answers "I ate a sandwich." This input data is used as data for memory support for the user.

[1652] Step 3:

[1653] The device sends the user's response data to the server, which then processes the data by converting it into a number or categorizing it. For example, it converts "sandwich" into a specific number or category.

[1654] Step 4:

[1655] The server evaluates the user's cognitive function based on the saved response data, compares it with past responses stored in the database, calculates progress and changes, and generates analysis results using a generative AI model.

[1656] Step 5:

[1657] The server visualizes the analysis results as graphs and charts and sends them to the device, where users and their families can view the results on a dashboard.

[1658] Step 6:

[1659] A user inputs schedule data into a terminal. For example, the user inputs "I have a meeting tomorrow at 10:00 AM." This input data is used as schedule management data.

[1660] Step 7:

[1661] The terminal sends the input schedule data to the server, which receives the data and sets it to send a reminder notification at the specified time.

[1662] Step 8:

[1663] When the specified reminder time approaches, the server generates a reminder notification and sends it to the device. For example, it creates a notification saying "I have a meeting in 10 minutes."

[1664] Step 9:

[1665] The device will display a reminder notification to the user, who can then review and modify the action based on the notification.

[1666] Step 10:

[1667] The device monitors the user's daily activity data in real time and detects abnormal behavior or unresponsiveness, such as when the user is unresponsive after their usual lunch break.

[1668] Step 11:

[1669] If any abnormal behavior is detected, the device will send the data to the server, which will then send a warning message to pre-defined contacts if it determines that something is wrong.

[1670] Step 12:

[1671] The device analyzes the user's voice and facial expressions to determine their emotional state. The emotion analysis data is sent to the server, which then generates an appropriate response. If the device detects that the user is depressed, it generates an encouraging message.

[1672] Step 13:

[1673] The server then sends the generated dialogue to the device, which displays it to the user and continues the conversation as needed. It may also send messages of safety confirmation or encouragement.

[1674] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1675] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1676] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1677] [Fourth embodiment]

[1678] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1679] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1680] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1681] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1682] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1683] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1684] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1685] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1686] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1687] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1688] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1689] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1690] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1691] The present invention is a system for the elderly equipped with productive artificial intelligence (generative AI) that comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[1692] Cognitive exercise features

[1693] 1. At a specific time each day, the device asks the elderly questions to assess their memory and cognitive function, such as "What did you have for breakfast today?"

[1694] 2. The user answers the displayed question by typing, for example, "I ate bread and coffee."

[1695] 3. The device sends the user's answer to the server.

[1696] 4. The server quantifies the received data and records it in a database. This data is used to track and analyze changes in the user's cognitive function.

[1697] Schedule reminder function

[1698] 1. The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM."

[1699] 2. The terminal sends this schedule data to the server.

[1700] 3. The server sets a reminder based on the received schedule data. When the specified time approaches, for example, a reminder notification is set 15 minutes before.

[1701] 4. When the reminder time arrives, the device will display a notification to the user, allowing the user to take the necessary action without forgetting the appointment.

[1702] Non-response and abnormality warning function

[1703] 1. The device monitors the elderly person's daily activity patterns, such as meal times, walking distance, and response status.

[1704] 2. The server analyzes the data collected daily and compares it with standard patterns. If an anomaly is detected, for example, if there is no response during lunchtime, it is deemed to be an anomaly.

[1705] 3. If the server detects an abnormality, it will send a warning message to pre-defined contacts, detailing the abnormality and recommending a course of action.

[1706] Behavioral analysis and visualization features

[1707] 1. The device collects real-time data on the elderly person's daily activities, including walking distance, dietary habits, and sleep duration.

[1708] 2. The device periodically sends the collected data to the server.

[1709] 3. The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status.

[1710] 4. The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance.

[1711] 5. The device provides visualized data to the user and their family through a dashboard and notification function.

[1712] Specific examples

[1713] The user answers the question on the device by saying, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[1714] In this way, the present invention comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1715] The processing flow will be explained below.

[1716] Cognitive exercise features

[1717] Step 1:

[1718] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess the user's memory.

[1719] Step 2:

[1720] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[1721] Step 3:

[1722] The device sends the user's response data to the server in real time.

[1723] Step 4:

[1724] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[1725] Step 5:

[1726] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[1727] Schedule reminder function

[1728] Step 1:

[1729] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[1730] Step 2:

[1731] The terminal transmits the input schedule data to the server.

[1732] Step 3:

[1733] The server stores the received schedule data and sets reminders, such as notifying you 15 minutes before the specified time.

[1734] Step 4:

[1735] When the set reminder time arrives, the server sends a reminder notification to the device.

[1736] Step 5:

[1737] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[1738] Non-response and abnormality warning function

[1739] Step 1:

[1740] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[1741] Step 2:

[1742] The device sends collected activity data, including sensor data and user input data, to a server.

[1743] Step 3:

[1744] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[1745] Step 4:

[1746] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check."

[1747] Step 5:

[1748] The device will notify contacts of the received warning message, allowing for a prompt response.

[1749] Behavioral analysis and visualization features

[1750] Step 1:

[1751] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[1752] Step 2:

[1753] The device periodically transmits the collected data to the server in real time.

[1754] Step 3:

[1755] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[1756] Step 4:

[1757] The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by users and their families.

[1758] Step 5:

[1759] The device will then communicate the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[1760] Through the above steps, this system provides comprehensive support for the lives of elderly people, enabling them to live their daily lives with peace of mind.

[1761] Example 1

[1762] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1763] Maintaining memory, managing schedules, and monitoring health are important in the daily lives of elderly people. However, these tasks are difficult to perform alone and can be a burden on family members and caregivers. Furthermore, a system is needed to respond quickly when elderly people exhibit abnormal behavior in their daily lives. This invention provides a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1764] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1765] In this invention, the server includes means for generating specific questions, collecting and transmitting answers from users to the server, means for transmitting schedule data entered by the user to the server and for the server to set reminders, and means for collecting behavioral data in real time and providing visualized data to the user and their family through an interface, thereby enabling memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization in the daily lives of elderly people.

[1766] "Generative artificial intelligence" is an automated intelligent system that responds and analyzes based on user data input and interactions.

[1767] The "memory support means" is a function that asks specific questions to the user to help the user's cognitive functions.

[1768] The "schedule management means" is a function that manages the schedule entered by the user and notifies the user at a specific time.

[1769] The "abnormal behavior detection means" is a function that monitors the user's daily activities and detects abnormal behavior or patterns.

[1770] "Behavioral analysis" is the process of collecting and analyzing users' daily behavioral data.

[1771] "Visualization" refers to the presentation of analytical results in a visual format such as a graph or chart.

[1772] "Means for generating specific questions" refers to a function that uses a generative AI model to automatically create appropriate questions for the user.

[1773] The "means for collecting user responses" is a function that collects the responses entered by the user as data and transmits them to the server.

[1774] The "means for setting a reminder" is a function for managing a schedule so as to notify the user at a specific time based on the schedule input by the user.

[1775] "Means for collecting behavioral data in real time" refers to a function that uses sensors and devices to detect and record users' daily life behavior in real time.

[1776] "Means for providing visualized data to users and their families through an interface" refers to a function that visually displays collected and analyzed data and provides it to users and their families in the form of a dashboard or notification.

[1777] The present invention is a system equipped with generative artificial intelligence (generative AI) that provides comprehensive support for the daily lives of elderly people. This system provides integrated functions for memory support, schedule management, abnormal behavior detection, and behavior analysis and visualization. Detailed embodiments of each function are described below.

[1778] Hardware Configuration

[1779] The system uses the following hardware components:

[1780] Terminal: A handheld device, such as a tablet or smartphone, that acts as an interface for user input.

[1781] Server: Cloud server or corporate server that analyzes, stores, and manages data.

[1782] Sensor devices: Wearable devices (e.g., smart watches) and home sensors that collect daily activity data of older adults.

[1783] Software Configuration

[1784] Generative AI models (e.g., GPT-4): Used to generate specific questions, providing users with the ability to ask appropriate questions in natural language.

[1785] Database management system: Stores and manages user response data and behavioral data.

[1786] Analysis algorithm: An AI algorithm that analyzes collected data and evaluates the user's health status and behavioral patterns.

[1787] Notification system: A system for notifying users or stakeholders of reminders and anomaly detection messages.

[1788] Memory support function

[1789] The device generates questions to assess memory and cognitive function at specific times each day. For example, a generative AI model is used to create a question such as "What did you have for breakfast today?" The user responds by typing "I had bread and coffee." The device sends this response to a server, which quantifies the data and records it in a database. This allows changes in cognitive function to be tracked and analyzed.

[1790] Schedule management function

[1791] The user inputs their schedule into the device. For example, they input an appointment to "go to the hospital tomorrow at 10:00 AM." The device sends this schedule data to the server, and the server sets a reminder for the specified time. When the specified time approaches, the device displays a notification to the user. This allows the user to remember their appointment.

[1792] Abnormal behavior detection function

[1793] The terminal monitors the elderly person's daily activity patterns through sensor devices, including meal times, walking distance, and response status. The server analyzes the collected data and compares it with standard patterns to detect abnormalities. For example, a lack of response during lunchtime is deemed abnormal. If an abnormality is detected, the server sends a warning message to family members or caregivers.

[1794] Behavioral analysis and visualization features

[1795] The device collects daily behavioral data of the elderly in real time and periodically transmits it to a server. The server then analyzes the received data using AI algorithms to evaluate behavioral patterns and health conditions. The analysis results are visualized as graphs and charts and provided to the user and their family via the device's dashboard.

[1796] Examples and prompts

[1797] Examples:

[1798] The user answers the question displayed on the device by saying, "I had bread and coffee for breakfast today." The device then sends this answer to the server, which then digitizes the data and records it.

[1799] When a user enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[1800] If daily activity is abnormal, the server will send an alert to the family.

[1801] Ultimately, behavioral patterns are visualized and the data is always available for reference.

[1802] Example prompt sentence:

[1803] "What did you have for breakfast today?"

[1804] "Please enter an appointment to go to the hospital tomorrow at 10 AM."

[1805] Check your walking distance.

[1806] In this way, the present invention realizes a system that comprehensively supports the daily lives of elderly people and provides an environment in which they can live with peace of mind.

[1807] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1808] Cognitive exercise features

[1809] Step 1:

[1810] At a specific time each day, the device uses a generative AI model (e.g., GPT-4) to generate questions for the user to assess their memory and cognitive function. For example, a question might be generated: "What did you have for breakfast today?" Input: Current time, generative AI model. Output: Specific question.

[1811] Step 2:

[1812] The terminal displays the generated question to the user. The user inputs the answer using a keyboard or voice input. For example, the user answers "I ate bread and coffee." Input: The generated question. Output: The user's answer data.

[1813] Step 3:

[1814] The terminal collects the user's response data and sends it to the server using a secure protocol (e.g. HTTPS). Input: User's response data. Output: Data sent to the server.

[1815] Step 4:

[1816] The server analyzes the received response data and organizes it as numerical data. It analyzes specific keywords (e.g., bread, coffee) and tags them. Input: User response data. Output: Tagged numerical data.

[1817] Step 5:

[1818] The server records the analyzed and quantified data in a database, which is later used to track changes in the user's cognitive function. Input: Analyzed and quantified data. Output: Data recorded in the database.

[1819] Schedule reminder function

[1820] Step 1:

[1821] The user inputs their schedule into the terminal. For example, they input a schedule such as "I'm going to the hospital tomorrow at 10:00 AM." Input: User's schedule data. Output: Input schedule data.

[1822] Step 2:

[1823] The terminal sends this schedule data to the server. Input: User's schedule data. Output: Schedule data sent to the server.

[1824] Step 3:

[1825] The server sets a reminder based on the received schedule data. The reminder is set to notify 15 minutes before the scheduled time. Input: Schedule data. Output: Set reminder.

[1826] Step 4:

[1827] When the reminder time approaches, the device will remind the user with a pop-up notification or a voice notification. For example, it may notify the user that it is almost time to go to the hospital. Input: The set reminder. Output: The reminder notification.

[1828] Non-response and abnormality warning function

[1829] Step 1:

[1830] The device uses sensors and user input to monitor the elderly person's daily activity patterns, including meal times, walking distance, and responsiveness. Input: Activity data from sensors and user input. Output: Collected activity data.

[1831] Step 2:

[1832] The server analyzes the activity data collected periodically and compares it with standard patterns to detect anomalies. For example, if there is no response during lunch time, it will be judged as an anomaly. Input: Collected activity data. Output: Detected anomalous data.

[1833] Step 3:

[1834] When an abnormality is detected, the server sends a warning message to pre-defined contacts. This message describes the abnormality and the required action. Input: Detected abnormal data. Output: Warning message.

[1835] Behavioral analysis and visualization features

[1836] Step 1:

[1837] The device collects daily behavioral data of elderly people in real time, including walking distance, dietary habits, and sleep duration. Input: Behavioral data collected in real time. Output: Collected behavioral data.

[1838] Step 2:

[1839] The device periodically sends the collected behavioral data to the server. Input: Collected behavioral data. Output: Data sent to the server.

[1840] Step 3:

[1841] The server analyzes the received data using AI algorithms to evaluate behavioral patterns and health status. Input: Data sent to the server. Output: Analysis results.

[1842] Step 4:

[1843] The server visualizes the analysis results as graphs and charts, allowing users and their families to understand their health status and behavioral patterns at a glance. Input: Analysis results. Output: Visualized data (graphs and charts).

[1844] Step 5:

[1845] The device provides visualized data in the form of a dashboard to the user and their family. Periodic reports are also delivered via the notification function. Input: Visualized data. Output: Reports via dashboards and notifications.

[1846] (Application example 1)

[1847] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1848] Current support systems for the elderly are limited to use at home and lack support when out and about, especially in physical stores. This can lead to elderly people getting lost in stores or forgetting important shopping lists. Furthermore, there is also the problem of not being able to respond immediately when abnormal behavior occurs in stores. These issues need to be resolved.

[1849] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1850] In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the elderly's daily memory, means for managing the elderly's schedule, means for detecting abnormal behavior of the elderly, means for analyzing and visualizing the elderly's daily behavior, means for displaying product information and prices when the elderly approaches a product shelf, means for detecting and notifying an abnormality when the elderly does not move within a certain period of time in the store, and means for collecting, analyzing, and visualizing data on the elderly's behavior in the store. This enables the elderly to shop safely and efficiently in physical stores, and allows for quick response even if abnormal behavior occurs.

[1851] "Generative AI" refers to AI that has the ability to automatically generate new information and judgments using data.

[1852] "Memory support means" refers to a device or system that provides a function to support the memory of elderly people by displaying specific questions and collecting and analyzing the user's answers to those questions.

[1853] "Schedule management means" refers to a device or system that saves the schedule entered by the user and notifies the user at the specified time.

[1854] "Abnormal behavior detection means" refers to a device or system that has the function of monitoring the behavior of an elderly person and detecting abnormal behavior that deviates from normal patterns.

[1855] The term "behavioral analysis means" refers to a device or system that has the function of collecting and analyzing daily behavioral data of elderly people.

[1856] "Visualization means" refers to a device or system that has the function of displaying analyzed behavioral data in a visual format such as a graph or chart, and providing it in a form that is easy for users to understand.

[1857] "Product information display means" refers to a device or system that has the function of displaying product information and prices when an elderly person approaches a product shelf.

[1858] "Abnormality detection and notification means" refers to a device or system that has the function of detecting an abnormality and issuing a notification if an elderly person does not move within a certain period of time inside the store.

[1859] "Behavioral data collection means" refers to a device or system that has the function of continuously collecting data on the behavior of elderly people in a store.

[1860] The term "analysis result visualization means" refers to a device or system that has the function of analyzing collected behavioral data and visually expressing the results to provide to the user.

[1861] The present invention aims to provide support for elderly people in brick-and-mortar stores, particularly by providing a system that maintains cognitive abilities, monitors health status, and assists in efficient shopping. To achieve this, the following hardware and software are used:

[1862] Hardware and Software

[1863] Hardware:

[1864] Smartphone

[1865] Smart Glasses

[1866] server

[1867] software:

[1868] Programming language: Python

[1869] Machine learning library: TensorFlow

[1870] Web framework: Flask

[1871] Database: SQLite

[1872] Notification Service: Twilio API

[1873] System flow

[1874] 1. Memory Support Tools:

[1875] The server periodically generates questions for the elderly, which are generated based on a generative AI model.

[1876] The device (smartphone or smart glasses) displays this question to the elderly person and collects their answer.

[1877] The server stores the received response data and analyzes it to evaluate the cognitive status of the elderly person.

[1878] 2. Scheduling Management Methods:

[1879] The user inputs the schedule into the terminal.

[1880] The server stores this schedule data and uses the Twilio API to send reminder notifications when the specified time approaches.

[1881] 3. Abnormal Operation Detection Method:

[1882] The device uses sensors in smartphones and smart glasses to monitor the movements of elderly people.

[1883] The server analyzes the received operational data and sends a warning message to the specified contacts if an abnormality is detected.

[1884] 4. Behavioral analysis and visualization tools:

[1885] The terminal collects data on the elderly's behavior within the store and sends it to a server.

[1886] The server analyzes the collected data using TensorFlow to evaluate behavioral patterns and health status.

[1887] The analysis results will be visualized as graphs and charts using Flask, making them accessible to the elderly and their families.

[1888] 5. Product information display means:

[1889] When an elderly person approaches a store shelf, the device uses RFID or NFC to obtain product information and displays it on the smart glasses or smartphone.

[1890] 6. Anomaly detection notification method:

[1891] The device detects this abnormality if the elderly person does not move within a certain period of time.

[1892] The server analyzes the abnormal condition and sends a notification using the Twilio API.

[1893] Specific examples

[1894] For example, when an elderly person approaches the bread shelf, the smart glasses will display a message saying, "This product is bread. The price is 300 yen." If the elderly person sets a reminder to "not forget to take today's medicine," a notification will be sent at the scheduled time. If an abnormality is detected, a notification will be sent to store staff saying, "The customer has been stationary for a long time."

[1895] Prompt Sentence Examples

[1896] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[1897] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1898] Step 1:

[1899] server:

[1900] A generative AI model is used to generate specific questions to assess the elderly person's cognition and memory, such as "What did you have for breakfast today?", and the generated questions are sent to the device.

[1901] Input: User data, past response data

[1902] Output: Generated question text

[1903] Step 2:

[1904] Device:

[1905] The generated question is displayed to the user. The device uses a smartphone or smart glasses to visually present the question to the elderly.

[1906] Input: Generated question text

[1907] Output: Screen showing the question

[1908] Step 3:

[1909] User:

[1910] Enter your answer to the question. For example, you might enter, "I had bread and coffee for breakfast today." After you enter the answer, the data is recorded on the device.

[1911] Input: The text of the user's answer to the question

[1912] Output: User's answer text

[1913] Step 4:

[1914] Device:

[1915] The system receives the user's answers and sends them to the server, where the answers are stored in a database for later analysis.

[1916] Input: User's answer text

[1917] Output: Data sent to the server

[1918] Step 5:

[1919] server:

[1920] The received response data is analyzed to assess the user's cognitive function. This includes comparing it with past data and quantifying it to track memory fluctuations. The analysis results are stored in a database.

[1921] Input: User response data, past data

[1922] Output: Analysis result data

[1923] Step 6:

[1924] User:

[1925] Enter your schedule. For example, enter "I'll go to the hospital tomorrow at 10:00 AM." The entered data will be saved on your device.

[1926] Input: Schedule data (date, time, content)

[1927] Output: Schedule registration data to the terminal

[1928] Step 7:

[1929] Device:

[1930] The entered schedule data is sent to the server.

[1931] Input: Schedule data

[1932] Output: Data sent to the server

[1933] Step 8:

[1934] server:

[1935] It saves the received schedule data, generates reminder notifications when the specified time approaches, and sends notifications to the user using the Twilio API.

[1936] Input: Schedule data

[1937] Output: Reminder notification

[1938] Step 9:

[1939] Device:

[1940] The movement of elderly people is monitored using sensors on smartphones and smart glasses, and the collected data is sent to a server.

[1941] Input: Motion data (sensor data)

[1942] Output: Data sent to the server

[1943] Step 10:

[1944] server:

[1945] The received operational data is analyzed, and if an abnormality is detected, a warning message is sent to the specified contacts. Notifications are sent using the Twilio API.

[1946] Input: Operation data

[1947] Output: Warning notice

[1948] Step 11:

[1949] Device:

[1950] When an elderly person approaches a product shelf, product information is obtained using RFID or NFC and displayed on the smart glasses or smartphone.

[1951] Input: Product data (RFID / NFC tag information)

[1952] Output: Product information display

[1953] Step 12:

[1954] User:

[1955] The user checks the product information and purchases it if necessary. The user's behavior data and purchase data are recorded on the device.

[1956] Input: Product information, purchase action

[1957] Output: Purchase data

[1958] Step 13:

[1959] Device:

[1960] Data on the behavior of elderly people in the store is collected and sent to a server.

[1961] Input: Behavioral data (location information, movement history)

[1962] Output: Data sent to the server

[1963] Step 14:

[1964] server:

[1965] The collected behavioral data is analyzed, and the results are visualized to evaluate behavioral patterns and health conditions, generating reports that can be viewed by seniors and their families.

[1966] Input: Behavioral data

[1967] Output: Visualized data of analysis results (graphs, charts)

[1968] Prompt Sentence Examples

[1969] "Write code for a shopping assistant app for seniors that displays product information and prices when approaching a bread shelf."

[1970] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1971] The present invention is a system for the elderly equipped with generative artificial intelligence (generative AI) and an emotion engine, which comprehensively supports the daily lives of the elderly. Specific embodiments for this purpose are described below.

[1972] Cognitive exercise features

[1973] 1. The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.). The questions evaluate the user's memory, such as "What did you have for breakfast today?"

[1974] 2. The user enters an answer to the question displayed on the terminal. For example, the user enters "I ate bread and coffee."

[1975] 3. The device sends the user's response data to the server in real time.

[1976] 4. The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[1977] 5. The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[1978] Schedule reminder function

[1979] 1. The user enters their schedule into the device. For example, they might enter, "I'm going to the hospital tomorrow at 10:00 AM."

[1980] 2. The terminal sends the entered schedule data to the server.

[1981] 3. The server saves the received schedule data and sets a reminder, such as notifying you 15 minutes before the specified time.

[1982] 4. When the set reminder time arrives, the server sends a reminder notification to the device.

[1983] 5. The device displays a reminder notification to the user, for example, a pop-up notification saying, "You have an appointment to go to the doctor in 10 minutes."

[1984] Non-response and abnormality warning function

[1985] 1. The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[1986] 2. The device sends the collected activity data, including sensor data and user input data, to the server.

[1987] 3. The server analyzes the received data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch time, it is deemed to be an anomaly.

[1988] 4. If the server detects an abnormality, it will send a warning message to pre-defined contacts, for example, "Mom was unresponsive at lunchtime. Please check on her."

[1989] 5. The device will notify the contacts of the received warning message, allowing for a prompt response.

[1990] Behavioral analysis and visualization features

[1991] 1. The device collects daily user behavior data, including walking distance, dietary intake, and sleep duration.

[1992] 2. The device periodically transmits the collected data to the server in real time.

[1993] 3. The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[1994] 4. The server visualizes the analysis results as graphs and charts, and displays them on a dashboard that can be viewed by the user and their family.

[1995] 5. The device notifies the user and their family of the visualized data, for example, by displaying a graph of the change in walking distance each week.

[1996] Emotion Engine Functions

[1997] 1. The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[1998] 2. The device sends emotion analysis data to the server, including voice data and facial expression data.

[1999] 3. The server generates an appropriate response based on the received emotion analysis data. For example, if the user is feeling down, it generates an encouraging message.

[2000] 4. The server sends the generated dialogue content to the terminal.

[2001] 5. The device displays the generated dialogue to the user, continues the conversation, and, if necessary, sends messages of encouragement or confirmation of the user's safety based on the user's emotional state.

[2002] Specific examples

[2003] The user answers the device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital at 10 a.m. tomorrow," the device sends a reminder to the server, and a reminder notification is displayed on the device at the specified time. If daily activity is abnormal, the server sends an alert to family members, and ultimately, behavioral patterns are visualized and data can be constantly referenced.

[2004] Furthermore, the device analyzes the user's emotions and, for example, if the user is feeling down, displays an encouraging message such as, "Are you OK? Is there anything I can help you with?" In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[2005] The processing flow will be explained below.

[2006] Cognitive exercise features

[2007] Step 1:

[2008] The device displays questions to the user at a specific time every day (e.g., 9:00 a.m.), such as "What did you have for breakfast today?", to assess memory ability.

[2009] Step 2:

[2010] The user inputs an answer to the question displayed on the terminal. For example, the user inputs "I ate bread and coffee."

[2011] Step 3:

[2012] The device sends the user's response data to the server in real time.

[2013] Step 4:

[2014] The server converts the received data into a number and stores it in a database. For example, "bread" is converted into a number with 1, and "coffee" is converted into a number with 1.

[2015] Step 5:

[2016] The server analyzes the user's cognitive function status based on the stored data, compares it with past data, and evaluates progress and changes.

[2017] Schedule reminder function

[2018] Step 1:

[2019] The user inputs their schedule into the terminal, for example, "I'm going to the hospital tomorrow at 10:00 AM."

[2020] Step 2:

[2021] The terminal transmits the input schedule data to the server.

[2022] Step 3:

[2023] The server stores the received schedule data and sets a reminder, for example, to notify you 15 minutes before the specified time.

[2024] Step 4:

[2025] When the set reminder time arrives, the server sends a reminder notification to the device.

[2026] Step 5:

[2027] The device will display a reminder notification to the user, for example, a pop-up notification saying "You have a doctor's appointment in 10 minutes."

[2028] Non-response and abnormality warning function

[2029] Step 1:

[2030] The device monitors the user's daily activity patterns, such as meal times, walking distance, and response status, in real time.

[2031] Step 2:

[2032] The device sends collected activity data, including sensor data and user input data, to a server.

[2033] Step 3:

[2034] The server analyzes the data and compares it with standard behavioral patterns. If an anomaly is detected, for example, if there is no response during lunch, it is deemed to be an anomaly.

[2035] Step 4:

[2036] If the server detects an anomaly, it will send a warning message to pre-defined contacts, for example, "Mom wasn't responding at lunchtime. Please check on her."

[2037] Step 5:

[2038] The device will notify contacts of the received warning message, allowing for a prompt response.

[2039] Behavioral analysis and visualization features

[2040] Step 1:

[2041] The device collects data on the user's daily activities, including the distance walked, what they ate, and how much sleep they had.

[2042] Step 2:

[2043] The device periodically transmits the collected data to the server in real time.

[2044] Step 3:

[2045] The server analyzes the received data using AI algorithms to evaluate the user's behavioral patterns and health status.

[2046] Step 4:

[2047] The server visualizes the analysis results as graphs and charts, and the visualized data is displayed on a dashboard or similar.

[2048] Step 5:

[2049] The device then communicates the visualized data to the user and their family, for example displaying a graph of weekly changes in walking distance.

[2050] Emotion Engine Functions

[2051] Step 1:

[2052] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state.

[2053] Step 2:

[2054] The device sends emotion analysis data, including voice data and facial expression data, to the server.

[2055] Step 3:

[2056] The server generates appropriate responses based on the received emotion analysis data, for example, an encouraging message if the user is feeling down.

[2057] Step 4:

[2058] The server transmits the generated dialogue content to the terminal.

[2059] Step 5:

[2060] The device displays the generated dialogue content to the user, continues the dialogue appropriately, and, if necessary, sends messages of encouragement or confirmation of safety based on the user's emotional state.

[2061] As a specific example, if a user says, "I'm feeling a little down today," the device analyzes the user's voice and facial expressions to recognize the emotion. The device sends this data to a server, which then generates a dialogue such as, "Are you OK? Is there anything I can help you with?" and sends it back to the device. The device then displays this message to the user and the dialogue continues. In this way, the present invention comprehensively supports the daily lives of the elderly and provides an environment in which they can live with peace of mind.

[2062] Example 2

[2063] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2064] In the lives of elderly people, there is a growing need for early detection of memory decline, difficulty in schedule management, and abnormal behavior. Furthermore, analyzing and visualizing daily behavior is important for elderly people and their families. However, systems that comprehensively and effectively support these issues are still insufficient. Furthermore, there is a need for systems that can grasp the emotional state of elderly people and generate appropriate responses.

[2065] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2066] In this invention, the server includes a means for supporting the elderly's memory in daily life, a means for managing the elderly's schedule, a means for detecting abnormal behavior of the elderly, a means for analyzing and visualizing the elderly's daily behavior, and a means for analyzing the elderly's emotional state and generating appropriate responses. This enables not only memory support and schedule management, but also early detection of abnormal behavior, visualization of behavioral analysis, emotion analysis, and appropriate responses.

[2067] "Generative AI" is AI that generates sentences and dialogues using natural language processing and machine learning.

[2068] The "memory support means" is a function that evaluates and supports the user's memory by displaying specific questions to the user and collecting, recording, and analyzing the user's answers.

[2069] The "schedule management means" is a function that saves the schedule entered by the user and sends a reminder notification at a specified time.

[2070] The "abnormal behavior detection means" is a function that monitors the behavioral data of elderly people and issues an alert if any unusual behavior or reaction is detected.

[2071] The "behavioral analysis means" is a function that evaluates the user's behavioral patterns and health condition by collecting and analyzing the user's daily behavioral data.

[2072] "Visualization means" is a function that visually displays the results of behavioral analysis as graphs or charts.

[2073] The "emotional state analysis means" is a function that analyzes data such as voice and facial expression to determine the user's emotional state.

[2074] The "response generation means" is a function that generates appropriate dialogue content and messages based on the results of the emotional state analysis.

[2075] A "server" is a centralized computer system that stores, analyzes, and processes data.

[2076] A "terminal" is a computer device that is directly operated by a user and that communicates with a server to provide various functions.

[2077] This invention is a system for supporting the daily lives of elderly people, utilizing generative artificial intelligence to support memory, manage schedules, detect abnormal behavior, analyze and visualize behavior, and analyze emotional states. This system mainly consists of a server and a terminal, and monitors user input data and daily activity data in real time, analyzing and responding appropriately.

[2078] Hardware and Software

[2079] A server is a centralized computer system that stores, analyzes, and processes data. It requires powerful processors, memory, and large amounts of storage. The software required includes a database management system (DBMS), machine learning libraries (e.g., TensorFlow, PyTorch), data analysis tools, and visualization tools (e.g., Tableau).

[2080] A terminal is a computing device that is directly operated by a user, and is typically a compact tablet or smartphone. Terminals are equipped with various sensors (e.g., camera, microphone, accelerometer) and use these sensors to collect data. Terminal software includes a user interface, a data collection application, and a communication module.

[2081] Program processing

[2082] At a specific time each day, the device displays a question to the user to assess their memory. For example, "What did you have for breakfast today?" The user responds by answering "I had bread and coffee." The device then sends this response data to the server in real time. The server then converts the received data into a numerical value and stores it in a database. For example, "bread" is converted into a numerical value of 1, and "coffee" into a numerical value of 1. The server then analyzes the state of the user's cognitive function based on the saved data, comparing it with past data, and evaluates progress and changes.

[2083] When a user inputs their schedule into a device, for example, "I'm going to the hospital tomorrow at 10 AM," the device sends this schedule data to the server, which saves the received data and sets a reminder. A reminder is sent 15 minutes before the specified time. When the time for this reminder arrives, the server sends a reminder to the device, and the device displays a pop-up notification saying, "You have an appointment to go to the hospital in 10 minutes."

[2084] Furthermore, the device monitors the user's daily activity patterns. For example, it monitors meal times, walking distance, and response status in real time. The device sends the collected activity data to a server, which analyzes the received data and compares it with standard behavioral patterns to determine whether there are any abnormalities. If an abnormality is detected, for example, if there is no response during lunch, the server determines that this is an abnormality and sends a warning message to the specified contacts saying, "Mom did not respond during lunch. Please check." The device then notifies the contacts of the received warning message.

[2085] The server analyzes the behavioral data using AI algorithms to evaluate the user's behavioral patterns and health status. The analysis results are visualized as graphs and charts and displayed to the user and their family. For example, a graph showing changes in walking distance each week is displayed.

[2086] The device analyzes the user's voice and facial expressions in real time to determine their emotional state. For example, if the user is feeling down, the device sends this analysis data to the server, which then generates an encouraging message. The server then sends the generated dialogue content to the device, which then displays a message such as, "Are you OK? Is there anything I can help you with?"

[2087] Specific examples

[2088] For example, a user might answer to their device, "I had bread and coffee for breakfast today." The device sends this answer to the server, which then quantifies and records the data. If the user also enters a schedule such as "I'll go to the hospital tomorrow at 10 a.m.", the device sends a reminder to the server, which displays a reminder notification on the device at the specified time. If their daily activities are abnormal, the server sends an alert to their family, and ultimately, their behavioral patterns are visualized and the data can be constantly referenced. Furthermore, the device analyzes the user's emotions. For example, if the user is feeling down, it displays an encouraging message such as, "Are you okay? Is there anything I can help you with?"

[2089] Prompt Sentence Examples

[2090] To generate a message of encouragement for a user who is feeling down, an example of a prompt to input to the AI ​​model is as follows:

[2091] "When the user is feeling down, generate an encouraging message. The user's name is Takahashi."

[2092] This allows the generative AI model to generate dialogue such as, "Takahashi-san, are you okay? Is there anything I can help you with?"

[2093] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2094] Cognitive exercise features

[2095] Step 1:

[2096] The device displays a question to assess memory ability every morning at 9:00, such as "What did you have for breakfast today?" The input is a specific time (e.g., 9:00), and the output is the display of the question.

[2097] Step 2:

[2098] The user inputs "I ate bread and coffee" into the terminal. The input is the user's answer, and the output is text data.

[2099] Step 3:

[2100] The terminal transmits the user's answers to the server in real time. The input is the user's answer text data, and the output is the transmitted data. This is done using a communication module.

[2101] Step 4:

[2102] The server digitizes the received text data and stores it in a database. The input is the received text data, and the output is the digitized data and its storage. For example, "bread" is digitized as 1 and "coffee" is digitized as 1.

[2103] Step 5:

[2104] The server analyzes the user's cognitive function status based on the stored data and compares it with past data. The input is the stored data, and the output is the analysis results. Machine learning algorithms are used for these analyses.

[2105] Schedule reminder function

[2106] Step 1:

[2107] The user inputs an appointment into the terminal, such as "I will go to the hospital tomorrow at 10:00 AM." The input is the user's schedule data, and the output is the input data.

[2108] Step 2:

[2109] The terminal transmits the input schedule data to the server. The input is the schedule data, and the output is the transmission data. The data is transmitted via the communication module.

[2110] Step 3:

[2111] The server saves the received data and sets the reminder time. The input is the received data and the output is the reminder setting. For example, set it to notify 15 minutes before the specified time.

[2112] Step 4:

[2113] The server sends a notification to the device when the specified reminder time arrives. The input is the reminder time and the output is the notification data.

[2114] Step 5:

[2115] The device displays a pop-up notification saying, "I have an appointment to go to the hospital in 10 minutes." The input is the notification data, and the output is the pop-up display.

[2116] Non-response and abnormality warning function

[2117] Step 1:

[2118] The device monitors daily user activity data such as meal times, walking distance, and response status. The input is sensor data and user operation data, and the output is collected data.

[2119] Step 2:

[2120] The terminal transmits the collected activity data to the server in real time. The input is the collected data and the output is the transmitted data.

[2121] Step 3:

[2122] The server analyzes the received data and compares it with standard behavioral patterns to detect anomalies. The input is the received data, and the output is the analysis results. For example, if there is no response during lunch time, an anomaly is detected.

[2123] Step 4:

[2124] If an abnormality is detected, the server sends a warning message to pre-defined contacts. The input is the analysis result, and the output is the warning message. For example, "Mom was unresponsive at lunchtime. Please check."

[2125] Step 5:

[2126] The terminal notifies the contact of the received warning message. The input is the warning message and the output is the notification display.

[2127] Behavioral analysis and visualization features

[2128] Step 1:

[2129] The device collects daily user behavior data, including walking distance, dietary habits, sleep duration, etc. The input is sensor data and user operation data, and the output is the collected data.

[2130] Step 2:

[2131] The terminal transmits the collected data to the server in real time. The input is the collected data and the output is the transmitted data.

[2132] Step 3:

[2133] The server uses AI algorithms to analyze the received data and evaluate the user's behavioral patterns and health status. The input is the received data, and the output is the analysis results.

[2134] Step 4:

[2135] The server visualizes the analysis results as graphs and charts and displays them on a dashboard. The input is the analysis results, and the output is the visualized data. For example, a graph showing changes in walking distance each week is displayed.

[2136] Step 5:

[2137] The terminal notifies the user and their family of the visualized data. The input is the visualized data, and the output is a notification display.

[2138] Emotion Engine Functions

[2139] Step 1:

[2140] The device analyzes the user's voice and facial expressions in real time to determine the user's emotional state. The input is voice data and facial expression data, and the output is emotion analysis data.

[2141] Step 2:

[2142] The device transmits the emotion analysis data to the server in real time. The input is the emotion analysis data, and the output is the transmitted data.

[2143] Step 3:

[2144] The server generates appropriate dialogue content based on the received data. The input is emotion analysis data, and the output is the generated dialogue content. For example, if the user is feeling down, it generates an encouraging message.

[2145] Step 4:

[2146] The server sends the generated dialogue content to the terminal. The input is the generated dialogue content, and the output is the transmitted data.

[2147] Step 5:

[2148] The terminal displays the generated dialogue to the user and continues the conversation. The input is the transmitted data, and the output is the displayed dialogue. For example, it displays a message such as "Are you OK? Is there anything I can help you with?"

[2149] (Application example 2)

[2150] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2151] There is a need for support for the daily lives of the elderly, as well as cognitive function support, schedule management, abnormal behavior detection, and emotion monitoring for factory workers. Current systems have difficulty providing these functions comprehensively, resulting in increased human resources and management costs. Furthermore, it is difficult to detect abnormal behavior and emotional states of elderly people and workers in real time and respond quickly. For this reason, a comprehensive support system is needed to ensure that elderly people and workers can live their daily lives and work with peace of mind.

[2152] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is equipped with generative artificial intelligence and includes means for supporting the memory of elderly people in their daily lives, means for managing the schedule of elderly people, means for detecting abnormal movements of elderly people, means for analyzing and visualizing the daily behavior of elderly people, means for supporting the cognitive function of workers, means for notifying factory workers of schedules and meetings, means for detecting and notifying non-response and abnormal behavior of workers, and means for monitoring the emotional state of workers and providing support. This enables comprehensive support for elderly people and workers.

[2153] "Generative AI" is AI that has the ability to generate new information and solutions based on data.

[2154] "Elderly people" is a broad term referring to users who provide support for the daily lives of the elderly.

[2155] "Memory Support" is a method of presenting questions and tasks to assess, maintain, and improve a user's memory.

[2156] "Schedule management" is a means of saving the schedule entered by the user and notifying them at the appropriate time.

[2157] "Abnormal behavior detection" is a method for monitoring and detecting in real time any behavior that deviates from a user's normal behavior pattern.

[2158] "Behavioral analysis and visualization" is a method of collecting and analyzing users' daily behavioral data and displaying the results in graphs and charts.

[2159] "Workers" is a broad term referring to workers in factories and other places.

[2160] "Cognitive support" is a method of presenting questions and tasks to assess workers' cognitive abilities and to maintain and improve them.

[2161] "Schedule and meeting notifications" are a means of notifying workers of their shifts and scheduled meetings at the appropriate time.

[2162] "Detection of non-responsiveness and abnormal behavior" refers to the means of detecting when a worker does not behave or respond normally or when the worker behaves abnormally.

[2163] "Emotional state monitoring" is a means of monitoring and analyzing workers' emotional states in real time.

[2164] "Providing support" is a means of providing encouragement and appropriate responses that take into account the worker's emotional state.

[2165] System Program

[2166] The system for this application consists of a number of hardware and software components. The system has the following functions:

[2167] 1. Generative AI: Has the ability to generate new information and solutions based on data.

[2168] 2. Memory support for the elderly: Ask questions and collect answers to support the memory of the elderly in their daily lives.

[2169] 3. Managing schedules for seniors: Save schedules entered by seniors and notify them at the appropriate time.

[2170] 4. Detection of abnormal behavior of elderly people: Detects and notifies abnormal behavior of elderly people in real time.

[2171] 5. Analysis and visualization of elderly people's behavior: Collect and analyze data on the daily behavior of elderly people and visualize the results.

[2172] 6. Supporting workers' cognitive function: Presents questions and tasks to assess, maintain, and improve the cognitive function of factory workers.

[2173] 7. Worker Schedule and Meeting Notification: Providing timely notification of factory workers' shift and meeting schedules.

[2174] 8. Detecting unresponsiveness or abnormal behavior of workers: Detecting unresponsiveness or abnormal behavior of factory workers and notifying management.

[2175] 9. Monitoring workers' emotional states: Monitor the emotional states of factory workers in real time and provide encouragement and support.

[2176] Hardware and Software

[2177] 1. Hardware:

[2178] The company will use devices designed for use by the elderly and factory robots that work in collaboration with workers within the factory.

[2179] 2. Software:

[2180] Real-time data processing server equipped with generative AI models

[2181] Analysis software using AI algorithms

[2182] Processing logic written in Python scripts

[2183] Process Overview

[2184] 1. Memory support

[2185] The server generates specific questions and sends them to the terminal, which displays the questions to the user and collects the user's answers and sends them to the server, which records and analyzes the answer data.

[2186] 2. Schedule Management

[2187] The server receives and stores the schedule data entered by the user, and when the designated time approaches, the server sends a corresponding reminder notification to the terminal, which then displays the notification to the user.

[2188] 3. Abnormal behavior detection

[2189] The device uses sensors to monitor the daily activities of elderly people and workers, and if an abnormality is detected, the data is sent to a server, which then sends a warning message to pre-set contacts.

[2190] 4. Behavioral analysis and visualization

[2191] The device collects daily activity data (e.g., walking distance, meal contents, sleep time, etc.) and periodically sends it to a server. The server analyzes the received data and visualizes the results as graphs and charts. This visualized data is displayed on a dashboard that can be viewed by the user and their family.

[2192] 5. Monitoring your emotional state

[2193] The device analyzes the user's voice and facial expressions to determine the user's emotional state in real time. The emotional analysis data is sent to the server, which then generates an appropriate response and sends it to the device. The device then displays the generated dialogue to the user and, if necessary, sends a message of safety confirmation or encouragement based on the user's emotional state.

[2194] Examples and prompts

[2195] 1. When an elderly person answers the device, "I had bread and coffee for breakfast today," the device sends this answer to the server, which then converts the data into numerical values ​​and records them.

[2196] 2. When a factory worker enters "I have a meeting tomorrow at 10:00 AM" into their schedule, the device sends a reminder to the server and displays a reminder notification on the device at the specified time.

[2197] Example prompt:

[2198] Ask "What did you have for lunch yesterday?" and get the user's answer. Then send the answer data to the server and store it along with the analysis results.

[2199] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2200] Step 1:

[2201] The device displays a specific question to the user every morning. For example, "What did you have for lunch yesterday?" This question is obtained from the server. The server generates question data using a generative AI model and sends it to the device.

[2202] Step 2:

[2203] The user inputs an answer to the question displayed on the terminal. For example, the user answers "I ate a sandwich." This input data is used as data for memory support for the user.

[2204] Step 3:

[2205] The device sends the user's response data to the server, which then processes the data by converting it into a number or categorizing it. For example, it converts "sandwich" into a specific number or category.

[2206] Step 4:

[2207] The server evaluates the user's cognitive function based on the saved response data, compares it with past responses stored in the database, calculates progress and changes, and generates analysis results using a generative AI model.

[2208] Step 5:

[2209] The server visualizes the analysis results as graphs and charts and sends them to the device, where users and their families can view the results on a dashboard.

[2210] Step 6:

[2211] A user inputs schedule data into a terminal. For example, the user inputs "I have a meeting tomorrow at 10:00 AM." This input data is used as schedule management data.

[2212] Step 7:

[2213] The terminal sends the input schedule data to the server, which receives the data and sets it to send a reminder notification at the specified time.

[2214] Step 8:

[2215] When the specified reminder time approaches, the server generates a reminder notification and sends it to the device. For example, it creates a notification saying "I have a meeting in 10 minutes."

[2216] Step 9:

[2217] The device will display a reminder notification to the user, who can then review and modify the action based on the notification.

[2218] Step 10:

[2219] The device monitors the user's daily activity data in real time and detects abnormal behavior or unresponsiveness, such as when the user is unresponsive after their usual lunch break.

[2220] Step 11:

[2221] If any abnormal behavior is detected, the device will send the data to the server, which will then send a warning message to pre-defined contacts if it determines that something is wrong.

[2222] Step 12:

[2223] The device analyzes the user's voice and facial expressions to determine their emotional state. The emotion analysis data is sent to the server, which then generates an appropriate response. If the device detects that the user is depressed, it generates an encouraging message.

[2224] Step 13:

[2225] The server then sends the generated dialogue to the device, which displays it to the user and continues the conversation as needed. It may also send messages of safety confirmation or encouragement.

[2226] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2227] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2228] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2229] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2230] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions general...

Claims

1. Equipped with generative artificial intelligence, it is a means of providing memory support for the elderly in their daily lives; A means of managing the schedule of seniors; A means for detecting abnormal movements of an elderly person; A means to analyze and visualize the daily behavior of elderly people, A system including:

2. 2. The system of claim 1, wherein said memory support means displays specific questions and collects the user's answers thereto.

3. 2. The system according to claim 1, wherein said schedule management means saves a schedule input by a user and notifies the user at a designated time.

4. 2. The system according to claim 1, wherein the abnormal behavior detection means recognizes daily activity patterns and sends a warning to a set contact when an abnormality is detected.

5. 2. The system according to claim 1, wherein the behavior analysis means collects daily behavior data of the user and visualizes the data based on graphs and charts.

Images