system

Abstract

Provide a system. The present invention provides a means for enabling elderly people to interact with an artificial intelligence-enabled communicator; means for analyzing emotions and states based on the dialogue; A means for determining the timing of necessary care and support based on the analysis results; A means for linking information with external services such as nursing care services and medical institutions based on the determination; The system includes a means for notifying the elderly person of the results of the collaboration.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In an aging society, appropriate care and support are important to maintain and improve the quality of life of the elderly. However, problems such as a shortage of caregivers and an increasing number of bankruptcies have led to many cases of inadequate care. It is also difficult for the elderly themselves to recognize health risks and the need for support, and to respond autonomously. Under these circumstances, there is a need for a system that allows the elderly to proactively and passively identify the timing of care and support and take appropriate action through a trusted communicator. [Means for solving the problem]

[0005] The present invention provides a system that analyzes the emotions and state of an elderly person through a dialogue with a communicator equipped with artificial intelligence, and automatically determines the timing of necessary care and support. Specifically, the system includes the following means.

[0006] A way for seniors to interact with AI-powered communicators

[0007] A means of analyzing emotions and states based on dialogue

[0008] A means of determining the timing of necessary care and support based on analysis results

[0009] A means of linking information with external services such as nursing care services and medical institutions based on the assessment

[0010] A means of notifying the elderly of the results of collaboration

[0011] The emotion and condition analysis means performs analysis using voice and text data, and the means for determining the timing of necessary care and support utilizes a health risk assessment algorithm. The means for information exchange with external services has an automatic notification function to medical institutions and care managers, and the means for notifying the elderly person communicates the content of actions in real time by voice or text. This series of means can improve the quality of life of the elderly and contribute to reducing the burden on caregivers.

[0012] "Elderly people" refers to adults over a certain age (usually 65 years or older) who often require special assistance with their lifestyle and health.

[0013] An "artificial intelligence-enabled communicator" is a device or program equipped with interactive artificial intelligence technology that has the ability to exchange information and provide support to elderly people through conversation.

[0014] "Dialogue" refers to the process of conversation between an elderly person and an AI-equipped communicator, during which various information is collected and analyzed.

[0015] "Analyzing emotions and states" refers to the process of assessing the psychological and physiological states of elderly people based on voice and text data obtained during the dialogue.

[0016] "Analysis results" refers to the evaluations and conclusions obtained using emotion and state analysis means, and provide basic data for determining the next action.

[0017] "Determining the timing of necessary care and support" refers to the process of assessing the risks faced by elderly people based on the analysis results and determining the timing and circumstances in which appropriate care and support is needed.

[0018] "External services such as nursing care services and medical institutions" refers to organizations and providers that provide specialized care and treatment to support the lives and health of older people.

[0019] "Information sharing" refers to the process of exchanging necessary information with external services to provide integrated support and care.

[0020] "Notifying" refers to the act of communicating the analysis results and next steps to elderly people and relevant parties via voice or text.

[0021] "Audio data and text data" refers to recorded audio data collected during a conversation and text data converted from that audio.

[0022] A "health risk assessment algorithm" refers to a mathematical or statistical model used to quantify the health status or likelihood of abnormalities of an elderly person and determine the risk level.

[0023] "Automatic notification function" refers to a mechanism by which the system automatically sends warnings and information to external services and related parties based on pre-set conditions.

[0024] "Real-time communication" refers to the act of communicating analysis results and next actions to the elderly as soon as possible after processing. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of the elderly. Specific embodiments of the system are described below.

[0047] System Overview

[0048] The system of the present invention mainly comprises the following components:

[0049] 1. Communicator (terminal) equipped with artificial intelligence

[0050] 2. Backend Processing Server (Server)

[0051] 3. User Interface (User)

[0052] Communicator (terminal) equipped with artificial intelligence

[0053] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[0054] Backend processing server (server)

[0055] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0056] User Interface (User)

[0057] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0058] Program processing

[0059] The specific program processing is as follows.

[0060] 1. Initial Setup and User Authentication

[0061] The user turns on the AI-enabled doll and the device starts up. The camera and microphone automatically activate at startup to perform facial and voice recognition. The authentication data is sent to the server, where the matching results are confirmed. If authentication is successful, an individual elderly person profile is loaded.

[0062] 2. Starting everyday conversations

[0063] Once authentication is complete, the device asks, "Good morning, what are your plans for today?" The user (elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[0064] 3. Initial analysis

[0065] The server analyzes the received data and performs sentiment analysis and an initial assessment of health risks. For example, if the user says "I have a stomachache," the server may initially determine that there is something wrong with their health.

[0066] 4. Follow-up questions

[0067] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[0068] 5. Detailed analysis and risk assessment

[0069] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[0070] 6. Taking Action

[0071] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[0072] Specific examples

[0073] Example 1: Detecting and responding to poor health

[0074] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[0075] 2. The device sends this message to the server.

[0076] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[0077] 4. The device asks, "Tell me more about this. How long has this been going on?"

[0078] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0079] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[0080] Example 2: Regular follow-up

[0081] 1. Schedule the server to periodically check for users.

[0082] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[0083] 3. The user responded, "Much better."

[0084] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[0085] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support.

[0086] The processing flow will be explained below.

[0087] Step 1:

[0088] The user turns on the AI-equipped doll.

[0089] The device will automatically start up and use the camera and microphone to capture the user's face and voice to begin authentication.

[0090] Step 2:

[0091] The terminal sends the authentication data to the server.

[0092] The server verifies the received authentication data and loads the user's profile.

[0093] Step 3:

[0094] Once authentication is complete, the device will ask the user, "Good morning, what are your plans for today?"

[0095] The user replies, "I don't have any plans today, but I have a stomach ache."

[0096] Step 4:

[0097] The terminal transmits the user's response as text data to the server.

[0098] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[0099] Step 5:

[0100] Based on the results of the initial evaluation, the server creates follow-up questions and sends instructions to the terminal.

[0101] The device asks the user, "How long has this been going on? Are you experiencing any other symptoms?"

[0102] Step 6:

[0103] The user replies, "It's been going on for about two days now, nothing else."

[0104] The terminal sends this new information to the server as text data.

[0105] Step 7:

[0106] The server analyzes the new data and reassess the health risk.

[0107] If the analysis results indicate a high risk, appropriate actions will be suggested.

[0108] Step 8:

[0109] The server automatically sends alerts to nearby medical institutions and care managers to arrange for the necessary support.

[0110] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0111] Step 9:

[0112] The server continuously monitors the conversation data, periodically creates health status reports, and sends them to the user's family or care manager.

[0113] The device continues to provide support by periodically following up with the user with, "How's your stomach pain going since then?"

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] It is difficult to provide timely and appropriate care and support to improve the quality of life of the elderly. Furthermore, there is a lack of a system that can accurately grasp the emotions and health status of the elderly through everyday conversations and respond when necessary. Furthermore, there is a need for effective support, including long-term assessment of health risks and necessary follow-up.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes: means for enabling the elderly person to interact with an AI-equipped communicator; means for analyzing the elderly person's emotions and health status based on the interaction; means for determining the timing of necessary care and support based on the analysis results; means for linking information with external organizations based on the determination; means for notifying the elderly person of the results of the linking; means for generating follow-up questions and collecting additional information; means for using an algorithm to perform long-term health risk assessment; and means including a function for automatically sending alerts to nearby medical institutions. This enables the elderly person's health status to be continuously monitored and nursing care and medical services to be provided at the appropriate time.

[0119] "Elderly" refers to people who are older and often require care and assistance in their daily lives.

[0120] An "artificial intelligence-enabled communicator" refers to a device equipped with conversational artificial intelligence that can hold natural conversations with the elderly.

[0121] "Dialogue" refers to communication between two people, and in this system refers to voice or text exchanges between the elderly person and an AI-powered communicator.

[0122] "Analysis of emotions and health status" means analyzing the emotions and health-related information expressed by elderly people through conversations using voice data and text data.

[0123] "Determining the timing of care and support" refers to determining the timing of the need for care and support based on the analysis of emotions and health status.

[0124] "Sharing information with external organizations" means sharing information with external organizations such as medical institutions and care service providers when the timing of necessary care or support is determined.

[0125] "Notifying the results of collaboration" refers to notifying the elderly of the results of information collaboration with external organizations.

[0126] "Generating follow-up questions" refers to the AI-enabled communicator asking additional questions to the elderly person to gather additional information based on the results of the initial analysis.

[0127] "Health risk assessment algorithm" refers to a program that uses mathematical or statistical methods to assess health risks based on collected data.

[0128] The "automatic warning sending function" refers to a function that automatically sends a warning to pre-set medical institutions or care managers when the health condition of an elderly person is judged to be at risk.

[0129] A "deep learning model" is a type of artificial intelligence technology that uses neural networks to learn patterns from large amounts of data and is applied to analyzing emotions and health conditions.

[0130] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of elderly people. Specific embodiments of the system are described below.

[0131] System configuration

[0132] The system consists of the following main components:

[0133] 1. Communicator (terminal) equipped with artificial intelligence

[0134] 2. Backend Processing Server (Server)

[0135] 3. User Interface (User)

[0136] Hardware and Software Configuration

[0137] Communicator (terminal) equipped with artificial intelligence

[0138] This device is equipped with conversational AI and is designed to engage in everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition for the user. The device uses natural language processing and voice recognition technologies to process dialogue with the user in real time.

[0139] Hardware used:

[0140] Camera module

[0141] microphone

[0142] speaker

[0143] Software used:

[0144] Conversational AI Platform

[0145] Voice recognition software (e.g., Google® Cloud Speech-to-Text)

[0146] Natural language processing engines (e.g., Python's NLTK, Microsoft® Azure®'s Text Analytics API)

[0147] Backend processing server (server)

[0148] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0149] Hardware used:

[0150] High-Performance Server

[0151] Software used:

[0152] Database Management Systems

[0153] Sentiment analysis algorithms (e.g., TENSORFLOW (registered trademark), PyTorch)

[0154] Health Risk Assessment Algorithm

[0155] User Interface (User)

[0156] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0157] Specific examples and prompts

[0158] Example 1: Detecting and responding to poor health

[0159] 1. The user consults an AI-enabled communicator, saying, "I haven't been able to sleep at night lately."

[0160] 2. The device collects this speech as audio data, converts it into text data, and sends it to the server.

[0161] 3. The server analyzes the user's recent sleep patterns and performs emotional analysis and health risk assessment. For example, if the user is likely to continue having difficulty falling asleep, it may recommend a visit to a medical institution.

[0162] 4. The device will ask, "Tell me more about this. How long has it been going on?"

[0163] 5. If the user answers "about a week ago," the server updates the risk assessment and instructs the appropriate response.

[0164] Example 2: Regular follow-up

[0165] 1. The server schedules periodic checks of the user.

[0166] 2. The device will periodically follow up by asking, "How is your stomach pain since then?"

[0167] 3. If the user responds "much better," the server analyzes the received data, updates the risk assessment, and determines it to be low risk.

[0168] 4. The device will notify you to "remain careful."

[0169] Example prompt sentence:

[0170] "I haven't been able to sleep at night lately. Do you have any advice?"

[0171] "I'm having persistent stomach pains, what should I do?"

[0172] "I haven't been feeling well lately, maybe something's wrong?"

[0173] This will enable the system to provide timely and appropriate care and support to improve the quality of life of the elderly.

[0174] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0175] Step 1: Initial configuration and user authentication

[0176] When the device is turned on, the camera and microphone automatically start working, and facial and voice recognition is performed.

[0177] Input: User's face image and voice data

[0178] Data processing: Analyze facial image and audio data using appropriate recognition algorithms (e.g., Google's Facial Recognition Engine, Amazon's AWS® Rekognition).

[0179] Output: Authentication result (success / failure) and user profile

[0180] The server receives the authentication data sent from the device and compares it with the database. If the authentication result is successful, the individual elderly person's profile is loaded and notified to the device.

[0181] Step 2: Starting everyday conversations

[0182] Once authentication is complete, the device greets you with, "Good morning, what's on your agenda today?"

[0183] Input: User response (e.g., "I don't have any plans today, but I have a stomach ache.")

[0184] Data processing: User responses are collected as voice data and converted into text data through voice recognition.

[0185] Output: Text data (user's speech)

[0186] The terminal converts the collected voice data into text data in real time and sends it to the server.

[0187] Step 3: Initial analysis

[0188] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[0189] Input: Text data (user speech)

[0190] Data processing: Natural language processing (e.g., Python's NLTK, Microsoft Azure's Text Analytics API) is used to extract keywords and assess emotions and health status.

[0191] Output: Initial assessment result (e.g., health risk based on the information "I have a stomachache")

[0192] The server determines whether there is anything abnormal in the user's health condition based on the user's comments.

[0193] Step 4: Follow-up questions

[0194] If the server detects an abnormality, it generates an additional question and sends it to the terminal.

[0195] Input: Initial health risk assessment results

[0196] Data processing: Applying algorithms to generate questions to gather more information from the user.

[0197] Output: Follow-up question (e.g., "How long has this been going on?")

[0198] The terminal then asks the generated follow-up questions to the user.

[0199] Step 5: Detailed analysis and risk assessment

[0200] The user answers the follow-up questions, which are received by the server and analyzed again in detail.

[0201] Input: User's detailed response (e.g., "It's been going on for about two days now, nothing else.")

[0202] Data processing: Using deep learning models (e.g., TensorFlow, PyTorch) to update risk assessments.

[0203] Output: Updated health risk assessment

[0204] The server reassess the health risk based on the user's detailed answers.

[0205] Step 6: Take Action

[0206] If the server determines that the risk is high, it sends a warning to nearby medical institutions and care managers.

[0207] Input: Updated health risk assessment

[0208] Data Processing: Contact appropriate external agencies based on pre-set protocols.

[0209] Output: Automatic alerts and delivery service instructions

[0210] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0211] The user receives this notification and takes the necessary action.

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] To improve the quality of life of the elderly, it is necessary to monitor their emotions and health status on a daily basis and provide care and support at the appropriate time. However, existing systems lack real-time monitoring to detect abnormalities, and even minor abnormalities may be overlooked. In addition, effective support may be delayed due to a lack of follow-up questions and appropriate measures after abnormalities are detected. As a result, there is a risk that elderly people will not receive appropriate care when they need it, and their quality of life will decline.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for analyzing emotions and conditions, means for determining the timing of necessary care and support, means for linking information with external services, means for notifying the elderly person of the linking results, means for monitoring changes in facial expressions and voice in real time, means for detecting abnormalities and presenting follow-up questions when an abnormality is detected, and means for analyzing answers to the follow-up questions and taking appropriate measures as necessary. This makes it possible to monitor the elderly person's condition in real time and take measures for care and support in a timely manner.

[0217] 1. "Elderly" refers to individuals over a certain age, often in retirement, who require special care and assistance in their daily lives.

[0218] 2. "Artificial intelligence communicator" refers to a device that uses AI technology to have a dialogue function and analyzes the emotions and health status of users (elderly people) through communication with them.

[0219] 3. "Means for analyzing emotions and states" refers to the process of assessing and analyzing the emotional and physical states of older adults based on input information such as voice data and text data.

[0220] 4. "Means for determining the timing of care or support" refers to algorithms or technologies that, based on the results of analysis, determine the timing and circumstances under which an elderly person will need care or support.

[0221] 5. "Means for sharing information with external services" refers to the process of sharing information with external institutions such as nursing care services and medical institutions, and coordinating the necessary care and support.

[0222] 6. "Means for notifying the elderly of the results of collaboration" refers to mechanisms and technologies for communicating the results of information collaboration with external services to the elderly in an easy-to-understand manner.

[0223] 7. "Means for monitoring changes in facial expressions and voice in real time" refers to the process of instantly observing changes in an elderly person's facial expressions and tone of voice through cameras and microphones and collecting data.

[0224] 8. "Means for detecting anomalies" refers to systems or technologies that analyze collected data and identify abnormalities that deviate from normal conditions.

[0225] 9. "Means for presenting follow-up questions" refers to a function that automatically generates additional questions to ask the elderly person when an abnormality is detected.

[0226] 10. "Means for analyzing responses to follow-up questions" refers to the process of analyzing responses from older people, extracting further information, and assessing their condition.

[0227] 11. "Means of taking appropriate measures" refers to the ability to take action to provide appropriate care and support to older people based on the results of the analysis.

[0228] This invention is a system that enables elderly people to communicate using a communicator equipped with artificial intelligence, analyzes their emotions and health status, and provides appropriate care and support. This system is implemented using the following hardware and software.

[0229] Hardware used

[0230] 1. Camera: A device for capturing facial expressions of elderly people in real time, used for facial recognition and expression analysis.

[0231] 2. Microphone: A device for obtaining voice input. It captures what the elderly person is saying and uses it to analyze their emotions and health status.

[0232] 3. Server: A central data processing unit for data processing and analysis. It is responsible for data management, analysis, and linking with external services.

[0233] 4. Communicator: A device for everyday communication for the elderly. Built-in AI enables natural conversation.

[0234] Software used

[0235] 1. OpenCV: A library for processing video data acquired from a camera and performing facial recognition and facial expression analysis.

[0236] 2. SpeechRecognition: A library for converting voice data into text data. It provides speech recognition functionality.

[0237] 3. TextBlob: A natural language processing library for parsing text data and performing sentiment assessment.

[0238] Data processing and calculation

[0239] 1. Real-time monitoring of facial expressions: Video data captured by the camera is processed in real time using OpenCV to perform facial recognition. Emotion analysis is performed from the recognized facial expressions to detect abnormalities.

[0240] 2. Real-time voice monitoring and analysis: Voice data acquired through a microphone is converted into text data using SpeechRecognition, and sentiment analysis is performed using TextBlob.

[0241] 3. Anomaly detection and follow-up: If an anomaly is detected as a result of sentiment analysis, the server generates follow-up questions and presents them to the elderly via the communicator. The elderly's answers are analyzed again to determine the necessary measures.

[0242] 4. Collaboration with external services: Based on the assessment results, the server will collaborate with external care services and medical institutions and notify the elderly of the results.

[0243] Specific examples

[0244] As a concrete example, consider the case where an elderly person says to the communicator, "I feel a little tired today." In this case, the microphone captures the voice and sends the data to the server. The server converts the voice data into text data using SpeechRecognition and performs sentiment analysis using TextBlob. If the sentiment analysis result is evaluated as "negative," the server generates a follow-up question, asking, "How long has this been going on?" The answer to this question is then reanalyzed, and appropriate measures are taken based on the results.

[0245] An example of a prompt is, "If a worker says, 'I'm feeling a bit tired today,' please write a short code that automatically analyzes their emotions and sends an alert based on the results." Based on this prompt, the system can automatically generate a corresponding code, enabling prompt and appropriate care for the elderly.

[0246] With the above configuration, the present invention makes it possible to monitor the emotions and health status of elderly people in real time and provide care and support at appropriate times.

[0247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0248] Step 1: User authentication and configuration

[0249] When a user turns on the communicator, the device boots up and the built-in camera and microphone start working automatically. The photographed facial recognition data and recorded voice recognition data are sent to the server for matching. If the authentication is successful, the profile of each elderly person is loaded. Input: Facial recognition data and voice recognition data. Data processing: Data matching. Output: Authentication results and loading of user profile.

[0250] Step 2: Starting everyday conversations

[0251] After authentication is complete, the device speaks to the user, saying, "Good morning. What do you have planned for today?" If the user (elderly person) replies, "I don't have any plans for today, but I have a stomachache," the voice data is sent to the server in real time. Input: User's speech. Data processing: Voice data converted to text. Output: Text data.

[0252] Step 3: Sentiment Analysis

[0253] The server analyzes the received text data and performs sentiment analysis and an initial assessment of health risks. For example, from the information "I have a stomachache," the server may initially determine that there is something wrong with the user's health. Input: User's spoken text data. Data processing: Sentiment analysis and health risk assessment. Output: Initial health assessment results.

[0254] Step 4: Follow-up questions

[0255] If the server detects an abnormality, it generates follow-up questions to gather more information and provides them to the terminal. The terminal asks the user, "How long has this been going on? Are there any other symptoms?" Input: Initial health assessment results. Data processing: Generation of follow-up questions. Output: Follow-up questions.

[0256] Step 5: Analyze follow-up responses

[0257] After receiving additional responses from the user, the device sends the responses to the server. The server analyzes them again and updates the health risk assessment. For example, if the user answers, "It's been going on for about two days now," the server reassess the detailed situation. Input: User's follow-up response. Data processing: Analysis of additional information and risk reassessment. Output: Updated health risk assessment results.

[0258] Step 6: Take Action

[0259] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care services, and notifies the user of this information via their device. It also arranges for food and medicine delivery services as needed. Input: Updated health risk assessment results. Data processing: Linkage processing with external services. Output: Sending a warning to medical institutions and notifying the user.

[0260] The above processing steps realize a system that monitors the emotions and health status of elderly people in real time and provides care and support at the appropriate time.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] This invention is a system that uses a communicator equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of the elderly. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[0263] System Overview

[0264] The system of the present invention mainly comprises the following components:

[0265] 1. Communicator (terminal) equipped with artificial intelligence

[0266] 2. Emotion Engine

[0267] 3. Backend Processing Server (Server)

[0268] 4. User Interface (User)

[0269] Communicator (terminal) equipped with artificial intelligence

[0270] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[0271] Emotion Engine

[0272] The emotion engine analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses emotion analysis algorithms to determine the user's psychological state and sends that information to a back-end processing server.

[0273] Backend processing server (server)

[0274] The server comprehensively analyzes data sent from the device and emotion engine to evaluate emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0275] User Interface (User)

[0276] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0277] Program processing

[0278] The specific program processing is as follows.

[0279] 1. Initial Setup and User Authentication

[0280] When a user turns on the AI-powered doll, the device automatically starts up and uses the camera and microphone to capture the user's face and voice to begin authentication. The authentication data is sent to the server, where the matched results are confirmed. If authentication is successful, a personalized elderly profile is loaded.

[0281] 2. Starting everyday conversations

[0282] Once authentication is complete, the device speaks to the user, asking, "Good morning. What are your plans for today?" The user (an elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[0283] 3. Initial Analysis and Sentiment Analysis

[0284] The server and emotion engine work together to analyze the received data. The emotion engine uses a natural language processing algorithm to evaluate the user's emotional state in real time from the voice data and to analyze the user's emotional state from the text data. For example, if the information says "I have a stomachache," the server will initially determine that there is something wrong with the user's health based on the emotion engine's analysis results.

[0285] 4. Follow-up questions

[0286] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[0287] 5. Detailed analysis and risk assessment

[0288] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[0289] 6. Taking Action

[0290] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[0291] Specific examples

[0292] Example 1: Detecting and responding to poor health

[0293] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[0294] 2. The device sends this message to the server.

[0295] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[0296] 4. The device asks, "Tell me more about this. How long has this been going on?"

[0297] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0298] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[0299] Example 2: Regular follow-up

[0300] 1. Schedule the server to periodically check for users.

[0301] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[0302] 3. The user responded, "Much better."

[0303] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[0304] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support. In particular, by combining it with an emotion engine, it will be possible to accurately grasp the user's emotional state and provide appropriate action immediately when necessary.

[0305] The processing flow will be explained below.

[0306] Step 1:

[0307] The user turns on the AI-powered doll. The device automatically starts up and uses the camera and microphone to capture the user's face and voice, then begins authentication.

[0308] Step 2:

[0309] The device sends authentication data to the server, which verifies the received authentication data and loads the user's profile.

[0310] Step 3:

[0311] Once authentication is complete, the device asks the user, "Good morning, what are your plans for today?" The user replies, "I don't have any plans for today, but I have a stomachache."

[0312] Step 4:

[0313] The device sends the user's response as text data to the server, which then analyzes the received text data and performs sentiment analysis and an initial assessment of health risks.

[0314] Step 5:

[0315] The emotion engine analyzes the voice data sent from the server and evaluates the user's emotional state in real time, for example, determining the level of anxiety or pain.

[0316] Step 6:

[0317] Based on the results of the initial assessment and the analysis results of the emotion engine, the server creates follow-up questions and sends instructions to the device. The device asks the user, "How long has this been going on? Are there any other symptoms?"

[0318] Step 7:

[0319] The user responds, "It's been going on for about two days now, and nothing else." The device sends this new information as text data to the server.

[0320] Step 8:

[0321] The server analyzes the new data and reassesses health risks. Using an analytical algorithm, it determines the risk level in detail. At the same time, the emotion engine reassessessments based on the updated voice and text data.

[0322] Step 9:

[0323] If the server determines that a patient is at high risk, it will automatically send a warning to nearby medical institutions and care managers. It can also instruct partner delivery services to provide specific medicines or ingredients.

[0324] Step 10:

[0325] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0326] Step 11:

[0327] The server continuously monitors the conversation data and the analysis results of the emotion engine, and periodically creates a health status report and sends it to the user's family or care manager. The device periodically follows up with the user, asking, "How's your stomach pain going?" to provide ongoing support.

[0328] Example 2

[0329] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] Today's elderly often experience a decline in their quality of life due to physical and psychological problems associated with aging. Therefore, it is necessary to create an environment that supports the elderly in their daily lives and allows them to receive medical and nursing care services in a timely manner. However, current systems make it difficult to analyze the emotions and health status of the elderly in real time and provide appropriate responses immediately. Furthermore, there is a lack of means to detect health risks faced by the elderly early and to quickly share information.

[0331] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for enabling the elderly person to interact with a communication device; a means for analyzing the elderly person's emotions and health status based on the interaction; a means for determining the timing of necessary care and support based on the analysis results; a means for linking information with external organizations such as nursing care services and medical institutions based on the determination; a means for notifying the elderly person of the linking results; a means for recognizing the user's face and voice and reading an individual profile; a means for asking follow-up questions and collecting detailed data; a means for evaluating health risks based on the received data and updating information; and a means for sending an alert to a medical institution and instructing the provision of necessary medicines and ingredients if an abnormality is detected. This makes it possible to analyze the elderly person's emotions and health status in real time and provide appropriate care and support promptly.

[0332] "Communication devices" refer to devices used by elderly people to communicate, including devices equipped with voice recognition and cameras.

[0333] "Dialogue" refers to the verbal exchange between the elderly person and the communication device, and takes place in the form of voice or text data.

[0334] "Analyzing emotions and health status" refers to analyzing the psychological and physical state of elderly people based on their voice and text data.

[0335] "Analysis results" refers to information obtained through analytical means regarding emotions and health status.

[0336] "Determining the timing of care and support" refers to determining the appropriate time to provide the necessary care and support to the elderly based on the analysis results.

[0337] "External agencies" refers to external organizations and services involved in supporting the elderly, such as nursing care services and medical institutions.

[0338] "Collaboration" refers to sharing information with external organizations and cooperating with each other.

[0339] "Collaboration results" refers to the results and feedback obtained after sharing information with external organizations.

[0340] "Recognizing the user's face and voice" refers to the communication device capturing the facial image and voice of the elderly person and identifying the individual.

[0341] "Loading an individual profile" refers to retrieving and applying individual data stored for each recognized user.

[0342] A "follow-up question" is a question asked to gather additional information based on the initial dialogue or analysis.

[0343] "Detailed data" refers to more specific information obtained through follow-up questions.

[0344] "Assessing health risks" refers to determining the risks associated with the health status of elderly people based on detailed data and analysis results.

[0345] "Updating information" refers to reflecting new data and analysis results in the records within the system.

[0346] "Sending an alert" refers to sending information to alert medical institutions and related parties when an abnormality is detected.

[0347] "Instructing the provision of medicines and food ingredients" refers to issuing instructions to arrange for the supply of necessary medicines and food ingredients based on the health condition of the elderly person.

[0348] This invention is a system that uses communication devices equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of elderly people. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[0349] Hardware and software used

[0350] Communication device: An interactive AI device for everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user. Through this device, conversations with the elderly are conducted in real time.

[0351] Emotion engine: A software module that analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses an emotion analysis algorithm to determine the user's psychological state.

[0352] Back-end processing server: Comprehensively analyzes data sent from communication devices and the emotion engine to evaluate emotions and health risks. This server uses a health risk assessment algorithm to determine the timing of care and support, and notifies external organizations as necessary.

[0353] Generative AI models: Use specific AI models for natural language processing and speech recognition.

[0354] Specific processing of the program

[0355] 1. The server receives the voice and text data sent from the communication device and analyzes it in cooperation with the emotion engine.

[0356] 2. The emotion engine evaluates the user's emotional state in real time from the voice data and also analyzes the emotional state from the text data, thereby identifying emotions such as "anxiety" or "anger."

[0357] 3. The server evaluates health risks based on the analysis results and determines the timing of necessary care and support. If a high risk is determined, the server shares information with external organizations and notifies the user.

[0358] 4. The communication device asks additional follow-up questions to gather more detailed data, such as "How long has this been going on?" or "Are you experiencing any other symptoms?" to get more detailed information from the user.

[0359] 5. The server re-analyzes the detailed data and updates the health risk assessment, taking necessary actions accordingly.

[0360] Specific examples

[0361] Example 1: Detecting and responding to poor health

[0362] 1. The user talks to the communication device saying, "I haven't been able to sleep at night lately."

[0363] 2. The communication device sends this message to the server.

[0364] 3. The server performs an emotional analysis of the "recent sleep patterns" and a health risk assessment, and if the "difficulty falling asleep" continues, recommends a visit to a medical institution.

[0365] 4. The communication device asks, "Tell me more about this. How long has this been going on?"

[0366] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0367] 6. The server automatically contacts a nearby medical institution and notifies the user via the communication device, saying, "We're worried, so we've contacted the clinic."

[0368] Example 2: Regular follow-up

[0369] 1. Schedule the server to periodically check for users.

[0370] 2. The communication device will follow up periodically with, "How's the stomach ache going since then?"

[0371] 3. The user responds, "It's much better."

[0372] 4. The server analyzes the received data and determines that the risk has decreased. The communication device sends a message saying, "Please continue to be careful."

[0373] Example prompts for generative AI models

[0374] "Please explain the process by which an elderly person talks to an AI-enabled communication device about their recent health condition, and the emotion engine analyzes that information and provides the necessary assistance."

[0375] The present invention makes it possible to perform analysis using an emotion engine in particular, thereby enabling the user's emotional state to be grasped with high accuracy and appropriate care and support to be provided promptly.

[0376] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0377] Step 1:

[0378] The user turns on the communication device. The communication device uses a camera and microphone to capture the user's face and voice. The input data (facial recognition data and voice data) obtained through this process is sent to the server, which then authenticates the user. If authentication is successful, an individual elderly profile is loaded. Specifically, the camera on the communication device captures facial data and sends voice data to the recognition module. The server then uses facial recognition algorithms and voiceprint analysis algorithms to recognize the user.

[0379] Step 2:

[0380] Once authentication is complete, the communication device speaks to the user, asking, "Good morning. What do you have planned for today?" To which the user responds, "I don't have any plans for today, but I have a stomachache." This response is converted into text data in real time and sent to the server. The input includes the user's voice data, and the output includes the text conversion result of that voice data. Specifically, the voice recognition module of the communication device converts the user's speech into text data, and that text data is sent to the server via the Internet.

[0381] Step 3:

[0382] The server works with the emotion engine to analyze the received text data. The emotion engine evaluates the user's emotional state in real time from the voice data and similarly analyzes the emotional state from the text data. For example, emotions such as "discomfort" or "pain" are extracted from the text "my stomach hurts." Based on the results of this analysis, the server makes an initial judgment that there is something wrong with the user's health. The input data is text data, and the output data is the user's emotional state and health risk assessment. The emotion engine uses a natural language processing algorithm to analyze the text data and classify emotions.

[0383] Step 4:

[0384] If the server detects an abnormality, it instructs the communication device to ask follow-up questions to gather additional information. The communication device asks the user, "How long has this been going on? Are there any other symptoms?" to collect detailed information. If the user answers, "It's been going on for about two days now," the answer is again converted into text data and sent to the server. The input contains the user's answer to the follow-up question, and the output contains detailed text data. Specifically, the communication device performs voice recognition again and sends the follow-up question and its answer to the server as text data.

[0385] Step 5:

[0386] The server reassesses the health risk based on the detailed text data. For example, based on the response "It's been going on for about two days, but nothing else in particular," it uses an analysis algorithm to reassess the risk level. The input data is the response to the follow-up question, and the output data is the updated health risk assessment. The server uses a high-performance analysis module to analyze the obtained detailed data and reassess the health risk.

[0387] Step 6:

[0388] If the risk is determined to be high, the server sends an alert to nearby medical institutions and care managers, instructing them to provide the necessary medications and food ingredients. This information is transmitted to the user via the communication device, which sends a message saying, "I'm worried, so I've contacted a nearby clinic. Please rest." The input data is the updated health risk assessment and detailed text data, and the output data is the alert notification to the medical institution and the notification message to the user. Specifically, the server uses a communication module to send the alert data to the medical institution, and notifies the user via the speaker and display of the communication device.

[0389] keyword

[0390] Generative AI model, prompt sentence

[0391] (Application example 2)

[0392] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] In order to improve the quality of life of the elderly, not only appropriate nursing care and medical support but also daily health management is important. However, it is not easy for the elderly themselves to understand changes in their own health condition and emotions and take appropriate action. Furthermore, diet has a significant impact on health, so it is important to provide the elderly with appropriate meals, but it is difficult for them to choose the meals that are optimal for their health condition. To solve these issues, a system is needed that is deeply involved in the daily lives of the elderly and can make appropriate meal suggestions and provide them in real time based on their health condition.

[0394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0395] In this invention, the server includes a means for enabling the elderly person to converse with a communicator equipped with artificial intelligence, a means for analyzing emotions and conditions based on the converse, and a means for determining the timing of necessary care and support based on the analysis results. This makes it possible to provide appropriate care and support, as well as to suggest and provide appropriate meals based on emotions and health conditions.

[0396] "Elderly people" refers to people of an age who require care and assistance in their daily lives.

[0397] An "artificial intelligence communicator" refers to a computer system that converses with elderly people and collects and analyzes voice and text data.

[0398] "Dialogue" refers to voice or text communication between an elderly person and an AI-enabled communicator.

[0399] "Means for analyzing emotions and conditions" refers to algorithms and systems that evaluate the emotional and health conditions of elderly people based on acquired voice and text data.

[0400] "Means for determining the timing of necessary care and support" refers to algorithms and systems that determine when care and support is needed based on the analysis results.

[0401] "Means for sharing information with external services" refers to communication functions for sharing analysis and assessment results with external nursing care services and medical institutions.

[0402] "Means for notifying the elderly of the results of collaboration" refers to a system that communicates the results of collaboration with external services to the elderly via voice or text message.

[0403] "Means for making appropriate meal suggestions" refers to algorithms or systems that select and suggest meal menus that are optimal for a person's health based on the results of analysis and assessment.

[0404] "Means for automated delivery" refers to a system that automatically orders the necessary ingredients and food based on a proposed meal menu and delivers them to the elderly person's home.

[0405] MODE FOR CARRYING OUT THE INVENTION

[0406] To improve the quality of life of the elderly, this invention provides a system in which the elderly interact with a communicator equipped with artificial intelligence, and the emotions and health status obtained through the interaction are analyzed to determine the timing of necessary care and support. This system automates the entire process from suggesting appropriate meals based on the analysis results to delivering those meals.

[0407] System configuration

[0408] The system mainly consists of the following components:

[0409] 1. Artificial Intelligence Communicator

[0410] 2. Emotion Engine

[0411] 3. Backend Processing Server

[0412] 4. User Interface

[0413] 5. External service integration system

[0414] Artificial Intelligence-Powered Communicator

[0415] The Communicator is a device for communicating with the elderly. It has a built-in camera and microphone, and can perform facial recognition and voice recognition of the elderly. For facial recognition, it uses Google Face API or Amazon Rekognition, and for voice recognition, it uses Google Cloud Speech-to-Text.

[0416] Emotion Engine

[0417] The emotion engine analyzes the acquired voice and text data and evaluates the emotional state of the elderly person in real time. For example, IBM Watson (registered trademark) Tone Analyzer can be used.

[0418] Backend Processing Server

[0419] The server evaluates health risks based on the data sent from the emotion engine and makes appropriate dietary recommendations. It also determines the timing of necessary care and support and shares the results with external services. The server uses cloud services such as AWS EC2.

[0420] User Interface

[0421] Through the user interface, elderly people can communicate their condition to the communicator through natural dialogue, and notifications and suggestions from the server are also displayed through this interface.

[0422] External service integration system

[0423] The external service integration system suggests appropriate meals based on the analysis results and automatically handles the ordering and delivery procedures for delivering those meals.

[0424] How the system works

[0425] 1. Initial Setup and User Authentication

[0426] The server authenticates the elderly person's face and voice and sends the authentication data to the server, which then loads an individual elderly person profile.

[0427] 2. Starting everyday conversations

[0428] The communicator talks to the elderly and asks them questions about their everyday health, for example, a conversation about "stomach pain."

[0429] 3. Initial Analysis and Sentiment Analysis

[0430] The emotion engine analyzes the voice and text data obtained through the dialogue and sends the results to the server.

[0431] 4. Follow-up questions and detailed analysis

[0432] The server will ask follow-up questions to gather additional information and conduct a detailed health risk assessment.

[0433] 5. Meal suggestions and delivery arrangements

[0434] We suggest an appropriate meal menu based on the elderly person's health condition, and arrange for the ingredients to be ordered and delivered by a delivery service based on that menu.

[0435] Specific examples

[0436] Example 1: User complaining of indigestion

[0437] 1. The user tells the communicator, "I have a stomachache."

[0438] 2. The communicator sends this utterance to the server.

[0439] 3. The emotion engine determines the emotion of "painful" and sends the result to the server.

[0440] 4. The server suggests to the user a menu of easily digestible foods (e.g., porridge or soup).

[0441] 5. Order food from a delivery service and arrange for it to be delivered to the senior's home.

[0442] Prompt Sentence Examples

[0443] You are an AI-powered communicator. A 70-year-old woman has recently been complaining of stomach aches and would like recommendations for foods that are easy to digest. Use your emotion engine to analyze this information and suggest appropriate menu items.

[0444] By combining these detailed procedures and techniques, the present invention can significantly improve the quality of life for the elderly.

[0445] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0446] Step 1:

[0447] The user speaks to an AI-powered communicator, saying, "My stomach hurts."

[0448] How it works: The device's microphone captures audio data and converts it into text data in real time.

[0449] Input: User's voice

[0450] Output: Text data

[0451] Step 2:

[0452] The terminal transmits the text data to the server.

[0453] How it works: The communicator sends the generated text data to the server via the cloud.

[0454] Input: Text data

[0455] Output: Data received by the server

[0456] Step 3:

[0457] The server sends the text data to the emotion engine for emotion analysis.

[0458] How it works: The server sends the text data to an analysis engine to assess the emotional state, for example using IBM Watson's Tone Analyzer.

[0459] Input: Text data

[0460] Output: Emotion analysis results

[0461] Step 4:

[0462] The emotion engine sends the analysis results back to the server.

[0463] How it works: The emotion engine identifies emotions such as "sad" and sends the results to the server.

[0464] Input: Text data

[0465] Output: Emotion analysis results

[0466] Step 5:

[0467] The server assesses the health risk and generates appropriate follow-up questions.

[0468] How it works: The server runs an algorithm to assess health risk based on the sentiment analysis results. If a risk is determined, it generates a follow-up question (e.g., "How long has this been going on?").

[0469] Input: Sentiment analysis results

[0470] Output: Follow-up question

[0471] Step 6:

[0472] The terminal presents the user with follow-up questions.

[0473] How it works: The device speaks the generated follow-up question to the user.

[0474] Input: Follow-up question

[0475] Output: Audio output

[0476] Step 7:

[0477] The user answers the follow-up question.

[0478] How it works: The user answers the device, "It's been going on for about two days," and the device again captures the voice data and converts it into text data.

[0479] Input: Voice response

[0480] Output: Text data

[0481] Step 8:

[0482] The device sends the text data to a server for detailed analysis.

[0483] How it works: The device sends the text data to a server via the cloud, where it begins detailed analysis. The server then uses an analytical algorithm to reassess the health risk.

[0484] Input: Text data

[0485] Output: Detailed analysis results

[0486] Step 9:

[0487] The server generates appropriate meal suggestions.

[0488] How it works: Based on the results of the detailed analysis, the server generates an appropriate meal menu and creates data to order from a delivery service.

[0489] Input: Detailed analysis results

[0490] Output: Meal suggestions and ordering data

[0491] Step 10:

[0492] The delivery service will process the delivery of the ingredients.

[0493] How it works: The server sends the order data to the delivery service and automatically processes the delivery of the ingredients.

[0494] Input: Order data

[0495] Output: Delivery arrangement

[0496] Step 11:

[0497] The terminal notifies the user of the delivery information.

[0498] How it works: The device receives a notification from the server and notifies the user of the order details and delivery time via voice or text.

[0499] Input:Shipping information

[0500] Output: Voice or text notification

[0501] This will enable elderly people to receive appropriate care at home while smoothly obtaining the optimal diet for their health.

[0502] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0503] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0504] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0505] [Second embodiment]

[0506] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0507] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0508] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0509] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0510] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0511] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0512] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0513] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0514] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0515] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0516] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0517] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0518] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of the elderly. Specific embodiments of the system are described below.

[0519] System Overview

[0520] The system of the present invention mainly comprises the following components:

[0521] 1. Communicator (terminal) equipped with artificial intelligence

[0522] 2. Backend Processing Server (Server)

[0523] 3. User Interface (User)

[0524] Communicator (terminal) equipped with artificial intelligence

[0525] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[0526] Backend processing server (server)

[0527] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0528] User Interface (User)

[0529] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0530] Program processing

[0531] The specific program processing is as follows.

[0532] 1. Initial Setup and User Authentication

[0533] The user turns on the AI-enabled doll and the device starts up. The camera and microphone automatically activate at startup to perform facial and voice recognition. The authentication data is sent to the server, where the matching results are confirmed. If authentication is successful, an individual elderly person profile is loaded.

[0534] 2. Starting everyday conversations

[0535] Once authentication is complete, the device asks, "Good morning, what are your plans for today?" The user (elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[0536] 3. Initial analysis

[0537] The server analyzes the received data and performs sentiment analysis and an initial assessment of health risks. For example, if the user says "I have a stomachache," the server may initially determine that there is something wrong with their health.

[0538] 4. Follow-up questions

[0539] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[0540] 5. Detailed analysis and risk assessment

[0541] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[0542] 6. Taking Action

[0543] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[0544] Specific examples

[0545] Example 1: Detecting and responding to poor health

[0546] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[0547] 2. The device sends this message to the server.

[0548] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[0549] 4. The device asks, "Tell me more about this. How long has this been going on?"

[0550] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0551] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[0552] Example 2: Regular follow-up

[0553] 1. Schedule the server to periodically check for users.

[0554] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[0555] 3. The user responded, "Much better."

[0556] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[0557] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support.

[0558] The processing flow will be explained below.

[0559] Step 1:

[0560] The user turns on the AI-equipped doll.

[0561] The device will automatically start up and use the camera and microphone to capture the user's face and voice to begin authentication.

[0562] Step 2:

[0563] The terminal sends the authentication data to the server.

[0564] The server verifies the received authentication data and loads the user's profile.

[0565] Step 3:

[0566] Once authentication is complete, the device will ask the user, "Good morning, what are your plans for today?"

[0567] The user replies, "I don't have any plans today, but I have a stomach ache."

[0568] Step 4:

[0569] The terminal transmits the user's response as text data to the server.

[0570] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[0571] Step 5:

[0572] Based on the results of the initial evaluation, the server creates follow-up questions and sends instructions to the terminal.

[0573] The device asks the user, "How long has this been going on? Are you experiencing any other symptoms?"

[0574] Step 6:

[0575] The user replies, "It's been going on for about two days now, nothing else."

[0576] The terminal sends this new information to the server as text data.

[0577] Step 7:

[0578] The server analyzes the new data and reassess the health risk.

[0579] If the analysis results indicate a high risk, appropriate actions will be suggested.

[0580] Step 8:

[0581] The server automatically sends alerts to nearby medical institutions and care managers to arrange for the necessary support.

[0582] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0583] Step 9:

[0584] The server continuously monitors the conversation data, periodically creates health status reports, and sends them to the user's family or care manager.

[0585] The device continues to provide support by periodically following up with the user with, "How's your stomach pain going since then?"

[0586] Example 1

[0587] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] It is difficult to provide timely and appropriate care and support to improve the quality of life of the elderly. Furthermore, there is a lack of a system that can accurately grasp the emotions and health status of the elderly through everyday conversations and respond when necessary. Furthermore, there is a need for effective support, including long-term assessment of health risks and necessary follow-up.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0590] In this invention, the server includes: means for enabling the elderly person to interact with an AI-equipped communicator; means for analyzing the elderly person's emotions and health status based on the interaction; means for determining the timing of necessary care and support based on the analysis results; means for linking information with external organizations based on the determination; means for notifying the elderly person of the results of the linking; means for generating follow-up questions and collecting additional information; means for using an algorithm to perform long-term health risk assessment; and means including a function for automatically sending alerts to nearby medical institutions. This enables the elderly person's health status to be continuously monitored and nursing care and medical services to be provided at the appropriate time.

[0591] "Elderly" refers to people who are older and often require care and assistance in their daily lives.

[0592] An "artificial intelligence-enabled communicator" refers to a device equipped with conversational artificial intelligence that can hold natural conversations with the elderly.

[0593] "Dialogue" refers to communication between two people, and in this system refers to voice or text exchanges between the elderly person and an AI-powered communicator.

[0594] "Analysis of emotions and health status" means analyzing the emotions and health-related information expressed by elderly people through conversations using voice data and text data.

[0595] "Determining the timing of care and support" refers to determining the timing of the need for care and support based on the analysis of emotions and health status.

[0596] "Sharing information with external organizations" means sharing information with external organizations such as medical institutions and care service providers when the timing of necessary care or support is determined.

[0597] "Notifying the results of collaboration" refers to notifying the elderly of the results of information collaboration with external organizations.

[0598] "Generating follow-up questions" refers to the AI-enabled communicator asking additional questions to the elderly person to gather additional information based on the results of the initial analysis.

[0599] "Health risk assessment algorithm" refers to a program that uses mathematical or statistical methods to assess health risks based on collected data.

[0600] The "automatic warning sending function" refers to a function that automatically sends a warning to pre-set medical institutions or care managers when the health condition of an elderly person is judged to be at risk.

[0601] A "deep learning model" is a type of artificial intelligence technology that uses neural networks to learn patterns from large amounts of data and is applied to analyzing emotions and health conditions.

[0602] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of elderly people. Specific embodiments of the system are described below.

[0603] System configuration

[0604] The system consists of the following main components:

[0605] 1. Communicator (terminal) equipped with artificial intelligence

[0606] 2. Backend Processing Server (Server)

[0607] 3. User Interface (User)

[0608] Hardware and Software Configuration

[0609] Communicator (terminal) equipped with artificial intelligence

[0610] This device is equipped with conversational AI and is designed to engage in everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition for the user. The device uses natural language processing and voice recognition technologies to process dialogue with the user in real time.

[0611] Hardware used:

[0612] Camera module

[0613] microphone

[0614] speaker

[0615] Software used:

[0616] Conversational AI Platform

[0617] Voice recognition software (e.g., Google Cloud Speech-to-Text)

[0618] Natural language processing engines (e.g., Python's NLTK, Microsoft Azure's Text Analytics API)

[0619] Backend processing server (server)

[0620] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0621] Hardware used:

[0622] High-Performance Server

[0623] Software used:

[0624] Database Management Systems

[0625] Sentiment analysis algorithms (e.g., TensorFlow, PyTorch)

[0626] Health Risk Assessment Algorithm

[0627] User Interface (User)

[0628] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0629] Specific examples and prompts

[0630] Example 1: Detecting and responding to poor health

[0631] 1. The user consults an AI-enabled communicator, saying, "I haven't been able to sleep at night lately."

[0632] 2. The device collects this speech as audio data, converts it into text data, and sends it to the server.

[0633] 3. The server analyzes the user's recent sleep patterns and performs emotional analysis and health risk assessment. For example, if the user is likely to continue having difficulty falling asleep, it may recommend a visit to a medical institution.

[0634] 4. The device will ask, "Tell me more about this. How long has it been going on?"

[0635] 5. If the user answers "about a week ago," the server updates the risk assessment and instructs the appropriate response.

[0636] Example 2: Regular follow-up

[0637] 1. The server schedules periodic checks of the user.

[0638] 2. The device will periodically follow up by asking, "How is your stomach pain since then?"

[0639] 3. If the user responds "much better," the server analyzes the received data, updates the risk assessment, and determines it to be low risk.

[0640] 4. The device will notify you to "remain careful."

[0641] Example prompt sentence:

[0642] "I haven't been able to sleep at night lately. Do you have any advice?"

[0643] "I'm having persistent stomach pains, what should I do?"

[0644] "I haven't been feeling well lately, maybe something's wrong?"

[0645] This will enable the system to provide timely and appropriate care and support to improve the quality of life of the elderly.

[0646] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0647] Step 1: Initial configuration and user authentication

[0648] When the device is turned on, the camera and microphone automatically start working, and facial and voice recognition is performed.

[0649] Input: User's face image and voice data

[0650] Data processing: Analyze facial image and audio data using appropriate recognition algorithms (e.g., Google's Facial Recognition Engine, Amazon's AWS Rekognition).

[0651] Output: Authentication result (success / failure) and user profile

[0652] The server receives the authentication data sent from the device and compares it with the database. If the authentication result is successful, the individual elderly person's profile is loaded and notified to the device.

[0653] Step 2: Starting everyday conversations

[0654] Once authentication is complete, the device greets you with, "Good morning, what's on your agenda today?"

[0655] Input: User response (e.g., "I don't have any plans today, but I have a stomach ache.")

[0656] Data processing: User responses are collected as voice data and converted into text data through voice recognition.

[0657] Output: Text data (user's speech)

[0658] The terminal converts the collected voice data into text data in real time and sends it to the server.

[0659] Step 3: Initial analysis

[0660] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[0661] Input: Text data (user speech)

[0662] Data processing: Natural language processing (e.g., Python's NLTK, Microsoft Azure's Text Analytics API) is used to extract keywords and assess emotions and health status.

[0663] Output: Initial assessment result (e.g., health risk based on the information "I have a stomachache")

[0664] The server determines whether there is anything abnormal in the user's health condition based on the user's comments.

[0665] Step 4: Follow-up questions

[0666] If the server detects an abnormality, it generates an additional question and sends it to the terminal.

[0667] Input: Initial health risk assessment results

[0668] Data processing: Applying algorithms to generate questions to gather more information from the user.

[0669] Output: Follow-up question (e.g., "How long has this been going on?")

[0670] The terminal then asks the generated follow-up questions to the user.

[0671] Step 5: Detailed analysis and risk assessment

[0672] The user answers the follow-up questions, which are received by the server and analyzed again in detail.

[0673] Input: User's detailed response (e.g., "It's been going on for about two days now, nothing else.")

[0674] Data processing: Using deep learning models (e.g., TensorFlow, PyTorch) to update risk assessments.

[0675] Output: Updated health risk assessment

[0676] The server reassess the health risk based on the user's detailed answers.

[0677] Step 6: Take Action

[0678] If the server determines that the risk is high, it sends a warning to nearby medical institutions and care managers.

[0679] Input: Updated health risk assessment

[0680] Data Processing: Contact appropriate external agencies based on pre-set protocols.

[0681] Output: Automatic alerts and delivery service instructions

[0682] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0683] The user receives this notification and takes the necessary action.

[0684] (Application example 1)

[0685] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0686] To improve the quality of life of the elderly, it is necessary to monitor their emotions and health status on a daily basis and provide care and support at the appropriate time. However, existing systems lack real-time monitoring to detect abnormalities, and even minor abnormalities may be overlooked. In addition, effective support may be delayed due to a lack of follow-up questions and appropriate measures after abnormalities are detected. As a result, there is a risk that elderly people will not receive appropriate care when they need it, and their quality of life will decline.

[0687] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0688] In this invention, the server includes means for analyzing emotions and conditions, means for determining the timing of necessary care and support, means for linking information with external services, means for notifying the elderly person of the linking results, means for monitoring changes in facial expressions and voice in real time, means for detecting abnormalities and presenting follow-up questions when an abnormality is detected, and means for analyzing answers to the follow-up questions and taking appropriate measures as necessary. This makes it possible to monitor the elderly person's condition in real time and take measures for care and support in a timely manner.

[0689] 1. "Elderly" refers to individuals over a certain age, often in retirement, who require special care and assistance in their daily lives.

[0690] 2. "Artificial intelligence communicator" refers to a device that uses AI technology to have a dialogue function and analyzes the emotions and health status of users (elderly people) through communication with them.

[0691] 3. "Means for analyzing emotions and states" refers to the process of assessing and analyzing the emotional and physical states of older adults based on input information such as voice data and text data.

[0692] 4. "Means for determining the timing of care or support" refers to algorithms or technologies that, based on the results of analysis, determine the timing and circumstances under which an elderly person will need care or support.

[0693] 5. "Means for sharing information with external services" refers to the process of sharing information with external institutions such as nursing care services and medical institutions, and coordinating the necessary care and support.

[0694] 6. "Means for notifying the elderly of the results of collaboration" refers to mechanisms and technologies for communicating the results of information collaboration with external services to the elderly in an easy-to-understand manner.

[0695] 7. "Means for monitoring changes in facial expressions and voice in real time" refers to the process of instantly observing changes in an elderly person's facial expressions and tone of voice through cameras and microphones and collecting data.

[0696] 8. "Means for detecting anomalies" refers to systems or technologies that analyze collected data and identify abnormalities that deviate from normal conditions.

[0697] 9. "Means for presenting follow-up questions" refers to a function that automatically generates additional questions to ask the elderly person when an abnormality is detected.

[0698] 10. "Means for analyzing responses to follow-up questions" refers to the process of analyzing responses from older people, extracting further information, and assessing their condition.

[0699] 11. "Means of taking appropriate measures" refers to the ability to take action to provide appropriate care and support to older people based on the results of the analysis.

[0700] This invention is a system that enables elderly people to communicate using a communicator equipped with artificial intelligence, analyzes their emotions and health status, and provides appropriate care and support. This system is implemented using the following hardware and software.

[0701] Hardware used

[0702] 1. Camera: A device for capturing facial expressions of elderly people in real time, used for facial recognition and expression analysis.

[0703] 2. Microphone: A device for obtaining voice input. It captures what the elderly person is saying and uses it to analyze their emotions and health status.

[0704] 3. Server: A central data processing unit for data processing and analysis. It is responsible for data management, analysis, and linking with external services.

[0705] 4. Communicator: A device for everyday communication for the elderly. Built-in AI enables natural conversation.

[0706] Software used

[0707] 1. OpenCV: A library for processing video data acquired from a camera and performing facial recognition and facial expression analysis.

[0708] 2. SpeechRecognition: A library for converting voice data into text data. It provides speech recognition functionality.

[0709] 3. TextBlob: A natural language processing library for parsing text data and performing sentiment assessment.

[0710] Data processing and calculation

[0711] 1. Real-time monitoring of facial expressions: Video data captured by the camera is processed in real time using OpenCV to perform facial recognition. Emotion analysis is performed from the recognized facial expressions to detect abnormalities.

[0712] 2. Real-time voice monitoring and analysis: Voice data acquired through a microphone is converted into text data using SpeechRecognition, and sentiment analysis is performed using TextBlob.

[0713] 3. Anomaly detection and follow-up: If an anomaly is detected as a result of sentiment analysis, the server generates follow-up questions and presents them to the elderly via the communicator. The elderly's answers are analyzed again to determine the necessary measures.

[0714] 4. Collaboration with external services: Based on the assessment results, the server will collaborate with external care services and medical institutions and notify the elderly of the results.

[0715] Specific examples

[0716] As a concrete example, consider the case where an elderly person says to the communicator, "I feel a little tired today." In this case, the microphone captures the voice and sends the data to the server. The server converts the voice data into text data using SpeechRecognition and performs sentiment analysis using TextBlob. If the sentiment analysis result is evaluated as "negative," the server generates a follow-up question, asking, "How long has this been going on?" The answer to this question is then reanalyzed, and appropriate measures are taken based on the results.

[0717] An example of a prompt is, "If a worker says, 'I'm feeling a bit tired today,' please write a short code that automatically analyzes their emotions and sends an alert based on the results." Based on this prompt, the system can automatically generate a corresponding code, enabling prompt and appropriate care for the elderly.

[0718] With the above configuration, the present invention makes it possible to monitor the emotions and health status of elderly people in real time and provide care and support at appropriate times.

[0719] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0720] Step 1: User authentication and configuration

[0721] When a user turns on the communicator, the device boots up and the built-in camera and microphone start working automatically. The photographed facial recognition data and recorded voice recognition data are sent to the server for matching. If the authentication is successful, the profile of each elderly person is loaded. Input: Facial recognition data and voice recognition data. Data processing: Data matching. Output: Authentication results and loading of user profile.

[0722] Step 2: Starting everyday conversations

[0723] After authentication is complete, the device speaks to the user, saying, "Good morning. What do you have planned for today?" If the user (elderly person) replies, "I don't have any plans for today, but I have a stomachache," the voice data is sent to the server in real time. Input: User's speech. Data processing: Voice data converted to text. Output: Text data.

[0724] Step 3: Sentiment Analysis

[0725] The server analyzes the received text data and performs sentiment analysis and an initial assessment of health risks. For example, from the information "I have a stomachache," the server may initially determine that there is something wrong with the user's health. Input: User's spoken text data. Data processing: Sentiment analysis and health risk assessment. Output: Initial health assessment results.

[0726] Step 4: Follow-up questions

[0727] If the server detects an abnormality, it generates follow-up questions to gather more information and provides them to the terminal. The terminal asks the user, "How long has this been going on? Are there any other symptoms?" Input: Initial health assessment results. Data processing: Generation of follow-up questions. Output: Follow-up questions.

[0728] Step 5: Analyze follow-up responses

[0729] After receiving additional responses from the user, the device sends the responses to the server. The server analyzes them again and updates the health risk assessment. For example, if the user answers, "It's been going on for about two days now," the server reassess the detailed situation. Input: User's follow-up response. Data processing: Analysis of additional information and risk reassessment. Output: Updated health risk assessment results.

[0730] Step 6: Take Action

[0731] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care services, and notifies the user of this information via their device. It also arranges for food and medicine delivery services as needed. Input: Updated health risk assessment results. Data processing: Linkage processing with external services. Output: Sending a warning to medical institutions and notifying the user.

[0732] The above processing steps realize a system that monitors the emotions and health status of elderly people in real time and provides care and support at the appropriate time.

[0733] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0734] This invention is a system that uses a communicator equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of the elderly. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[0735] System Overview

[0736] The system of the present invention mainly comprises the following components:

[0737] 1. Communicator (terminal) equipped with artificial intelligence

[0738] 2. Emotion Engine

[0739] 3. Backend Processing Server (Server)

[0740] 4. User Interface (User)

[0741] Communicator (terminal) equipped with artificial intelligence

[0742] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[0743] Emotion Engine

[0744] The emotion engine analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses emotion analysis algorithms to determine the user's psychological state and sends that information to a back-end processing server.

[0745] Backend processing server (server)

[0746] The server comprehensively analyzes data sent from the device and emotion engine to evaluate emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[0747] User Interface (User)

[0748] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[0749] Program processing

[0750] The specific program processing is as follows.

[0751] 1. Initial Setup and User Authentication

[0752] When a user turns on the AI-powered doll, the device automatically starts up and uses the camera and microphone to capture the user's face and voice to begin authentication. The authentication data is sent to the server, where the matched results are confirmed. If authentication is successful, a personalized elderly profile is loaded.

[0753] 2. Starting everyday conversations

[0754] Once authentication is complete, the device speaks to the user, asking, "Good morning. What are your plans for today?" The user (an elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[0755] 3. Initial Analysis and Sentiment Analysis

[0756] The server and emotion engine work together to analyze the received data. The emotion engine uses a natural language processing algorithm to evaluate the user's emotional state in real time from the voice data and to analyze the user's emotional state from the text data. For example, if the information says "I have a stomachache," the server will initially determine that there is something wrong with the user's health based on the emotion engine's analysis results.

[0757] 4. Follow-up questions

[0758] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[0759] 5. Detailed analysis and risk assessment

[0760] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[0761] 6. Taking Action

[0762] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[0763] Specific examples

[0764] Example 1: Detecting and responding to poor health

[0765] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[0766] 2. The device sends this message to the server.

[0767] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[0768] 4. The device asks, "Tell me more about this. How long has this been going on?"

[0769] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0770] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[0771] Example 2: Regular follow-up

[0772] 1. Schedule the server to periodically check for users.

[0773] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[0774] 3. The user responded, "Much better."

[0775] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[0776] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support. In particular, by combining it with an emotion engine, it will be possible to accurately grasp the user's emotional state and provide appropriate action immediately when necessary.

[0777] The processing flow will be explained below.

[0778] Step 1:

[0779] The user turns on the AI-powered doll. The device automatically starts up and uses the camera and microphone to capture the user's face and voice, then begins authentication.

[0780] Step 2:

[0781] The device sends authentication data to the server, which verifies the received authentication data and loads the user's profile.

[0782] Step 3:

[0783] Once authentication is complete, the device asks the user, "Good morning, what are your plans for today?" The user replies, "I don't have any plans for today, but I have a stomachache."

[0784] Step 4:

[0785] The device sends the user's response as text data to the server, which then analyzes the received text data and performs sentiment analysis and an initial assessment of health risks.

[0786] Step 5:

[0787] The emotion engine analyzes the voice data sent from the server and evaluates the user's emotional state in real time, for example, determining the level of anxiety or pain.

[0788] Step 6:

[0789] Based on the results of the initial assessment and the analysis results of the emotion engine, the server creates follow-up questions and sends instructions to the device. The device asks the user, "How long has this been going on? Are there any other symptoms?"

[0790] Step 7:

[0791] The user responds, "It's been going on for about two days now, and nothing else." The device sends this new information as text data to the server.

[0792] Step 8:

[0793] The server analyzes the new data and reassesses health risks. Using an analytical algorithm, it determines the risk level in detail. At the same time, the emotion engine reassessessments based on the updated voice and text data.

[0794] Step 9:

[0795] If the server determines that a patient is at high risk, it will automatically send a warning to nearby medical institutions and care managers. It can also instruct partner delivery services to provide specific medicines or ingredients.

[0796] Step 10:

[0797] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[0798] Step 11:

[0799] The server continuously monitors the conversation data and the analysis results of the emotion engine, and periodically creates a health status report and sends it to the user's family or care manager. The device periodically follows up with the user, asking, "How's your stomach pain going?" to provide ongoing support.

[0800] Example 2

[0801] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0802] Today's elderly often experience a decline in their quality of life due to physical and psychological problems associated with aging. Therefore, it is necessary to create an environment that supports the elderly in their daily lives and allows them to receive medical and nursing care services in a timely manner. However, current systems make it difficult to analyze the emotions and health status of the elderly in real time and provide appropriate responses immediately. Furthermore, there is a lack of means to detect health risks faced by the elderly early and to quickly share information.

[0803] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for enabling the elderly person to interact with a communication device; a means for analyzing the elderly person's emotions and health status based on the interaction; a means for determining the timing of necessary care and support based on the analysis results; a means for linking information with external organizations such as nursing care services and medical institutions based on the determination; a means for notifying the elderly person of the linking results; a means for recognizing the user's face and voice and reading an individual profile; a means for asking follow-up questions and collecting detailed data; a means for evaluating health risks based on the received data and updating information; and a means for sending an alert to a medical institution and instructing the provision of necessary medicines and ingredients if an abnormality is detected. This makes it possible to analyze the elderly person's emotions and health status in real time and provide appropriate care and support promptly.

[0804] "Communication devices" refer to devices used by elderly people to communicate, including devices equipped with voice recognition and cameras.

[0805] "Dialogue" refers to the verbal exchange between the elderly person and the communication device, and takes place in the form of voice or text data.

[0806] "Analyzing emotions and health status" refers to analyzing the psychological and physical state of elderly people based on their voice and text data.

[0807] "Analysis results" refers to information obtained through analytical means regarding emotions and health status.

[0808] "Determining the timing of care and support" refers to determining the appropriate time to provide the necessary care and support to the elderly based on the analysis results.

[0809] "External agencies" refers to external organizations and services involved in supporting the elderly, such as nursing care services and medical institutions.

[0810] "Collaboration" refers to sharing information with external organizations and cooperating with each other.

[0811] "Collaboration results" refers to the results and feedback obtained after sharing information with external organizations.

[0812] "Recognizing the user's face and voice" refers to the communication device capturing the facial image and voice of the elderly person and identifying the individual.

[0813] "Loading an individual profile" refers to retrieving and applying individual data stored for each recognized user.

[0814] A "follow-up question" is a question asked to gather additional information based on the initial dialogue or analysis.

[0815] "Detailed data" refers to more specific information obtained through follow-up questions.

[0816] "Assessing health risks" refers to determining the risks associated with the health status of elderly people based on detailed data and analysis results.

[0817] "Updating information" refers to reflecting new data and analysis results in the records within the system.

[0818] "Sending an alert" refers to sending information to alert medical institutions and related parties when an abnormality is detected.

[0819] "Instructing the provision of medicines and food ingredients" refers to issuing instructions to arrange for the supply of necessary medicines and food ingredients based on the health condition of the elderly person.

[0820] This invention is a system that uses communication devices equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of elderly people. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[0821] Hardware and software used

[0822] Communication device: An interactive AI device for everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user. Through this device, conversations with the elderly are conducted in real time.

[0823] Emotion engine: A software module that analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses an emotion analysis algorithm to determine the user's psychological state.

[0824] Back-end processing server: Comprehensively analyzes data sent from communication devices and the emotion engine to evaluate emotions and health risks. This server uses a health risk assessment algorithm to determine the timing of care and support, and notifies external organizations as necessary.

[0825] Generative AI models: Use specific AI models for natural language processing and speech recognition.

[0826] Specific processing of the program

[0827] 1. The server receives the voice and text data sent from the communication device and analyzes it in cooperation with the emotion engine.

[0828] 2. The emotion engine evaluates the user's emotional state in real time from the voice data and also analyzes the emotional state from the text data, thereby identifying emotions such as "anxiety" or "anger."

[0829] 3. The server evaluates health risks based on the analysis results and determines the timing of necessary care and support. If a high risk is determined, the server shares information with external organizations and notifies the user.

[0830] 4. The communication device asks additional follow-up questions to gather more detailed data, such as "How long has this been going on?" or "Are you experiencing any other symptoms?" to get more detailed information from the user.

[0831] 5. The server re-analyzes the detailed data and updates the health risk assessment, taking necessary actions accordingly.

[0832] Specific examples

[0833] Example 1: Detecting and responding to poor health

[0834] 1. The user talks to the communication device saying, "I haven't been able to sleep at night lately."

[0835] 2. The communication device sends this message to the server.

[0836] 3. The server performs an emotional analysis of the "recent sleep patterns" and a health risk assessment, and if the "difficulty falling asleep" continues, recommends a visit to a medical institution.

[0837] 4. The communication device asks, "Tell me more about this. How long has this been going on?"

[0838] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[0839] 6. The server automatically contacts a nearby medical institution and notifies the user via the communication device, saying, "We're worried, so we've contacted the clinic."

[0840] Example 2: Regular follow-up

[0841] 1. Schedule the server to periodically check for users.

[0842] 2. The communication device will follow up periodically with, "How's the stomach ache going since then?"

[0843] 3. The user responds, "It's much better."

[0844] 4. The server analyzes the received data and determines that the risk has decreased. The communication device sends a message saying, "Please continue to be careful."

[0845] Example prompts for generative AI models

[0846] "Please explain the process by which an elderly person talks to an AI-enabled communication device about their recent health condition, and the emotion engine analyzes that information and provides the necessary assistance."

[0847] The present invention makes it possible to perform analysis using an emotion engine in particular, thereby enabling the user's emotional state to be grasped with high accuracy and appropriate care and support to be provided promptly.

[0848] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0849] Step 1:

[0850] The user turns on the communication device. The communication device uses a camera and microphone to capture the user's face and voice. The input data (facial recognition data and voice data) obtained through this process is sent to the server, which then authenticates the user. If authentication is successful, an individual elderly profile is loaded. Specifically, the camera on the communication device captures facial data and sends voice data to the recognition module. The server then uses facial recognition algorithms and voiceprint analysis algorithms to recognize the user.

[0851] Step 2:

[0852] Once authentication is complete, the communication device speaks to the user, asking, "Good morning. What do you have planned for today?" To which the user responds, "I don't have any plans for today, but I have a stomachache." This response is converted into text data in real time and sent to the server. The input includes the user's voice data, and the output includes the text conversion result of that voice data. Specifically, the voice recognition module of the communication device converts the user's speech into text data, and that text data is sent to the server via the Internet.

[0853] Step 3:

[0854] The server works with the emotion engine to analyze the received text data. The emotion engine evaluates the user's emotional state in real time from the voice data and similarly analyzes the emotional state from the text data. For example, emotions such as "discomfort" or "pain" are extracted from the text "my stomach hurts." Based on the results of this analysis, the server makes an initial judgment that there is something wrong with the user's health. The input data is text data, and the output data is the user's emotional state and health risk assessment. The emotion engine uses a natural language processing algorithm to analyze the text data and classify emotions.

[0855] Step 4:

[0856] If the server detects an abnormality, it instructs the communication device to ask follow-up questions to gather additional information. The communication device asks the user, "How long has this been going on? Are there any other symptoms?" to collect detailed information. If the user answers, "It's been going on for about two days now," the answer is again converted into text data and sent to the server. The input contains the user's answer to the follow-up question, and the output contains detailed text data. Specifically, the communication device performs voice recognition again and sends the follow-up question and its answer to the server as text data.

[0857] Step 5:

[0858] The server reassesses the health risk based on the detailed text data. For example, based on the response "It's been going on for about two days, but nothing else in particular," it uses an analysis algorithm to reassess the risk level. The input data is the response to the follow-up question, and the output data is the updated health risk assessment. The server uses a high-performance analysis module to analyze the obtained detailed data and reassess the health risk.

[0859] Step 6:

[0860] If the risk is determined to be high, the server sends an alert to nearby medical institutions and care managers, instructing them to provide the necessary medications and food ingredients. This information is transmitted to the user via the communication device, which sends a message saying, "I'm worried, so I've contacted a nearby clinic. Please rest." The input data is the updated health risk assessment and detailed text data, and the output data is the alert notification to the medical institution and the notification message to the user. Specifically, the server uses a communication module to send the alert data to the medical institution, and notifies the user via the speaker and display of the communication device.

[0861] keyword

[0862] Generative AI model, prompt sentence

[0863] (Application example 2)

[0864] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0865] In order to improve the quality of life of the elderly, not only appropriate nursing care and medical support but also daily health management is important. However, it is not easy for the elderly themselves to understand changes in their own health condition and emotions and take appropriate action. Furthermore, diet has a significant impact on health, so it is important to provide the elderly with appropriate meals, but it is difficult for them to choose the meals that are optimal for their health condition. To solve these issues, a system is needed that is deeply involved in the daily lives of the elderly and can make appropriate meal suggestions and provide them in real time based on their health condition.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0867] In this invention, the server includes a means for enabling the elderly person to converse with a communicator equipped with artificial intelligence, a means for analyzing emotions and conditions based on the converse, and a means for determining the timing of necessary care and support based on the analysis results. This makes it possible to provide appropriate care and support, as well as to suggest and provide appropriate meals based on emotions and health conditions.

[0868] "Elderly people" refers to people of an age who require care and assistance in their daily lives.

[0869] An "artificial intelligence communicator" refers to a computer system that converses with elderly people and collects and analyzes voice and text data.

[0870] "Dialogue" refers to voice or text communication between an elderly person and an AI-enabled communicator.

[0871] "Means for analyzing emotions and conditions" refers to algorithms and systems that evaluate the emotional and health conditions of elderly people based on acquired voice and text data.

[0872] "Means for determining the timing of necessary care and support" refers to algorithms and systems that determine when care and support is needed based on the analysis results.

[0873] "Means for sharing information with external services" refers to communication functions for sharing analysis and assessment results with external nursing care services and medical institutions.

[0874] "Means for notifying the elderly of the results of collaboration" refers to a system that communicates the results of collaboration with external services to the elderly via voice or text message.

[0875] "Means for making appropriate meal suggestions" refers to algorithms or systems that select and suggest meal menus that are optimal for a person's health based on the results of analysis and assessment.

[0876] "Means for automated delivery" refers to a system that automatically orders the necessary ingredients and food based on a proposed meal menu and delivers them to the elderly person's home.

[0877] MODE FOR CARRYING OUT THE INVENTION

[0878] To improve the quality of life of the elderly, this invention provides a system in which the elderly interact with a communicator equipped with artificial intelligence, and the emotions and health status obtained through the interaction are analyzed to determine the timing of necessary care and support. This system automates the entire process from suggesting appropriate meals based on the analysis results to delivering those meals.

[0879] System configuration

[0880] The system mainly consists of the following components:

[0881] 1. Artificial Intelligence Communicator

[0882] 2. Emotion Engine

[0883] 3. Backend Processing Server

[0884] 4. User Interface

[0885] 5. External service integration system

[0886] Artificial Intelligence-Powered Communicator

[0887] The Communicator is a device for communicating with the elderly. It has a built-in camera and microphone, and can perform facial recognition and voice recognition of the elderly. For facial recognition, it uses Google Face API or Amazon Rekognition, and for voice recognition, it uses Google Cloud Speech-to-Text.

[0888] Emotion Engine

[0889] The emotion engine analyzes the acquired voice and text data and evaluates the emotional state of the elderly person in real time. For example, it can use IBM Watson's Tone Analyzer.

[0890] Backend Processing Server

[0891] The server evaluates health risks based on the data sent from the emotion engine and makes appropriate dietary recommendations. It also determines the timing of necessary care and support and shares the results with external services. The server uses cloud services such as AWS EC2.

[0892] User Interface

[0893] Through the user interface, elderly people can communicate their condition to the communicator through natural dialogue, and notifications and suggestions from the server are also displayed through this interface.

[0894] External service integration system

[0895] The external service integration system suggests appropriate meals based on the analysis results and automatically handles the ordering and delivery procedures for delivering those meals.

[0896] How the system works

[0897] 1. Initial Setup and User Authentication

[0898] The server authenticates the elderly person's face and voice and sends the authentication data to the server, which then loads an individual elderly person profile.

[0899] 2. Starting everyday conversations

[0900] The communicator talks to the elderly and asks them questions about their everyday health, for example, a conversation about "stomach pain."

[0901] 3. Initial Analysis and Sentiment Analysis

[0902] The emotion engine analyzes the voice and text data obtained through the dialogue and sends the results to the server.

[0903] 4. Follow-up questions and detailed analysis

[0904] The server will ask follow-up questions to gather additional information and conduct a detailed health risk assessment.

[0905] 5. Meal suggestions and delivery arrangements

[0906] We suggest an appropriate meal menu based on the elderly person's health condition, and arrange for the ingredients to be ordered and delivered by a delivery service based on that menu.

[0907] Specific examples

[0908] Example 1: User complaining of indigestion

[0909] 1. The user tells the communicator, "I have a stomachache."

[0910] 2. The communicator sends this utterance to the server.

[0911] 3. The emotion engine determines the emotion of "painful" and sends the result to the server.

[0912] 4. The server suggests to the user a menu of easily digestible foods (e.g., porridge or soup).

[0913] 5. Order food from a delivery service and arrange for it to be delivered to the senior's home.

[0914] Prompt Sentence Examples

[0915] You are an AI-powered communicator. A 70-year-old woman has recently been complaining of stomach aches and would like recommendations for foods that are easy to digest. Use your emotion engine to analyze this information and suggest appropriate menu items.

[0916] By combining these detailed procedures and techniques, the present invention can significantly improve the quality of life for the elderly.

[0917] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0918] Step 1:

[0919] The user speaks to an AI-powered communicator, saying, "My stomach hurts."

[0920] How it works: The device's microphone captures audio data and converts it into text data in real time.

[0921] Input: User's voice

[0922] Output: Text data

[0923] Step 2:

[0924] The terminal transmits the text data to the server.

[0925] How it works: The communicator sends the generated text data to the server via the cloud.

[0926] Input: Text data

[0927] Output: Data received by the server

[0928] Step 3:

[0929] The server sends the text data to the emotion engine for emotion analysis.

[0930] How it works: The server sends the text data to an analysis engine to assess the emotional state, for example using IBM Watson's Tone Analyzer.

[0931] Input: Text data

[0932] Output: Emotion analysis results

[0933] Step 4:

[0934] The emotion engine sends the analysis results back to the server.

[0935] How it works: The emotion engine identifies emotions such as "sad" and sends the results to the server.

[0936] Input: Text data

[0937] Output: Emotion analysis results

[0938] Step 5:

[0939] The server assesses the health risk and generates appropriate follow-up questions.

[0940] How it works: The server runs an algorithm to assess health risk based on the sentiment analysis results. If a risk is determined, it generates a follow-up question (e.g., "How long has this been going on?").

[0941] Input: Sentiment analysis results

[0942] Output: Follow-up question

[0943] Step 6:

[0944] The terminal presents the user with follow-up questions.

[0945] How it works: The device speaks the generated follow-up question to the user.

[0946] Input: Follow-up question

[0947] Output: Audio output

[0948] Step 7:

[0949] The user answers the follow-up question.

[0950] How it works: The user answers the device, "It's been going on for about two days," and the device again captures the voice data and converts it into text data.

[0951] Input: Voice response

[0952] Output: Text data

[0953] Step 8:

[0954] The device sends the text data to a server for detailed analysis.

[0955] How it works: The device sends the text data to a server via the cloud, where it begins detailed analysis. The server then uses an analytical algorithm to reassess the health risk.

[0956] Input: Text data

[0957] Output: Detailed analysis results

[0958] Step 9:

[0959] The server generates appropriate meal suggestions.

[0960] How it works: Based on the results of the detailed analysis, the server generates an appropriate meal menu and creates data to order from a delivery service.

[0961] Input: Detailed analysis results

[0962] Output: Meal suggestions and ordering data

[0963] Step 10:

[0964] The delivery service will process the delivery of the ingredients.

[0965] How it works: The server sends the order data to the delivery service and automatically processes the delivery of the ingredients.

[0966] Input: Order data

[0967] Output: Delivery arrangement

[0968] Step 11:

[0969] The terminal notifies the user of the delivery information.

[0970] How it works: The device receives a notification from the server and notifies the user of the order details and delivery time via voice or text.

[0971] Input:Shipping information

[0972] Output: Voice or text notification

[0973] This will enable elderly people to receive appropriate care at home while smoothly obtaining the optimal diet for their health.

[0974] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0975] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0976] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0977] [Third embodiment]

[0978] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0979] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0980] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0981] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0982] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0983] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0984] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0985] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0986] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0987] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0988] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0989] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0990] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of the elderly. Specific embodiments of the system are described below.

[0991] System Overview

[0992] The system of the present invention mainly comprises the following components:

[0993] 1. Communicator (terminal) equipped with artificial intelligence

[0994] 2. Backend Processing Server (Server)

[0995] 3. User Interface (User)

[0996] Communicator (terminal) equipped with artificial intelligence

[0997] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[0998] Backend processing server (server)

[0999] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1000] User Interface (User)

[1001] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1002] Program processing

[1003] The specific program processing is as follows.

[1004] 1. Initial Setup and User Authentication

[1005] The user turns on the AI-enabled doll and the device starts up. The camera and microphone automatically activate at startup to perform facial and voice recognition. The authentication data is sent to the server, where the matching results are confirmed. If authentication is successful, an individual elderly person profile is loaded.

[1006] 2. Starting everyday conversations

[1007] Once authentication is complete, the device asks, "Good morning, what are your plans for today?" The user (elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[1008] 3. Initial analysis

[1009] The server analyzes the received data and performs sentiment analysis and an initial assessment of health risks. For example, if the user says "I have a stomachache," the server may initially determine that there is something wrong with their health.

[1010] 4. Follow-up questions

[1011] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[1012] 5. Detailed analysis and risk assessment

[1013] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[1014] 6. Taking Action

[1015] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[1016] Specific examples

[1017] Example 1: Detecting and responding to poor health

[1018] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[1019] 2. The device sends this message to the server.

[1020] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[1021] 4. The device asks, "Tell me more about this. How long has this been going on?"

[1022] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1023] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[1024] Example 2: Regular follow-up

[1025] 1. Schedule the server to periodically check for users.

[1026] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[1027] 3. The user responded, "Much better."

[1028] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[1029] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support.

[1030] The processing flow will be explained below.

[1031] Step 1:

[1032] The user turns on the AI-equipped doll.

[1033] The device will automatically start up and use the camera and microphone to capture the user's face and voice to begin authentication.

[1034] Step 2:

[1035] The terminal sends the authentication data to the server.

[1036] The server verifies the received authentication data and loads the user's profile.

[1037] Step 3:

[1038] Once authentication is complete, the device will ask the user, "Good morning, what are your plans for today?"

[1039] The user replies, "I don't have any plans today, but I have a stomach ache."

[1040] Step 4:

[1041] The terminal transmits the user's response as text data to the server.

[1042] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[1043] Step 5:

[1044] Based on the results of the initial evaluation, the server creates follow-up questions and sends instructions to the terminal.

[1045] The device asks the user, "How long has this been going on? Are you experiencing any other symptoms?"

[1046] Step 6:

[1047] The user replies, "It's been going on for about two days now, nothing else."

[1048] The terminal sends this new information to the server as text data.

[1049] Step 7:

[1050] The server analyzes the new data and reassess the health risk.

[1051] If the analysis results indicate a high risk, appropriate actions will be suggested.

[1052] Step 8:

[1053] The server automatically sends alerts to nearby medical institutions and care managers to arrange for the necessary support.

[1054] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1055] Step 9:

[1056] The server continuously monitors the conversation data, periodically creates health status reports, and sends them to the user's family or care manager.

[1057] The device continues to provide support by periodically following up with the user with, "How's your stomach pain going since then?"

[1058] Example 1

[1059] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1060] It is difficult to provide timely and appropriate care and support to improve the quality of life of the elderly. Furthermore, there is a lack of a system that can accurately grasp the emotions and health status of the elderly through everyday conversations and respond when necessary. Furthermore, there is a need for effective support, including long-term assessment of health risks and necessary follow-up.

[1061] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1062] In this invention, the server includes: means for enabling the elderly person to interact with an AI-equipped communicator; means for analyzing the elderly person's emotions and health status based on the interaction; means for determining the timing of necessary care and support based on the analysis results; means for linking information with external organizations based on the determination; means for notifying the elderly person of the results of the linking; means for generating follow-up questions and collecting additional information; means for using an algorithm to perform long-term health risk assessment; and means including a function for automatically sending alerts to nearby medical institutions. This enables the elderly person's health status to be continuously monitored and nursing care and medical services to be provided at the appropriate time.

[1063] "Elderly" refers to people who are older and often require care and assistance in their daily lives.

[1064] An "artificial intelligence-enabled communicator" refers to a device equipped with conversational artificial intelligence that can hold natural conversations with the elderly.

[1065] "Dialogue" refers to communication between two people, and in this system refers to voice or text exchanges between the elderly person and an AI-powered communicator.

[1066] "Analysis of emotions and health status" means analyzing the emotions and health-related information expressed by elderly people through conversations using voice data and text data.

[1067] "Determining the timing of care and support" refers to determining the timing of the need for care and support based on the analysis of emotions and health status.

[1068] "Sharing information with external organizations" means sharing information with external organizations such as medical institutions and care service providers when the timing of necessary care or support is determined.

[1069] "Notifying the results of collaboration" refers to notifying the elderly of the results of information collaboration with external organizations.

[1070] "Generating follow-up questions" refers to the AI-enabled communicator asking additional questions to the elderly person to gather additional information based on the results of the initial analysis.

[1071] "Health risk assessment algorithm" refers to a program that uses mathematical or statistical methods to assess health risks based on collected data.

[1072] The "automatic warning sending function" refers to a function that automatically sends a warning to pre-set medical institutions or care managers when the health condition of an elderly person is judged to be at risk.

[1073] A "deep learning model" is a type of artificial intelligence technology that uses neural networks to learn patterns from large amounts of data and is applied to analyzing emotions and health conditions.

[1074] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of elderly people. Specific embodiments of the system are described below.

[1075] System configuration

[1076] The system consists of the following main components:

[1077] 1. Communicator (terminal) equipped with artificial intelligence

[1078] 2. Backend Processing Server (Server)

[1079] 3. User Interface (User)

[1080] Hardware and Software Configuration

[1081] Communicator (terminal) equipped with artificial intelligence

[1082] This device is equipped with conversational AI and is designed to engage in everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition for the user. The device uses natural language processing and voice recognition technologies to process dialogue with the user in real time.

[1083] Hardware used:

[1084] Camera module

[1085] microphone

[1086] speaker

[1087] Software used:

[1088] Conversational AI Platform

[1089] Voice recognition software (e.g., Google Cloud Speech-to-Text)

[1090] Natural language processing engines (e.g., Python's NLTK, Microsoft Azure's Text Analytics API)

[1091] Backend processing server (server)

[1092] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1093] Hardware used:

[1094] High-Performance Server

[1095] Software used:

[1096] Database Management Systems

[1097] Sentiment analysis algorithms (e.g., TensorFlow, PyTorch)

[1098] Health Risk Assessment Algorithm

[1099] User Interface (User)

[1100] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1101] Specific examples and prompts

[1102] Example 1: Detecting and responding to poor health

[1103] 1. The user consults an AI-enabled communicator, saying, "I haven't been able to sleep at night lately."

[1104] 2. The device collects this speech as audio data, converts it into text data, and sends it to the server.

[1105] 3. The server analyzes the user's recent sleep patterns and performs emotional analysis and health risk assessment. For example, if the user is likely to continue having difficulty falling asleep, it may recommend a visit to a medical institution.

[1106] 4. The device will ask, "Tell me more about this. How long has it been going on?"

[1107] 5. If the user answers "about a week ago," the server updates the risk assessment and instructs the appropriate response.

[1108] Example 2: Regular follow-up

[1109] 1. The server schedules periodic checks of the user.

[1110] 2. The device will periodically follow up by asking, "How is your stomach pain since then?"

[1111] 3. If the user responds "much better," the server analyzes the received data, updates the risk assessment, and determines it to be low risk.

[1112] 4. The device will notify you to "remain careful."

[1113] Example prompt sentence:

[1114] "I haven't been able to sleep at night lately. Do you have any advice?"

[1115] "I'm having persistent stomach pains, what should I do?"

[1116] "I haven't been feeling well lately, maybe something's wrong?"

[1117] This will enable the system to provide timely and appropriate care and support to improve the quality of life of the elderly.

[1118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1119] Step 1: Initial configuration and user authentication

[1120] When the device is turned on, the camera and microphone automatically start working, and facial and voice recognition is performed.

[1121] Input: User's face image and voice data

[1122] Data processing: Analyze facial image and audio data using appropriate recognition algorithms (e.g., Google's Facial Recognition Engine, Amazon's AWS Rekognition).

[1123] Output: Authentication result (success / failure) and user profile

[1124] The server receives the authentication data sent from the device and compares it with the database. If the authentication result is successful, the individual elderly person's profile is loaded and notified to the device.

[1125] Step 2: Starting everyday conversations

[1126] Once authentication is complete, the device greets you with, "Good morning, what's on your agenda today?"

[1127] Input: User response (e.g., "I don't have any plans today, but I have a stomach ache.")

[1128] Data processing: User responses are collected as voice data and converted into text data through voice recognition.

[1129] Output: Text data (user's speech)

[1130] The terminal converts the collected voice data into text data in real time and sends it to the server.

[1131] Step 3: Initial analysis

[1132] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[1133] Input: Text data (user speech)

[1134] Data processing: Natural language processing (e.g., Python's NLTK, Microsoft Azure's Text Analytics API) is used to extract keywords and assess emotions and health status.

[1135] Output: Initial assessment result (e.g., health risk based on the information "I have a stomachache")

[1136] The server determines whether there is anything abnormal in the user's health condition based on the user's comments.

[1137] Step 4: Follow-up questions

[1138] If the server detects an abnormality, it generates an additional question and sends it to the terminal.

[1139] Input: Initial health risk assessment results

[1140] Data processing: Applying algorithms to generate questions to gather more information from the user.

[1141] Output: Follow-up question (e.g., "How long has this been going on?")

[1142] The terminal then asks the generated follow-up questions to the user.

[1143] Step 5: Detailed analysis and risk assessment

[1144] The user answers the follow-up questions, which are received by the server and analyzed again in detail.

[1145] Input: User's detailed response (e.g., "It's been going on for about two days now, nothing else.")

[1146] Data processing: Using deep learning models (e.g., TensorFlow, PyTorch) to update risk assessments.

[1147] Output: Updated health risk assessment

[1148] The server reassess the health risk based on the user's detailed answers.

[1149] Step 6: Take Action

[1150] If the server determines that the risk is high, it sends a warning to nearby medical institutions and care managers.

[1151] Input: Updated health risk assessment

[1152] Data Processing: Contact appropriate external agencies based on pre-set protocols.

[1153] Output: Automatic alerts and delivery service instructions

[1154] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1155] The user receives this notification and takes the necessary action.

[1156] (Application example 1)

[1157] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1158] To improve the quality of life of the elderly, it is necessary to monitor their emotions and health status on a daily basis and provide care and support at the appropriate time. However, existing systems lack real-time monitoring to detect abnormalities, and even minor abnormalities may be overlooked. In addition, effective support may be delayed due to a lack of follow-up questions and appropriate measures after abnormalities are detected. As a result, there is a risk that elderly people will not receive appropriate care when they need it, and their quality of life will decline.

[1159] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1160] In this invention, the server includes means for analyzing emotions and conditions, means for determining the timing of necessary care and support, means for linking information with external services, means for notifying the elderly person of the linking results, means for monitoring changes in facial expressions and voice in real time, means for detecting abnormalities and presenting follow-up questions when an abnormality is detected, and means for analyzing answers to the follow-up questions and taking appropriate measures as necessary. This makes it possible to monitor the elderly person's condition in real time and take measures for care and support in a timely manner.

[1161] 1. "Elderly" refers to individuals over a certain age, often in retirement, who require special care and assistance in their daily lives.

[1162] 2. "Artificial intelligence communicator" refers to a device that uses AI technology to have a dialogue function and analyzes the emotions and health status of users (elderly people) through communication with them.

[1163] 3. "Means for analyzing emotions and states" refers to the process of assessing and analyzing the emotional and physical states of older adults based on input information such as voice data and text data.

[1164] 4. "Means for determining the timing of care or support" refers to algorithms or technologies that, based on the results of analysis, determine the timing and circumstances under which an elderly person will need care or support.

[1165] 5. "Means for sharing information with external services" refers to the process of sharing information with external institutions such as nursing care services and medical institutions, and coordinating the necessary care and support.

[1166] 6. "Means for notifying the elderly of the results of collaboration" refers to mechanisms and technologies for communicating the results of information collaboration with external services to the elderly in an easy-to-understand manner.

[1167] 7. "Means for monitoring changes in facial expressions and voice in real time" refers to the process of instantly observing changes in an elderly person's facial expressions and tone of voice through cameras and microphones and collecting data.

[1168] 8. "Means for detecting anomalies" refers to systems or technologies that analyze collected data and identify abnormalities that deviate from normal conditions.

[1169] 9. "Means for presenting follow-up questions" refers to a function that automatically generates additional questions to ask the elderly person when an abnormality is detected.

[1170] 10. "Means for analyzing responses to follow-up questions" refers to the process of analyzing responses from older people, extracting further information, and assessing their condition.

[1171] 11. "Means of taking appropriate measures" refers to the ability to take action to provide appropriate care and support to older people based on the results of the analysis.

[1172] This invention is a system that enables elderly people to communicate using a communicator equipped with artificial intelligence, analyzes their emotions and health status, and provides appropriate care and support. This system is implemented using the following hardware and software.

[1173] Hardware used

[1174] 1. Camera: A device for capturing facial expressions of elderly people in real time, used for facial recognition and expression analysis.

[1175] 2. Microphone: A device for obtaining voice input. It captures what the elderly person is saying and uses it to analyze their emotions and health status.

[1176] 3. Server: A central data processing unit for data processing and analysis. It is responsible for data management, analysis, and linking with external services.

[1177] 4. Communicator: A device for everyday communication for the elderly. Built-in AI enables natural conversation.

[1178] Software used

[1179] 1. OpenCV: A library for processing video data acquired from a camera and performing facial recognition and facial expression analysis.

[1180] 2. SpeechRecognition: A library for converting voice data into text data. It provides speech recognition functionality.

[1181] 3. TextBlob: A natural language processing library for parsing text data and performing sentiment assessment.

[1182] Data processing and calculation

[1183] 1. Real-time monitoring of facial expressions: Video data captured by the camera is processed in real time using OpenCV to perform facial recognition. Emotion analysis is performed from the recognized facial expressions to detect abnormalities.

[1184] 2. Real-time voice monitoring and analysis: Voice data acquired through a microphone is converted into text data using SpeechRecognition, and sentiment analysis is performed using TextBlob.

[1185] 3. Anomaly detection and follow-up: If an anomaly is detected as a result of sentiment analysis, the server generates follow-up questions and presents them to the elderly via the communicator. The elderly's answers are analyzed again to determine the necessary measures.

[1186] 4. Collaboration with external services: Based on the assessment results, the server will collaborate with external care services and medical institutions and notify the elderly of the results.

[1187] Specific examples

[1188] As a concrete example, consider the case where an elderly person says to the communicator, "I feel a little tired today." In this case, the microphone captures the voice and sends the data to the server. The server converts the voice data into text data using SpeechRecognition and performs sentiment analysis using TextBlob. If the sentiment analysis result is evaluated as "negative," the server generates a follow-up question, asking, "How long has this been going on?" The answer to this question is then reanalyzed, and appropriate measures are taken based on the results.

[1189] An example of a prompt is, "If a worker says, 'I'm feeling a bit tired today,' please write a short code that automatically analyzes their emotions and sends an alert based on the results." Based on this prompt, the system can automatically generate a corresponding code, enabling prompt and appropriate care for the elderly.

[1190] With the above configuration, the present invention makes it possible to monitor the emotions and health status of elderly people in real time and provide care and support at appropriate times.

[1191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1192] Step 1: User authentication and configuration

[1193] When a user turns on the communicator, the device boots up and the built-in camera and microphone start working automatically. The photographed facial recognition data and recorded voice recognition data are sent to the server for matching. If the authentication is successful, the profile of each elderly person is loaded. Input: Facial recognition data and voice recognition data. Data processing: Data matching. Output: Authentication results and loading of user profile.

[1194] Step 2: Starting everyday conversations

[1195] After authentication is complete, the device speaks to the user, saying, "Good morning. What do you have planned for today?" If the user (elderly person) replies, "I don't have any plans for today, but I have a stomachache," the voice data is sent to the server in real time. Input: User's speech. Data processing: Voice data converted to text. Output: Text data.

[1196] Step 3: Sentiment Analysis

[1197] The server analyzes the received text data and performs sentiment analysis and an initial assessment of health risks. For example, from the information "I have a stomachache," the server may initially determine that there is something wrong with the user's health. Input: User's spoken text data. Data processing: Sentiment analysis and health risk assessment. Output: Initial health assessment results.

[1198] Step 4: Follow-up questions

[1199] If the server detects an abnormality, it generates follow-up questions to gather more information and provides them to the terminal. The terminal asks the user, "How long has this been going on? Are there any other symptoms?" Input: Initial health assessment results. Data processing: Generation of follow-up questions. Output: Follow-up questions.

[1200] Step 5: Analyze follow-up responses

[1201] After receiving additional responses from the user, the device sends the responses to the server. The server analyzes them again and updates the health risk assessment. For example, if the user answers, "It's been going on for about two days now," the server reassess the detailed situation. Input: User's follow-up response. Data processing: Analysis of additional information and risk reassessment. Output: Updated health risk assessment results.

[1202] Step 6: Take Action

[1203] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care services, and notifies the user of this information via their device. It also arranges for food and medicine delivery services as needed. Input: Updated health risk assessment results. Data processing: Linkage processing with external services. Output: Sending a warning to medical institutions and notifying the user.

[1204] The above processing steps realize a system that monitors the emotions and health status of elderly people in real time and provides care and support at the appropriate time.

[1205] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1206] This invention is a system that uses a communicator equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of the elderly. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[1207] System Overview

[1208] The system of the present invention mainly comprises the following components:

[1209] 1. Communicator (terminal) equipped with artificial intelligence

[1210] 2. Emotion Engine

[1211] 3. Backend Processing Server (Server)

[1212] 4. User Interface (User)

[1213] Communicator (terminal) equipped with artificial intelligence

[1214] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[1215] Emotion Engine

[1216] The emotion engine analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses emotion analysis algorithms to determine the user's psychological state and sends that information to a back-end processing server.

[1217] Backend processing server (server)

[1218] The server comprehensively analyzes data sent from the device and emotion engine to evaluate emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1219] User Interface (User)

[1220] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1221] Program processing

[1222] The specific program processing is as follows.

[1223] 1. Initial Setup and User Authentication

[1224] When a user turns on the AI-powered doll, the device automatically starts up and uses the camera and microphone to capture the user's face and voice to begin authentication. The authentication data is sent to the server, where the matched results are confirmed. If authentication is successful, a personalized elderly profile is loaded.

[1225] 2. Starting everyday conversations

[1226] Once authentication is complete, the device speaks to the user, asking, "Good morning. What are your plans for today?" The user (an elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[1227] 3. Initial Analysis and Sentiment Analysis

[1228] The server and emotion engine work together to analyze the received data. The emotion engine uses a natural language processing algorithm to evaluate the user's emotional state in real time from the voice data and to analyze the user's emotional state from the text data. For example, if the information says "I have a stomachache," the server will initially determine that there is something wrong with the user's health based on the emotion engine's analysis results.

[1229] 4. Follow-up questions

[1230] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[1231] 5. Detailed analysis and risk assessment

[1232] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[1233] 6. Taking Action

[1234] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[1235] Specific examples

[1236] Example 1: Detecting and responding to poor health

[1237] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[1238] 2. The device sends this message to the server.

[1239] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[1240] 4. The device asks, "Tell me more about this. How long has this been going on?"

[1241] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1242] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[1243] Example 2: Regular follow-up

[1244] 1. Schedule the server to periodically check for users.

[1245] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[1246] 3. The user responded, "Much better."

[1247] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[1248] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support. In particular, by combining it with an emotion engine, it will be possible to accurately grasp the user's emotional state and provide appropriate action immediately when necessary.

[1249] The processing flow will be explained below.

[1250] Step 1:

[1251] The user turns on the AI-powered doll. The device automatically starts up and uses the camera and microphone to capture the user's face and voice, then begins authentication.

[1252] Step 2:

[1253] The device sends authentication data to the server, which verifies the received authentication data and loads the user's profile.

[1254] Step 3:

[1255] Once authentication is complete, the device asks the user, "Good morning, what are your plans for today?" The user replies, "I don't have any plans for today, but I have a stomachache."

[1256] Step 4:

[1257] The device sends the user's response as text data to the server, which then analyzes the received text data and performs sentiment analysis and an initial assessment of health risks.

[1258] Step 5:

[1259] The emotion engine analyzes the voice data sent from the server and evaluates the user's emotional state in real time, for example, determining the level of anxiety or pain.

[1260] Step 6:

[1261] Based on the results of the initial assessment and the analysis results of the emotion engine, the server creates follow-up questions and sends instructions to the device. The device asks the user, "How long has this been going on? Are there any other symptoms?"

[1262] Step 7:

[1263] The user responds, "It's been going on for about two days now, and nothing else." The device sends this new information as text data to the server.

[1264] Step 8:

[1265] The server analyzes the new data and reassesses health risks. Using an analytical algorithm, it determines the risk level in detail. At the same time, the emotion engine reassessessments based on the updated voice and text data.

[1266] Step 9:

[1267] If the server determines that a patient is at high risk, it will automatically send a warning to nearby medical institutions and care managers. It can also instruct partner delivery services to provide specific medicines or ingredients.

[1268] Step 10:

[1269] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1270] Step 11:

[1271] The server continuously monitors the conversation data and the analysis results of the emotion engine, and periodically creates a health status report and sends it to the user's family or care manager. The device periodically follows up with the user, asking, "How's your stomach pain going?" to provide ongoing support.

[1272] Example 2

[1273] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1274] Today's elderly often experience a decline in their quality of life due to physical and psychological problems associated with aging. Therefore, it is necessary to create an environment that supports the elderly in their daily lives and allows them to receive medical and nursing care services in a timely manner. However, current systems make it difficult to analyze the emotions and health status of the elderly in real time and provide appropriate responses immediately. Furthermore, there is a lack of means to detect health risks faced by the elderly early and to quickly share information.

[1275] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for enabling the elderly person to interact with a communication device; a means for analyzing the elderly person's emotions and health status based on the interaction; a means for determining the timing of necessary care and support based on the analysis results; a means for linking information with external organizations such as nursing care services and medical institutions based on the determination; a means for notifying the elderly person of the linking results; a means for recognizing the user's face and voice and reading an individual profile; a means for asking follow-up questions and collecting detailed data; a means for evaluating health risks based on the received data and updating information; and a means for sending an alert to a medical institution and instructing the provision of necessary medicines and ingredients if an abnormality is detected. This makes it possible to analyze the elderly person's emotions and health status in real time and provide appropriate care and support promptly.

[1276] "Communication devices" refer to devices used by elderly people to communicate, including devices equipped with voice recognition and cameras.

[1277] "Dialogue" refers to the verbal exchange between the elderly person and the communication device, and takes place in the form of voice or text data.

[1278] "Analyzing emotions and health status" refers to analyzing the psychological and physical state of elderly people based on their voice and text data.

[1279] "Analysis results" refers to information obtained through analytical means regarding emotions and health status.

[1280] "Determining the timing of care and support" refers to determining the appropriate time to provide the necessary care and support to the elderly based on the analysis results.

[1281] "External agencies" refers to external organizations and services involved in supporting the elderly, such as nursing care services and medical institutions.

[1282] "Collaboration" refers to sharing information with external organizations and cooperating with each other.

[1283] "Collaboration results" refers to the results and feedback obtained after sharing information with external organizations.

[1284] "Recognizing the user's face and voice" refers to the communication device capturing the facial image and voice of the elderly person and identifying the individual.

[1285] "Loading an individual profile" refers to retrieving and applying individual data stored for each recognized user.

[1286] A "follow-up question" is a question asked to gather additional information based on the initial dialogue or analysis.

[1287] "Detailed data" refers to more specific information obtained through follow-up questions.

[1288] "Assessing health risks" refers to determining the risks associated with the health status of elderly people based on detailed data and analysis results.

[1289] "Updating information" refers to reflecting new data and analysis results in the records within the system.

[1290] "Sending an alert" refers to sending information to alert medical institutions and related parties when an abnormality is detected.

[1291] "Instructing the provision of medicines and food ingredients" refers to issuing instructions to arrange for the supply of necessary medicines and food ingredients based on the health condition of the elderly person.

[1292] This invention is a system that uses communication devices equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of elderly people. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[1293] Hardware and software used

[1294] Communication device: An interactive AI device for everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user. Through this device, conversations with the elderly are conducted in real time.

[1295] Emotion engine: A software module that analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses an emotion analysis algorithm to determine the user's psychological state.

[1296] Back-end processing server: Comprehensively analyzes data sent from communication devices and the emotion engine to evaluate emotions and health risks. This server uses a health risk assessment algorithm to determine the timing of care and support, and notifies external organizations as necessary.

[1297] Generative AI models: Use specific AI models for natural language processing and speech recognition.

[1298] Specific processing of the program

[1299] 1. The server receives the voice and text data sent from the communication device and analyzes it in cooperation with the emotion engine.

[1300] 2. The emotion engine evaluates the user's emotional state in real time from the voice data and also analyzes the emotional state from the text data, thereby identifying emotions such as "anxiety" or "anger."

[1301] 3. The server evaluates health risks based on the analysis results and determines the timing of necessary care and support. If a high risk is determined, the server shares information with external organizations and notifies the user.

[1302] 4. The communication device asks additional follow-up questions to gather more detailed data, such as "How long has this been going on?" or "Are you experiencing any other symptoms?" to get more detailed information from the user.

[1303] 5. The server re-analyzes the detailed data and updates the health risk assessment, taking necessary actions accordingly.

[1304] Specific examples

[1305] Example 1: Detecting and responding to poor health

[1306] 1. The user talks to the communication device saying, "I haven't been able to sleep at night lately."

[1307] 2. The communication device sends this message to the server.

[1308] 3. The server performs an emotional analysis of the "recent sleep patterns" and a health risk assessment, and if the "difficulty falling asleep" continues, recommends a visit to a medical institution.

[1309] 4. The communication device asks, "Tell me more about this. How long has this been going on?"

[1310] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1311] 6. The server automatically contacts a nearby medical institution and notifies the user via the communication device, saying, "We're worried, so we've contacted the clinic."

[1312] Example 2: Regular follow-up

[1313] 1. Schedule the server to periodically check for users.

[1314] 2. The communication device will follow up periodically with, "How's the stomach ache going since then?"

[1315] 3. The user responds, "It's much better."

[1316] 4. The server analyzes the received data and determines that the risk has decreased. The communication device sends a message saying, "Please continue to be careful."

[1317] Example prompts for generative AI models

[1318] "Please explain the process by which an elderly person talks to an AI-enabled communication device about their recent health condition, and the emotion engine analyzes that information and provides the necessary assistance."

[1319] The present invention makes it possible to perform analysis using an emotion engine in particular, thereby enabling the user's emotional state to be grasped with high accuracy and appropriate care and support to be provided promptly.

[1320] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1321] Step 1:

[1322] The user turns on the communication device. The communication device uses a camera and microphone to capture the user's face and voice. The input data (facial recognition data and voice data) obtained through this process is sent to the server, which then authenticates the user. If authentication is successful, an individual elderly profile is loaded. Specifically, the camera on the communication device captures facial data and sends voice data to the recognition module. The server then uses facial recognition algorithms and voiceprint analysis algorithms to recognize the user.

[1323] Step 2:

[1324] Once authentication is complete, the communication device speaks to the user, asking, "Good morning. What do you have planned for today?" To which the user responds, "I don't have any plans for today, but I have a stomachache." This response is converted into text data in real time and sent to the server. The input includes the user's voice data, and the output includes the text conversion result of that voice data. Specifically, the voice recognition module of the communication device converts the user's speech into text data, and that text data is sent to the server via the Internet.

[1325] Step 3:

[1326] The server works with the emotion engine to analyze the received text data. The emotion engine evaluates the user's emotional state in real time from the voice data and similarly analyzes the emotional state from the text data. For example, emotions such as "discomfort" or "pain" are extracted from the text "my stomach hurts." Based on the results of this analysis, the server makes an initial judgment that there is something wrong with the user's health. The input data is text data, and the output data is the user's emotional state and health risk assessment. The emotion engine uses a natural language processing algorithm to analyze the text data and classify emotions.

[1327] Step 4:

[1328] If the server detects an abnormality, it instructs the communication device to ask follow-up questions to gather additional information. The communication device asks the user, "How long has this been going on? Are there any other symptoms?" to collect detailed information. If the user answers, "It's been going on for about two days now," the answer is again converted into text data and sent to the server. The input contains the user's answer to the follow-up question, and the output contains detailed text data. Specifically, the communication device performs voice recognition again and sends the follow-up question and its answer to the server as text data.

[1329] Step 5:

[1330] The server reassesses the health risk based on the detailed text data. For example, based on the response "It's been going on for about two days, but nothing else in particular," it uses an analysis algorithm to reassess the risk level. The input data is the response to the follow-up question, and the output data is the updated health risk assessment. The server uses a high-performance analysis module to analyze the obtained detailed data and reassess the health risk.

[1331] Step 6:

[1332] If the risk is determined to be high, the server sends an alert to nearby medical institutions and care managers, instructing them to provide the necessary medications and food ingredients. This information is transmitted to the user via the communication device, which sends a message saying, "I'm worried, so I've contacted a nearby clinic. Please rest." The input data is the updated health risk assessment and detailed text data, and the output data is the alert notification to the medical institution and the notification message to the user. Specifically, the server uses a communication module to send the alert data to the medical institution, and notifies the user via the speaker and display of the communication device.

[1333] keyword

[1334] Generative AI model, prompt sentence

[1335] (Application example 2)

[1336] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1337] In order to improve the quality of life of the elderly, not only appropriate nursing care and medical support but also daily health management is important. However, it is not easy for the elderly themselves to understand changes in their own health condition and emotions and take appropriate action. Furthermore, diet has a significant impact on health, so it is important to provide the elderly with appropriate meals, but it is difficult for them to choose the meals that are optimal for their health condition. To solve these issues, a system is needed that is deeply involved in the daily lives of the elderly and can make appropriate meal suggestions and provide them in real time based on their health condition.

[1338] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1339] In this invention, the server includes a means for enabling the elderly person to converse with a communicator equipped with artificial intelligence, a means for analyzing emotions and conditions based on the converse, and a means for determining the timing of necessary care and support based on the analysis results. This makes it possible to provide appropriate care and support, as well as to suggest and provide appropriate meals based on emotions and health conditions.

[1340] "Elderly people" refers to people of an age who require care and assistance in their daily lives.

[1341] An "artificial intelligence communicator" refers to a computer system that converses with elderly people and collects and analyzes voice and text data.

[1342] "Dialogue" refers to voice or text communication between an elderly person and an AI-enabled communicator.

[1343] "Means for analyzing emotions and conditions" refers to algorithms and systems that evaluate the emotional and health conditions of elderly people based on acquired voice and text data.

[1344] "Means for determining the timing of necessary care and support" refers to algorithms and systems that determine when care and support is needed based on the analysis results.

[1345] "Means for sharing information with external services" refers to communication functions for sharing analysis and assessment results with external nursing care services and medical institutions.

[1346] "Means for notifying the elderly of the results of collaboration" refers to a system that communicates the results of collaboration with external services to the elderly via voice or text message.

[1347] "Means for making appropriate meal suggestions" refers to algorithms or systems that select and suggest meal menus that are optimal for a person's health based on the results of analysis and assessment.

[1348] "Means for automated delivery" refers to a system that automatically orders the necessary ingredients and food based on a proposed meal menu and delivers them to the elderly person's home.

[1349] MODE FOR CARRYING OUT THE INVENTION

[1350] To improve the quality of life of the elderly, this invention provides a system in which the elderly interact with a communicator equipped with artificial intelligence, and the emotions and health status obtained through the interaction are analyzed to determine the timing of necessary care and support. This system automates the entire process from suggesting appropriate meals based on the analysis results to delivering those meals.

[1351] System configuration

[1352] The system mainly consists of the following components:

[1353] 1. Artificial Intelligence Communicator

[1354] 2. Emotion Engine

[1355] 3. Backend Processing Server

[1356] 4. User Interface

[1357] 5. External service integration system

[1358] Artificial Intelligence-Powered Communicator

[1359] The Communicator is a device for communicating with the elderly. It has a built-in camera and microphone, and can perform facial recognition and voice recognition of the elderly. For facial recognition, it uses Google Face API or Amazon Rekognition, and for voice recognition, it uses Google Cloud Speech-to-Text.

[1360] Emotion Engine

[1361] The emotion engine analyzes the acquired voice and text data and evaluates the emotional state of the elderly person in real time. For example, it can use IBM Watson's Tone Analyzer.

[1362] Backend Processing Server

[1363] The server evaluates health risks based on the data sent from the emotion engine and makes appropriate dietary recommendations. It also determines the timing of necessary care and support and shares the results with external services. The server uses cloud services such as AWS EC2.

[1364] User Interface

[1365] Through the user interface, elderly people can communicate their condition to the communicator through natural dialogue, and notifications and suggestions from the server are also displayed through this interface.

[1366] External service integration system

[1367] The external service integration system suggests appropriate meals based on the analysis results and automatically handles the ordering and delivery procedures for delivering those meals.

[1368] How the system works

[1369] 1. Initial Setup and User Authentication

[1370] The server authenticates the elderly person's face and voice and sends the authentication data to the server, which then loads an individual elderly person profile.

[1371] 2. Starting everyday conversations

[1372] The communicator talks to the elderly and asks them questions about their everyday health, for example, a conversation about "stomach pain."

[1373] 3. Initial Analysis and Sentiment Analysis

[1374] The emotion engine analyzes the voice and text data obtained through the dialogue and sends the results to the server.

[1375] 4. Follow-up questions and detailed analysis

[1376] The server will ask follow-up questions to gather additional information and conduct a detailed health risk assessment.

[1377] 5. Meal suggestions and delivery arrangements

[1378] We suggest an appropriate meal menu based on the elderly person's health condition, and arrange for the ingredients to be ordered and delivered by a delivery service based on that menu.

[1379] Specific examples

[1380] Example 1: User complaining of indigestion

[1381] 1. The user tells the communicator, "I have a stomachache."

[1382] 2. The communicator sends this utterance to the server.

[1383] 3. The emotion engine determines the emotion of "painful" and sends the result to the server.

[1384] 4. The server suggests to the user a menu of easily digestible foods (e.g., porridge or soup).

[1385] 5. Order food from a delivery service and arrange for it to be delivered to the senior's home.

[1386] Prompt Sentence Examples

[1387] You are an AI-powered communicator. A 70-year-old woman has recently been complaining of stomach aches and would like recommendations for foods that are easy to digest. Use your emotion engine to analyze this information and suggest appropriate menu items.

[1388] By combining these detailed procedures and techniques, the present invention can significantly improve the quality of life for the elderly.

[1389] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1390] Step 1:

[1391] The user speaks to an AI-powered communicator, saying, "My stomach hurts."

[1392] How it works: The device's microphone captures audio data and converts it into text data in real time.

[1393] Input: User's voice

[1394] Output: Text data

[1395] Step 2:

[1396] The terminal transmits the text data to the server.

[1397] How it works: The communicator sends the generated text data to the server via the cloud.

[1398] Input: Text data

[1399] Output: Data received by the server

[1400] Step 3:

[1401] The server sends the text data to the emotion engine for emotion analysis.

[1402] How it works: The server sends the text data to an analysis engine to assess the emotional state, for example using IBM Watson's Tone Analyzer.

[1403] Input: Text data

[1404] Output: Emotion analysis results

[1405] Step 4:

[1406] The emotion engine sends the analysis results back to the server.

[1407] How it works: The emotion engine identifies emotions such as "sad" and sends the results to the server.

[1408] Input: Text data

[1409] Output: Emotion analysis results

[1410] Step 5:

[1411] The server assesses the health risk and generates appropriate follow-up questions.

[1412] How it works: The server runs an algorithm to assess health risk based on the sentiment analysis results. If a risk is determined, it generates a follow-up question (e.g., "How long has this been going on?").

[1413] Input: Sentiment analysis results

[1414] Output: Follow-up question

[1415] Step 6:

[1416] The terminal presents the user with follow-up questions.

[1417] How it works: The device speaks the generated follow-up question to the user.

[1418] Input: Follow-up question

[1419] Output: Audio output

[1420] Step 7:

[1421] The user answers the follow-up question.

[1422] How it works: The user answers the device, "It's been going on for about two days," and the device again captures the voice data and converts it into text data.

[1423] Input: Voice response

[1424] Output: Text data

[1425] Step 8:

[1426] The device sends the text data to a server for detailed analysis.

[1427] How it works: The device sends the text data to a server via the cloud, where it begins detailed analysis. The server then uses an analytical algorithm to reassess the health risk.

[1428] Input: Text data

[1429] Output: Detailed analysis results

[1430] Step 9:

[1431] The server generates appropriate meal suggestions.

[1432] How it works: Based on the results of the detailed analysis, the server generates an appropriate meal menu and creates data to order from a delivery service.

[1433] Input: Detailed analysis results

[1434] Output: Meal suggestions and ordering data

[1435] Step 10:

[1436] The delivery service will process the delivery of the ingredients.

[1437] How it works: The server sends the order data to the delivery service and automatically processes the delivery of the ingredients.

[1438] Input: Order data

[1439] Output: Delivery arrangement

[1440] Step 11:

[1441] The terminal notifies the user of the delivery information.

[1442] How it works: The device receives a notification from the server and notifies the user of the order details and delivery time via voice or text.

[1443] Input:Shipping information

[1444] Output: Voice or text notification

[1445] This will enable elderly people to receive appropriate care at home while smoothly obtaining the optimal diet for their health.

[1446] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1447] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1448] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1449] [Fourth embodiment]

[1450] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1451] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1452] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1453] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1454] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1455] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1456] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1457] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1458] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1459] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1460] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1461] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1462] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1463] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of the elderly. Specific embodiments of the system are described below.

[1464] System Overview

[1465] The system of the present invention mainly comprises the following components:

[1466] 1. Communicator (terminal) equipped with artificial intelligence

[1467] 2. Backend Processing Server (Server)

[1468] 3. User Interface (User)

[1469] Communicator (terminal) equipped with artificial intelligence

[1470] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[1471] Backend processing server (server)

[1472] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1473] User Interface (User)

[1474] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1475] Program processing

[1476] The specific program processing is as follows.

[1477] 1. Initial Setup and User Authentication

[1478] The user turns on the AI-enabled doll and the device starts up. The camera and microphone automatically activate at startup to perform facial and voice recognition. The authentication data is sent to the server, where the matching results are confirmed. If authentication is successful, an individual elderly person profile is loaded.

[1479] 2. Starting everyday conversations

[1480] Once authentication is complete, the device asks, "Good morning, what are your plans for today?" The user (elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[1481] 3. Initial analysis

[1482] The server analyzes the received data and performs sentiment analysis and an initial assessment of health risks. For example, if the user says "I have a stomachache," the server may initially determine that there is something wrong with their health.

[1483] 4. Follow-up questions

[1484] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[1485] 5. Detailed analysis and risk assessment

[1486] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[1487] 6. Taking Action

[1488] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[1489] Specific examples

[1490] Example 1: Detecting and responding to poor health

[1491] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[1492] 2. The device sends this message to the server.

[1493] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[1494] 4. The device asks, "Tell me more about this. How long has this been going on?"

[1495] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1496] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[1497] Example 2: Regular follow-up

[1498] 1. Schedule the server to periodically check for users.

[1499] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[1500] 3. The user responded, "Much better."

[1501] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[1502] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support.

[1503] The processing flow will be explained below.

[1504] Step 1:

[1505] The user turns on the AI-equipped doll.

[1506] The device will automatically start up and use the camera and microphone to capture the user's face and voice to begin authentication.

[1507] Step 2:

[1508] The terminal sends the authentication data to the server.

[1509] The server verifies the received authentication data and loads the user's profile.

[1510] Step 3:

[1511] Once authentication is complete, the device will ask the user, "Good morning, what are your plans for today?"

[1512] The user replies, "I don't have any plans today, but I have a stomach ache."

[1513] Step 4:

[1514] The terminal transmits the user's response as text data to the server.

[1515] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[1516] Step 5:

[1517] Based on the results of the initial evaluation, the server creates follow-up questions and sends instructions to the terminal.

[1518] The device asks the user, "How long has this been going on? Are you experiencing any other symptoms?"

[1519] Step 6:

[1520] The user replies, "It's been going on for about two days now, nothing else."

[1521] The terminal sends this new information to the server as text data.

[1522] Step 7:

[1523] The server analyzes the new data and reassess the health risk.

[1524] If the analysis results indicate a high risk, appropriate actions will be suggested.

[1525] Step 8:

[1526] The server automatically sends alerts to nearby medical institutions and care managers to arrange for the necessary support.

[1527] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1528] Step 9:

[1529] The server continuously monitors the conversation data, periodically creates health status reports, and sends them to the user's family or care manager.

[1530] The device continues to provide support by periodically following up with the user with, "How's your stomach pain going since then?"

[1531] Example 1

[1532] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1533] It is difficult to provide timely and appropriate care and support to improve the quality of life of the elderly. Furthermore, there is a lack of a system that can accurately grasp the emotions and health status of the elderly through everyday conversations and respond when necessary. Furthermore, there is a need for effective support, including long-term assessment of health risks and necessary follow-up.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1535] In this invention, the server includes: means for enabling the elderly person to interact with an AI-equipped communicator; means for analyzing the elderly person's emotions and health status based on the interaction; means for determining the timing of necessary care and support based on the analysis results; means for linking information with external organizations based on the determination; means for notifying the elderly person of the results of the linking; means for generating follow-up questions and collecting additional information; means for using an algorithm to perform long-term health risk assessment; and means including a function for automatically sending alerts to nearby medical institutions. This enables the elderly person's health status to be continuously monitored and nursing care and medical services to be provided at the appropriate time.

[1536] "Elderly" refers to people who are older and often require care and assistance in their daily lives.

[1537] An "artificial intelligence-enabled communicator" refers to a device equipped with conversational artificial intelligence that can hold natural conversations with the elderly.

[1538] "Dialogue" refers to communication between two people, and in this system refers to voice or text exchanges between the elderly person and an AI-powered communicator.

[1539] "Analysis of emotions and health status" means analyzing the emotions and health-related information expressed by elderly people through conversations using voice data and text data.

[1540] "Determining the timing of care and support" refers to determining the timing of the need for care and support based on the analysis of emotions and health status.

[1541] "Sharing information with external organizations" means sharing information with external organizations such as medical institutions and care service providers when the timing of necessary care or support is determined.

[1542] "Notifying the results of collaboration" refers to notifying the elderly of the results of information collaboration with external organizations.

[1543] "Generating follow-up questions" refers to the AI-enabled communicator asking additional questions to the elderly person to gather additional information based on the results of the initial analysis.

[1544] "Health risk assessment algorithm" refers to a program that uses mathematical or statistical methods to assess health risks based on collected data.

[1545] The "automatic warning sending function" refers to a function that automatically sends a warning to pre-set medical institutions or care managers when the health condition of an elderly person is judged to be at risk.

[1546] A "deep learning model" is a type of artificial intelligence technology that uses neural networks to learn patterns from large amounts of data and is applied to analyzing emotions and health conditions.

[1547] The present invention is a system that uses a communicator equipped with artificial intelligence to analyze the emotions and health status of elderly people and determine the timing of necessary care and support in order to improve the quality of life of elderly people. Specific embodiments of the system are described below.

[1548] System configuration

[1549] The system consists of the following main components:

[1550] 1. Communicator (terminal) equipped with artificial intelligence

[1551] 2. Backend Processing Server (Server)

[1552] 3. User Interface (User)

[1553] Hardware and Software Configuration

[1554] Communicator (terminal) equipped with artificial intelligence

[1555] This device is equipped with conversational AI and is designed to engage in everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition for the user. The device uses natural language processing and voice recognition technologies to process dialogue with the user in real time.

[1556] Hardware used:

[1557] Camera module

[1558] microphone

[1559] speaker

[1560] Software used:

[1561] Conversational AI Platform

[1562] Voice recognition software (e.g., Google Cloud Speech-to-Text)

[1563] Natural language processing engines (e.g., Python's NLTK, Microsoft Azure's Text Analytics API)

[1564] Backend processing server (server)

[1565] The server analyzes the voice and text data sent from the device to assess emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1566] Hardware used:

[1567] High-Performance Server

[1568] Software used:

[1569] Database Management Systems

[1570] Sentiment analysis algorithms (e.g., TensorFlow, PyTorch)

[1571] Health Risk Assessment Algorithm

[1572] User Interface (User)

[1573] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1574] Specific examples and prompts

[1575] Example 1: Detecting and responding to poor health

[1576] 1. The user consults an AI-enabled communicator, saying, "I haven't been able to sleep at night lately."

[1577] 2. The device collects this speech as audio data, converts it into text data, and sends it to the server.

[1578] 3. The server analyzes the user's recent sleep patterns and performs emotional analysis and health risk assessment. For example, if the user is likely to continue having difficulty falling asleep, it may recommend a visit to a medical institution.

[1579] 4. The device will ask, "Tell me more about this. How long has it been going on?"

[1580] 5. If the user answers "about a week ago," the server updates the risk assessment and instructs the appropriate response.

[1581] Example 2: Regular follow-up

[1582] 1. The server schedules periodic checks of the user.

[1583] 2. The device will periodically follow up by asking, "How is your stomach pain since then?"

[1584] 3. If the user responds "much better," the server analyzes the received data, updates the risk assessment, and determines it to be low risk.

[1585] 4. The device will notify you to "remain careful."

[1586] Example prompt sentence:

[1587] "I haven't been able to sleep at night lately. Do you have any advice?"

[1588] "I'm having persistent stomach pains, what should I do?"

[1589] "I haven't been feeling well lately, maybe something's wrong?"

[1590] This will enable the system to provide timely and appropriate care and support to improve the quality of life of the elderly.

[1591] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1592] Step 1: Initial configuration and user authentication

[1593] When the device is turned on, the camera and microphone automatically start working, and facial and voice recognition is performed.

[1594] Input: User's face image and voice data

[1595] Data processing: Analyze facial image and audio data using appropriate recognition algorithms (e.g., Google's Facial Recognition Engine, Amazon's AWS Rekognition).

[1596] Output: Authentication result (success / failure) and user profile

[1597] The server receives the authentication data sent from the device and compares it with the database. If the authentication result is successful, the individual elderly person's profile is loaded and notified to the device.

[1598] Step 2: Starting everyday conversations

[1599] Once authentication is complete, the device greets you with, "Good morning, what's on your agenda today?"

[1600] Input: User response (e.g., "I don't have any plans today, but I have a stomach ache.")

[1601] Data processing: User responses are collected as voice data and converted into text data through voice recognition.

[1602] Output: Text data (user's speech)

[1603] The terminal converts the collected voice data into text data in real time and sends it to the server.

[1604] Step 3: Initial analysis

[1605] The server analyzes the received text data and performs sentiment analysis and initial assessment of health risks.

[1606] Input: Text data (user speech)

[1607] Data processing: Natural language processing (e.g., Python's NLTK, Microsoft Azure's Text Analytics API) is used to extract keywords and assess emotions and health status.

[1608] Output: Initial assessment result (e.g., health risk based on the information "I have a stomachache")

[1609] The server determines whether there is anything abnormal in the user's health condition based on the user's comments.

[1610] Step 4: Follow-up questions

[1611] If the server detects an abnormality, it generates an additional question and sends it to the terminal.

[1612] Input: Initial health risk assessment results

[1613] Data processing: Applying algorithms to generate questions to gather more information from the user.

[1614] Output: Follow-up question (e.g., "How long has this been going on?")

[1615] The terminal then asks the generated follow-up questions to the user.

[1616] Step 5: Detailed analysis and risk assessment

[1617] The user answers the follow-up questions, which are received by the server and analyzed again in detail.

[1618] Input: User's detailed response (e.g., "It's been going on for about two days now, nothing else.")

[1619] Data processing: Using deep learning models (e.g., TensorFlow, PyTorch) to update risk assessments.

[1620] Output: Updated health risk assessment

[1621] The server reassess the health risk based on the user's detailed answers.

[1622] Step 6: Take Action

[1623] If the server determines that the risk is high, it sends a warning to nearby medical institutions and care managers.

[1624] Input: Updated health risk assessment

[1625] Data Processing: Contact appropriate external agencies based on pre-set protocols.

[1626] Output: Automatic alerts and delivery service instructions

[1627] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1628] The user receives this notification and takes the necessary action.

[1629] (Application example 1)

[1630] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] To improve the quality of life of the elderly, it is necessary to monitor their emotions and health status on a daily basis and provide care and support at the appropriate time. However, existing systems lack real-time monitoring to detect abnormalities, and even minor abnormalities may be overlooked. In addition, effective support may be delayed due to a lack of follow-up questions and appropriate measures after abnormalities are detected. As a result, there is a risk that elderly people will not receive appropriate care when they need it, and their quality of life will decline.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1633] In this invention, the server includes means for analyzing emotions and conditions, means for determining the timing of necessary care and support, means for linking information with external services, means for notifying the elderly person of the linking results, means for monitoring changes in facial expressions and voice in real time, means for detecting abnormalities and presenting follow-up questions when an abnormality is detected, and means for analyzing answers to the follow-up questions and taking appropriate measures as necessary. This makes it possible to monitor the elderly person's condition in real time and take measures for care and support in a timely manner.

[1634] 1. "Elderly" refers to individuals over a certain age, often in retirement, who require special care and assistance in their daily lives.

[1635] 2. "Artificial intelligence communicator" refers to a device that uses AI technology to have a dialogue function and analyzes the emotions and health status of users (elderly people) through communication with them.

[1636] 3. "Means for analyzing emotions and states" refers to the process of assessing and analyzing the emotional and physical states of older adults based on input information such as voice data and text data.

[1637] 4. "Means for determining the timing of care or support" refers to algorithms or technologies that, based on the results of analysis, determine the timing and circumstances under which an elderly person will need care or support.

[1638] 5. "Means for sharing information with external services" refers to the process of sharing information with external institutions such as nursing care services and medical institutions, and coordinating the necessary care and support.

[1639] 6. "Means for notifying the elderly of the results of collaboration" refers to mechanisms and technologies for communicating the results of information collaboration with external services to the elderly in an easy-to-understand manner.

[1640] 7. "Means for monitoring changes in facial expressions and voice in real time" refers to the process of instantly observing changes in an elderly person's facial expressions and tone of voice through cameras and microphones and collecting data.

[1641] 8. "Means for detecting anomalies" refers to systems or technologies that analyze collected data and identify abnormalities that deviate from normal conditions.

[1642] 9. "Means for presenting follow-up questions" refers to a function that automatically generates additional questions to ask the elderly person when an abnormality is detected.

[1643] 10. "Means for analyzing responses to follow-up questions" refers to the process of analyzing responses from older people, extracting further information, and assessing their condition.

[1644] 11. "Means of taking appropriate measures" refers to the ability to take action to provide appropriate care and support to older people based on the results of the analysis.

[1645] This invention is a system that enables elderly people to communicate using a communicator equipped with artificial intelligence, analyzes their emotions and health status, and provides appropriate care and support. This system is implemented using the following hardware and software.

[1646] Hardware used

[1647] 1. Camera: A device for capturing facial expressions of elderly people in real time, used for facial recognition and expression analysis.

[1648] 2. Microphone: A device for obtaining voice input. It captures what the elderly person is saying and uses it to analyze their emotions and health status.

[1649] 3. Server: A central data processing unit for data processing and analysis. It is responsible for data management, analysis, and linking with external services.

[1650] 4. Communicator: A device for everyday communication for the elderly. Built-in AI enables natural conversation.

[1651] Software used

[1652] 1. OpenCV: A library for processing video data acquired from a camera and performing facial recognition and facial expression analysis.

[1653] 2. SpeechRecognition: A library for converting voice data into text data. It provides speech recognition functionality.

[1654] 3. TextBlob: A natural language processing library for parsing text data and performing sentiment assessment.

[1655] Data processing and calculation

[1656] 1. Real-time monitoring of facial expressions: Video data captured by the camera is processed in real time using OpenCV to perform facial recognition. Emotion analysis is performed from the recognized facial expressions to detect abnormalities.

[1657] 2. Real-time voice monitoring and analysis: Voice data acquired through a microphone is converted into text data using SpeechRecognition, and sentiment analysis is performed using TextBlob.

[1658] 3. Anomaly detection and follow-up: If an anomaly is detected as a result of sentiment analysis, the server generates follow-up questions and presents them to the elderly via the communicator. The elderly's answers are analyzed again to determine the necessary measures.

[1659] 4. Collaboration with external services: Based on the assessment results, the server will collaborate with external care services and medical institutions and notify the elderly of the results.

[1660] Specific examples

[1661] As a concrete example, consider the case where an elderly person says to the communicator, "I feel a little tired today." In this case, the microphone captures the voice and sends the data to the server. The server converts the voice data into text data using SpeechRecognition and performs sentiment analysis using TextBlob. If the sentiment analysis result is evaluated as "negative," the server generates a follow-up question, asking, "How long has this been going on?" The answer to this question is then reanalyzed, and appropriate measures are taken based on the results.

[1662] An example of a prompt is, "If a worker says, 'I'm feeling a bit tired today,' please write a short code that automatically analyzes their emotions and sends an alert based on the results." Based on this prompt, the system can automatically generate a corresponding code, enabling prompt and appropriate care for the elderly.

[1663] With the above configuration, the present invention makes it possible to monitor the emotions and health status of elderly people in real time and provide care and support at appropriate times.

[1664] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1665] Step 1: User authentication and configuration

[1666] When a user turns on the communicator, the device boots up and the built-in camera and microphone start working automatically. The photographed facial recognition data and recorded voice recognition data are sent to the server for matching. If the authentication is successful, the profile of each elderly person is loaded. Input: Facial recognition data and voice recognition data. Data processing: Data matching. Output: Authentication results and loading of user profile.

[1667] Step 2: Starting everyday conversations

[1668] After authentication is complete, the device speaks to the user, saying, "Good morning. What do you have planned for today?" If the user (elderly person) replies, "I don't have any plans for today, but I have a stomachache," the voice data is sent to the server in real time. Input: User's speech. Data processing: Voice data converted to text. Output: Text data.

[1669] Step 3: Sentiment Analysis

[1670] The server analyzes the received text data and performs sentiment analysis and an initial assessment of health risks. For example, from the information "I have a stomachache," the server may initially determine that there is something wrong with the user's health. Input: User's spoken text data. Data processing: Sentiment analysis and health risk assessment. Output: Initial health assessment results.

[1671] Step 4: Follow-up questions

[1672] If the server detects an abnormality, it generates follow-up questions to gather more information and provides them to the terminal. The terminal asks the user, "How long has this been going on? Are there any other symptoms?" Input: Initial health assessment results. Data processing: Generation of follow-up questions. Output: Follow-up questions.

[1673] Step 5: Analyze follow-up responses

[1674] After receiving additional responses from the user, the device sends the responses to the server. The server analyzes them again and updates the health risk assessment. For example, if the user answers, "It's been going on for about two days now," the server reassess the detailed situation. Input: User's follow-up response. Data processing: Analysis of additional information and risk reassessment. Output: Updated health risk assessment results.

[1675] Step 6: Take Action

[1676] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care services, and notifies the user of this information via their device. It also arranges for food and medicine delivery services as needed. Input: Updated health risk assessment results. Data processing: Linkage processing with external services. Output: Sending a warning to medical institutions and notifying the user.

[1677] The above processing steps realize a system that monitors the emotions and health status of elderly people in real time and provides care and support at the appropriate time.

[1678] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1679] This invention is a system that uses a communicator equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of the elderly. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[1680] System Overview

[1681] The system of the present invention mainly comprises the following components:

[1682] 1. Communicator (terminal) equipped with artificial intelligence

[1683] 2. Emotion Engine

[1684] 3. Backend Processing Server (Server)

[1685] 4. User Interface (User)

[1686] Communicator (terminal) equipped with artificial intelligence

[1687] This device is equipped with conversational AI and is designed to engage in everyday conversations with elderly people. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user, making it possible to provide personalized support for each elderly person.

[1688] Emotion Engine

[1689] The emotion engine analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses emotion analysis algorithms to determine the user's psychological state and sends that information to a back-end processing server.

[1690] Backend processing server (server)

[1691] The server comprehensively analyzes data sent from the device and emotion engine to evaluate emotions and health risks. Based on the results of this analysis, it determines the timing of care and support, and shares information with care services and medical institutions as needed.

[1692] User Interface (User)

[1693] By interacting with the AI-equipped device, users (elderly people) can express their everyday problems and anxieties, and the system will analyze them and prompt appropriate actions. The elderly can receive support through natural conversation without having to think about how to operate the device.

[1694] Program processing

[1695] The specific program processing is as follows.

[1696] 1. Initial Setup and User Authentication

[1697] When a user turns on the AI-powered doll, the device automatically starts up and uses the camera and microphone to capture the user's face and voice to begin authentication. The authentication data is sent to the server, where the matched results are confirmed. If authentication is successful, a personalized elderly profile is loaded.

[1698] 2. Starting everyday conversations

[1699] Once authentication is complete, the device speaks to the user, asking, "Good morning. What are your plans for today?" The user (an elderly person) responds, "I don't have any plans for today, but I have a stomachache." This response is sent to the server in real time as text data.

[1700] 3. Initial Analysis and Sentiment Analysis

[1701] The server and emotion engine work together to analyze the received data. The emotion engine uses a natural language processing algorithm to evaluate the user's emotional state in real time from the voice data and to analyze the user's emotional state from the text data. For example, if the information says "I have a stomachache," the server will initially determine that there is something wrong with the user's health based on the emotion engine's analysis results.

[1702] 4. Follow-up questions

[1703] If the server detects an abnormality, it instructs the device to ask follow-up questions to gather additional information. Questions such as "How long has this been going on? Are you experiencing any other symptoms?" are posed to the user from the device, and more detailed information is gathered.

[1704] 5. Detailed analysis and risk assessment

[1705] The server then analyzes the data again and updates the health risk assessment based on the user's detailed response. For example, if the user replies, "It's been going on for about two days, and nothing else has happened," the server uses the analysis algorithm to reassess the risk.

[1706] 6. Taking Action

[1707] If the analysis results indicate a high risk, the server automatically sends a warning to nearby medical institutions and care managers. It can also instruct affiliated delivery services to provide specific medicines or food ingredients. This information is communicated to the user via the device. The device will then notify the user with a message such as, "We're worried, so we've contacted a nearby clinic. Please rest."

[1708] Specific examples

[1709] Example 1: Detecting and responding to poor health

[1710] 1. The user consults the AI-enabled doll, saying, "I haven't been able to sleep at night lately."

[1711] 2. The device sends this message to the server.

[1712] 3. The server performs an emotional analysis of the "recent sleep patterns" and assesses health risks, and if the "difficulty falling asleep" continues, it recommends a visit to a medical institution.

[1713] 4. The device asks, "Tell me more about this. How long has this been going on?"

[1714] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1715] 6. The server automatically contacts nearby medical institutions and notifies the user via their device. "I'm worried, so I contacted the clinic."

[1716] Example 2: Regular follow-up

[1717] 1. Schedule the server to periodically check for users.

[1718] 2. The device periodically follows up by asking, "How is your stomach pain since then?"

[1719] 3. The user responded, "Much better."

[1720] 4. The server analyzes the received data and determines that the risk has decreased. The device sends a message saying, "Please continue to be careful."

[1721] A system based on the above specific examples will improve the quality of life for the elderly and enable the provision of timely and appropriate care and support. In particular, by combining it with an emotion engine, it will be possible to accurately grasp the user's emotional state and provide appropriate action immediately when necessary.

[1722] The processing flow will be explained below.

[1723] Step 1:

[1724] The user turns on the AI-powered doll. The device automatically starts up and uses the camera and microphone to capture the user's face and voice, then begins authentication.

[1725] Step 2:

[1726] The device sends authentication data to the server, which verifies the received authentication data and loads the user's profile.

[1727] Step 3:

[1728] Once authentication is complete, the device asks the user, "Good morning, what are your plans for today?" The user replies, "I don't have any plans for today, but I have a stomachache."

[1729] Step 4:

[1730] The device sends the user's response as text data to the server, which then analyzes the received text data and performs sentiment analysis and an initial assessment of health risks.

[1731] Step 5:

[1732] The emotion engine analyzes the voice data sent from the server and evaluates the user's emotional state in real time, for example, determining the level of anxiety or pain.

[1733] Step 6:

[1734] Based on the results of the initial assessment and the analysis results of the emotion engine, the server creates follow-up questions and sends instructions to the device. The device asks the user, "How long has this been going on? Are there any other symptoms?"

[1735] Step 7:

[1736] The user responds, "It's been going on for about two days now, and nothing else." The device sends this new information as text data to the server.

[1737] Step 8:

[1738] The server analyzes the new data and reassesses health risks. Using an analytical algorithm, it determines the risk level in detail. At the same time, the emotion engine reassessessments based on the updated voice and text data.

[1739] Step 9:

[1740] If the server determines that a patient is at high risk, it will automatically send a warning to nearby medical institutions and care managers. It can also instruct partner delivery services to provide specific medicines or ingredients.

[1741] Step 10:

[1742] The device notifies the user, "We're worried about you, so we've contacted a nearby clinic. Please rest."

[1743] Step 11:

[1744] The server continuously monitors the conversation data and the analysis results of the emotion engine, and periodically creates a health status report and sends it to the user's family or care manager. The device periodically follows up with the user, asking, "How's your stomach pain going?" to provide ongoing support.

[1745] Example 2

[1746] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1747] Today's elderly often experience a decline in their quality of life due to physical and psychological problems associated with aging. Therefore, it is necessary to create an environment that supports the elderly in their daily lives and allows them to receive medical and nursing care services in a timely manner. However, current systems make it difficult to analyze the emotions and health status of the elderly in real time and provide appropriate responses immediately. Furthermore, there is a lack of means to detect health risks faced by the elderly early and to quickly share information.

[1748] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: a means for enabling the elderly person to interact with a communication device; a means for analyzing the elderly person's emotions and health status based on the interaction; a means for determining the timing of necessary care and support based on the analysis results; a means for linking information with external organizations such as nursing care services and medical institutions based on the determination; a means for notifying the elderly person of the linking results; a means for recognizing the user's face and voice and reading an individual profile; a means for asking follow-up questions and collecting detailed data; a means for evaluating health risks based on the received data and updating information; and a means for sending an alert to a medical institution and instructing the provision of necessary medicines and ingredients if an abnormality is detected. This makes it possible to analyze the elderly person's emotions and health status in real time and provide appropriate care and support promptly.

[1749] "Communication devices" refer to devices used by elderly people to communicate, including devices equipped with voice recognition and cameras.

[1750] "Dialogue" refers to the verbal exchange between the elderly person and the communication device, and takes place in the form of voice or text data.

[1751] "Analyzing emotions and health status" refers to analyzing the psychological and physical state of elderly people based on their voice and text data.

[1752] "Analysis results" refers to information obtained through analytical means regarding emotions and health status.

[1753] "Determining the timing of care and support" refers to determining the appropriate time to provide the necessary care and support to the elderly based on the analysis results.

[1754] "External agencies" refers to external organizations and services involved in supporting the elderly, such as nursing care services and medical institutions.

[1755] "Collaboration" refers to sharing information with external organizations and cooperating with each other.

[1756] "Collaboration results" refers to the results and feedback obtained after sharing information with external organizations.

[1757] "Recognizing the user's face and voice" refers to the communication device capturing the facial image and voice of the elderly person and identifying the individual.

[1758] "Loading an individual profile" refers to retrieving and applying individual data stored for each recognized user.

[1759] A "follow-up question" is a question asked to gather additional information based on the initial dialogue or analysis.

[1760] "Detailed data" refers to more specific information obtained through follow-up questions.

[1761] "Assessing health risks" refers to determining the risks associated with the health status of elderly people based on detailed data and analysis results.

[1762] "Updating information" refers to reflecting new data and analysis results in the records within the system.

[1763] "Sending an alert" refers to sending information to alert medical institutions and related parties when an abnormality is detected.

[1764] "Instructing the provision of medicines and food ingredients" refers to issuing instructions to arrange for the supply of necessary medicines and food ingredients based on the health condition of the elderly person.

[1765] This invention is a system that uses communication devices equipped with artificial intelligence to analyze emotions and health conditions and determine the timing of necessary care and support in order to improve the quality of life of elderly people. In particular, by combining it with an emotion engine that recognizes the user's emotions, it achieves more accurate analysis and appropriate support.

[1766] Hardware and software used

[1767] Communication device: An interactive AI device for everyday conversations with the elderly. It has a built-in camera and microphone, and performs facial recognition and voice recognition of the user. Through this device, conversations with the elderly are conducted in real time.

[1768] Emotion engine: A software module that analyzes voice and text data acquired through user interaction to assess the user's emotional state in real time. It uses an emotion analysis algorithm to determine the user's psychological state.

[1769] Back-end processing server: Comprehensively analyzes data sent from communication devices and the emotion engine to evaluate emotions and health risks. This server uses a health risk assessment algorithm to determine the timing of care and support, and notifies external organizations as necessary.

[1770] Generative AI models: Use specific AI models for natural language processing and speech recognition.

[1771] Specific processing of the program

[1772] 1. The server receives the voice and text data sent from the communication device and analyzes it in cooperation with the emotion engine.

[1773] 2. The emotion engine evaluates the user's emotional state in real time from the voice data and also analyzes the emotional state from the text data, thereby identifying emotions such as "anxiety" or "anger."

[1774] 3. The server evaluates health risks based on the analysis results and determines the timing of necessary care and support. If a high risk is determined, the server shares information with external organizations and notifies the user.

[1775] 4. The communication device asks additional follow-up questions to gather more detailed data, such as "How long has this been going on?" or "Are you experiencing any other symptoms?" to get more detailed information from the user.

[1776] 5. The server re-analyzes the detailed data and updates the health risk assessment, taking necessary actions accordingly.

[1777] Specific examples

[1778] Example 1: Detecting and responding to poor health

[1779] 1. The user talks to the communication device saying, "I haven't been able to sleep at night lately."

[1780] 2. The communication device sends this message to the server.

[1781] 3. The server performs an emotional analysis of the "recent sleep patterns" and a health risk assessment, and if the "difficulty falling asleep" continues, recommends a visit to a medical institution.

[1782] 4. The communication device asks, "Tell me more about this. How long has this been going on?"

[1783] 5. If the user answers "about a week ago," the server reassess the risk and determines it to be "high risk."

[1784] 6. The server automatically contacts a nearby medical institution and notifies the user via the communication device, saying, "We're worried, so we've contacted the clinic."

[1785] Example 2: Regular follow-up

[1786] 1. Schedule the server to periodically check for users.

[1787] 2. The communication device will follow up periodically with, "How's the stomach ache going since then?"

[1788] 3. The user responds, "It's much better."

[1789] 4. The server analyzes the received data and determines that the risk has decreased. The communication device sends a message saying, "Please continue to be careful."

[1790] Example prompts for generative AI models

[1791] "Please explain the process by which an elderly person talks to an AI-enabled communication device about their recent health condition, and the emotion engine analyzes that information and provides the necessary assistance."

[1792] The present invention makes it possible to perform analysis using an emotion engine in particular, thereby enabling the user's emotional state to be grasped with high accuracy and appropriate care and support to be provided promptly.

[1793] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1794] Step 1:

[1795] The user turns on the communication device. The communication device uses a camera and microphone to capture the user's face and voice. The input data (facial recognition data and voice data) obtained through this process is sent to the server, which then authenticates the user. If authentication is successful, an individual elderly profile is loaded. Specifically, the camera on the communication device captures facial data and sends voice data to the recognition module. The server then uses facial recognition algorithms and voiceprint analysis algorithms to recognize the user.

[1796] Step 2:

[1797] Once authentication is complete, the communication device speaks to the user, asking, "Good morning. What do you have planned for today?" To which the user responds, "I don't have any plans for today, but I have a stomachache." This response is converted into text data in real time and sent to the server. The input includes the user's voice data, and the output includes the text conversion result of that voice data. Specifically, the voice recognition module of the communication device converts the user's speech into text data, and that text data is sent to the server via the Internet.

[1798] Step 3:

[1799] The server works with the emotion engine to analyze the received text data. The emotion engine evaluates the user's emotional state in real time from the voice data and similarly analyzes the emotional state from the text data. For example, emotions such as "discomfort" or "pain" are extracted from the text "my stomach hurts." Based on the results of this analysis, the server makes an initial judgment that there is something wrong with the user's health. The input data is text data, and the output data is the user's emotional state and health risk assessment. The emotion engine uses a natural language processing algorithm to analyze the text data and classify emotions.

[1800] Step 4:

[1801] If the server detects an abnormality, it instructs the communication device to ask follow-up questions to gather additional information. The communication device asks the user, "How long has this been going on? Are there any other symptoms?" to collect detailed information. If the user answers, "It's been going on for about two days now," the answer is again converted into text data and sent to the server. The input contains the user's answer to the follow-up question, and the output contains detailed text data. Specifically, the communication device performs voice recognition again and sends the follow-up question and its answer to the server as text data.

[1802] Step 5:

[1803] The server reassesses the health risk based on the detailed text data. For example, based on the response "It's been going on for about two days, but nothing else in particular," it uses an analysis algorithm to reassess the risk level. The input data is the response to the follow-up question, and the output data is the updated health risk assessment. The server uses a high-performance analysis module to analyze the obtained detailed data and reassess the health risk.

[1804] Step 6:

[1805] If the risk is determined to be high, the server sends an alert to nearby medical institutions and care managers, instructing them to provide the necessary medications and food ingredients. This information is transmitted to the user via the communication device, which sends a message saying, "I'm worried, so I've contacted a nearby clinic. Please rest." The input data is the updated health risk assessment and detailed text data, and the output data is the alert notification to the medical institution and the notification message to the user. Specifically, the server uses a communication module to send the alert data to the medical institution, and notifies the user via the speaker and display of the communication device.

[1806] keyword

[1807] Generative AI model, prompt sentence

[1808] (Application example 2)

[1809] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1810] In order to improve the quality of life of the elderly, not only appropriate nursing care and medical support but also daily health management is important. However, it is not easy for the elderly themselves to understand changes in their own health condition and emotions and take appropriate action. Furthermore, diet has a significant impact on health, so it is important to provide the elderly with appropriate meals, but it is difficult for them to choose the meals that are optimal for their health condition. To solve these issues, a system is needed that is deeply involved in the daily lives of the elderly and can make appropriate meal suggestions and provide them in real time based on their health condition.

[1811] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1812] In this invention, the server includes a means for enabling the elderly person to converse with a communicator equipped with artificial intelligence, a means for analyzing emotions and conditions based on the converse, and a means for determining the timing of necessary care and support based on the analysis results. This makes it possible to provide appropriate care and support, as well as to suggest and provide appropriate meals based on emotions and health conditions.

[1813] "Elderly people" refers to people of an age who require care and assistance in their daily lives.

[1814] An "artificial intelligence communicator" refers to a computer system that converses with elderly people and collects and analyzes voice and text data.

[1815] "Dialogue" refers to voice or text communication between an elderly person and an AI-enabled communicator.

[1816] "Means for analyzing emotions and conditions" refers to algorithms and systems that evaluate the emotional and health conditions of elderly people based on acquired voice and text data.

[1817] "Means for determining the timing of necessary care and support" refers to algorithms and systems that determine when care and support is needed based on the analysis results.

[1818] "Means for sharing information with external services" refers to communication functions for sharing analysis and assessment results with external nursing care services and medical institutions.

[1819] "Means for notifying the elderly of the results of collaboration" refers to a system that communicates the results of collaboration with external services to the elderly via voice or text message.

[1820] "Means for making appropriate meal suggestions" refers to algorithms or systems that select and suggest meal menus that are optimal for a person's health based on the results of analysis and assessment.

[1821] "Means for automated delivery" refers to a system that automatically orders the necessary ingredients and food based on a proposed meal menu and delivers them to the elderly person's home.

[1822] MODE FOR CARRYING OUT THE INVENTION

[1823] To improve the quality of life of the elderly, this invention provides a system in which the elderly interact with a communicator equipped with artificial intelligence, and the emotions and health status obtained through the interaction are analyzed to determine the timing of necessary care and support. This system automates the entire process from suggesting appropriate meals based on the analysis results to delivering those meals.

[1824] System configuration

[1825] The system mainly consists of the following components:

[1826] 1. Artificial Intelligence Communicator

[1827] 2. Emotion Engine

[1828] 3. Backend Processing Server

[1829] 4. User Interface

[1830] 5. External service integration system

[1831] Artificial Intelligence-Powered Communicator

[1832] The Communicator is a device for communicating with the elderly. It has a built-in camera and microphone, and can perform facial recognition and voice recognition of the elderly. For facial recognition, it uses Google Face API or Amazon Rekognition, and for voice recognition, it uses Google Cloud Speech-to-Text.

[1833] Emotion Engine

[1834] The emotion engine analyzes the acquired voice and text data and evaluates the emotional state of the elderly person in real time. For example, it can use IBM Watson's Tone Analyzer.

[1835] Backend Processing Server

[1836] The server evaluates health risks based on the data sent from the emotion engine and makes appropriate dietary recommendations. It also determines the timing of necessary care and support and shares the results with external services. The server uses cloud services such as AWS EC2.

[1837] User Interface

[1838] Through the user interface, elderly people can communicate their condition to the communicator through natural dialogue, and notifications and suggestions from the server are also displayed through this interface.

[1839] External service integration system

[1840] The external service integration system suggests appropriate meals based on the analysis results and automatically handles the ordering and delivery procedures for delivering those meals.

[1841] How the system works

[1842] 1. Initial Setup and User Authentication

[1843] The server authenticates the elderly person's face and voice and sends the authentication data to the server, which then loads an individual elderly person profile.

[1844] 2. Starting everyday conversations

[1845] The communicator talks to the elderly and asks them questions about their everyday health, for example, a conversation about "stomach pain."

[1846] 3. Initial Analysis and Sentiment Analysis

[1847] The emotion engine analyzes the voice and text data obtained through the dialogue and sends the results to the server.

[1848] 4. Follow-up questions and detailed analysis

[1849] The server will ask follow-up questions to gather additional information and conduct a detailed health risk assessment.

[1850] 5. Meal suggestions and delivery arrangements

[1851] We suggest an appropriate meal menu based on the elderly person's health condition, and arrange for the ingredients to be ordered and delivered by a delivery service based on that menu.

[1852] Specific examples

[1853] Example 1: User complaining of indigestion

[1854] 1. The user tells the communicator, "I have a stomachache."

[1855] 2. The communicator sends this utterance to the server.

[1856] 3. The emotion engine determines the emotion of "painful" and sends the result to the server.

[1857] 4. The server suggests to the user a menu of easily digestible foods (e.g., porridge or soup).

[1858] 5. Order food from a delivery service and arrange for it to be delivered to the senior's home.

[1859] Prompt Sentence Examples

[1860] You are an AI-powered communicator. A 70-year-old woman has recently been complaining of stomach aches and would like recommendations for foods that are easy to digest. Use your emotion engine to analyze this information and suggest appropriate menu items.

[1861] By combining these detailed procedures and techniques, the present invention can significantly improve the quality of life for the elderly.

[1862] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1863] Step 1:

[1864] The user speaks to an AI-powered communicator, saying, "My stomach hurts."

[1865] How it works: The device's microphone captures audio data and converts it into text data in real time.

[1866] Input: User's voice

[1867] Output: Text data

[1868] Step 2:

[1869] The terminal transmits the text data to the server.

[1870] How it works: The communicator sends the generated text data to the server via the cloud.

[1871] Input: Text data

[1872] Output: Data received by the server

[1873] Step 3:

[1874] The server sends the text data to the emotion engine for emotion analysis.

[1875] How it works: The server sends the text data to an analysis engine to assess the emotional state, for example using IBM Watson's Tone Analyzer.

[1876] Input: Text data

[1877] Output: Emotion analysis results

[1878] Step 4:

[1879] The emotion engine sends the analysis results back to the server.

[1880] How it works: The emotion engine identifies emotions such as "sad" and sends the results to the server.

[1881] Input: Text data

[1882] Output: Emotion analysis results

[1883] Step 5:

[1884] The server assesses the health risk and generates appropriate follow-up questions.

[1885] How it works: The server runs an algorithm to assess health risk based on the sentiment analysis results. If a risk is determined, it generates a follow-up question (e.g., "How long has this been going on?").

[1886] Input: Sentiment analysis results

[1887] Output: Follow-up question

[1888] Step 6:

[1889] The terminal presents the user with follow-up questions.

[1890] How it works: The device speaks the generated follow-up question to the user.

[1891] Input: Follow-up question

[1892] Output: Audio output

[1893] Step 7:

[1894] The user answers the follow-up question.

[1895] How it works: The user answers the device, "It's been going on for about two days," and the device again captures the voice data and converts it into text data.

[1896] Input: Voice response

[1897] Output: Text data

[1898] Step 8:

[1899] The device sends the text data to a server for detailed analysis.

[1900] How it works: The device sends the text data to a server via the cloud, where it begins detailed analysis. The server then uses an analytical algorithm to reassess the health risk.

[1901] Input: Text data

[1902] Output: Detailed analysis results

[1903] Step 9:

[1904] The server generates appropriate meal suggestions.

[1905] How it works: Based on the results of the detailed analysis, the server generates an appropriate meal menu and creates data to order from a delivery service.

[1906] Input: Detailed analysis results

[1907] Output: Meal suggestions and ordering data

[1908] Step 10:

[1909] The delivery service will process the delivery of the ingredients.

[1910] How it works: The server sends the order data to the delivery service and automatically processes the delivery of the ingredients.

[1911] Input: Order data

[1912] Output: Delivery arrangement

[1913] Step 11:

[1914] The terminal notifies the user of the delivery information.

[1915] How it works: The device receives a notification from the server and notifies the user of the order details and delivery time via voice or text.

[1916] Input:Shipping information

[1917] Output: Voice or text notification

[1918] This will enable elderly people to receive appropriate care at home while smoothly obtaining the optimal diet for their health.

[1919] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1920] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1921] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1922] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1923] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1924] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1925] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1926] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1927] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1928] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1929] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1930] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1931] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1932] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1933] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1934] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1935] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1936] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1937] Furthermore, the hardware structure of the...

Claims

1. A means to enable older people to interact with an artificial intelligence-enabled communicator; means for analyzing emotions and states based on the dialogue; A means for determining the timing of necessary care and support based on the analysis results; A means for linking information with external services such as nursing care services and medical institutions based on the determination; The system includes a means for notifying the elderly person of the results of the collaboration.

2. 2. The system according to claim 1, wherein the emotion and state analysis means performs analysis using voice data and text data.

3. 2. The system according to claim 1, wherein the means for determining the timing of the necessary care or support utilizes a health risk assessment algorithm.

4. 2. The system according to claim 1, wherein the means for linking information with the external service has an automatic notification function for medical institutions and care managers.

5. 2. The system according to claim 1, wherein the means for notifying the elderly person transmits the content of the action in real time by voice or text.

Images