system

Abstract

We provide the system. [Solution] A means of communicating with the elderly on a daily basis using an interactive speaker equipped with generative artificial intelligence, A means for converting audio data acquired by the aforementioned interactive speaker into text data and analyzing it based on frailty criteria, A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, A means of sharing the recorded data with the user's family, Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly, A means of receiving feedback from elderly people and updating the feedback data in a database. A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, as the number of elderly people requiring care and living alone has been increasing, it has become an important social issue to prevent the progression of frailty (weakness) and maintain the health of these elderly people. However, it is difficult for family members living separately to grasp the daily health status of the elderly and provide appropriate support. Furthermore, it is also difficult for the elderly themselves to accurately recognize their own health status and practice effective health behaviors. There is a need for a system to address such a situation.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides the following means. First, it uses an interactive speaker equipped with generative artificial intelligence to communicate with elderly people on a daily basis. Next, it converts the voice data acquired by the interactive speaker into text data and analyzes it based on frailty criteria. Furthermore, it evaluates the progress of frailty based on the analysis results and records the evaluation results in a database. These evaluation results are shared with the user's family. In addition, it provides elderly people with reminders for health behaviors based on the evaluation results and updates the database with feedback from the elderly people. This makes it possible for family members living far away to understand the health status of elderly people and provide appropriate support, and enables elderly people themselves to recognize their health status and practice effective health behaviors.

[0006] "Generative artificial intelligence" refers to artificial intelligence that utilizes technologies such as natural language processing and machine learning to understand human language and behavior and engage in dialogue.

[0007] An "interactive speaker" is a speaker device that can respond to voice input and provide voice feedback.

[0008] The term "elderly" generally refers to people aged 60 and over, and signifies those who face increased health risks associated with aging.

[0009] "Voice data" refers to data that records the words and voice information spoken by the user in digital format.

[0010] "Text data" refers to a data format in which audio data is converted into text information.

[0011] "Frailty" refers to a state in which physical functions decline and one becomes weak due to aging.

[0012] "Frailty criteria" refer to standards and evaluation indicators used to assess the progression and risk of frailty.

[0013] "Analysis" refers to the act of evaluating data using calculations and algorithms to extract meaningful information.

[0014] A "database" refers to an electronic information storage system that systematically stores collected information and allows data to be retrieved as needed.

[0015] "Evaluation results" refers to the assessment of health status analyzed based on frailty criteria.

[0016] A "reminder" refers to a notification or message designed to prompt a user to take a specific action.

[0017] "Feedback" refers to information and responses provided by users, which are used to improve and adjust the system. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be described.

[0021] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention relates to a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. This system operates primarily with a server, terminals, and users.

[0040] 1. Initial setup and user registration

[0041] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, daily habits, etc.

[0042] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter "Taro Yamada, 65 years old, male, takes a walk every day."

[0043] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[0044] 2. Everyday communication

[0045] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[0046] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0047] Terminal: Records the user's voice response, converts it to text data, and sends it to the server. Using speech recognition technology, it generates text data from the voice statement, "I'm a little tired today."

[0048] Server: Analyzes text data and assesses the user's health status based on frailty criteria. The analysis results are recorded in a database.

[0049] 3. Data accumulation and analysis

[0050] Server: Aggregates data accumulated daily to comprehensively evaluate the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week."

[0051] Server: Calculates an assessment score based on frailty criteria. This score is shared with the user's family. For example, an assessment result such as "Frailty Risk Score: 70 / 100, Caution" is sent to the user's family via email.

[0052] 4. Support and Reminders

[0053] Device: Based on evaluation results, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today."

[0054] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I went for a walk."

[0055] Terminal: Receives user feedback and sends it to the server. The feedback data is updated in the database.

[0056] Server: Reassess the frailty risk based on the updated data and create the next action plan as needed.

[0057] As described above, the system of the present invention supports the maintenance of the health of the elderly by having the server, terminal, and user work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice.

[0058] The following describes the processing flow.

[0059] Step 1:

[0060] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[0061] Specific action: The screen displays the message "Please enter your name" and a text box.

[0062] Step 2:

[0063] User: Enter your information in the terminal's input field and press the submit button.

[0064] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[0065] Step 3:

[0066] Terminal: Sends the entered information to the server.

[0067] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[0068] Step 4:

[0069] Server: Stores received information in the database.

[0070] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[0071] Step 5:

[0072] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[0073] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[0074] Step 6:

[0075] User: Responds to the device's questions using voice.

[0076] Specific action: For example, the user responds, "I'm a little tired today."

[0077] Step 7:

[0078] Terminal: Records the user's voice response, converts it to text data, and sends it to the server.

[0079] Specific operation: Using speech recognition technology, the phrase "I'm a little tired today" is converted to text and sent to the server.

[0080] Step 8:

[0081] Server: Analyzes text data and evaluates the user's health status based on frailty criteria.

[0082] Specific actions: Analyze text data, evaluate it as "mild fatigue," and record it in the database.

[0083] Step 9:

[0084] Server: Collects data accumulated daily and evaluates the user's progress in frailty.

[0085] Specific actions: For example, aggregating data from the past month and detecting trends such as "feeling tired more than twice a week."

[0086] Step 10:

[0087] Server: Calculates an assessment score based on frailty criteria and notifies the user's family.

[0088] Specific actions: For example, notify family members via email of the result, "Frailty risk score: 70 / 100, caution advised."

[0089] Step 11:

[0090] Device: Provides users with health behavior reminders based on evaluation results.

[0091] Specific actions: For example, a voice reminder saying, "It would be good to take a 10-minute walk today."

[0092] Step 12:

[0093] User: Follow the reminder and report to the device whether or not the action was performed.

[0094] Specific actions: For example, answering, "Yes, I went for a walk."

[0095] Step 13:

[0096] Terminal: Receives user feedback and sends it to the server.

[0097] Specific operation: Feedback data is sent from the terminal to the server.

[0098] Step 14:

[0099] Server: Saves updated feedback data to the database and recalculates the frailty risk assessment.

[0100] Specific actions: Update the database and calculate the new frailty risk score.

[0101] The above describes the specific processing flow in the system of the present invention. This process makes it possible to continuously monitor the progress of frailty in elderly individuals and provide appropriate support.

[0102] (Example 1)

[0103] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0104] In modern society, understanding the health status of the elderly and providing appropriate care is a crucial challenge. In particular, there is a need to detect and prevent the progression of frailty in the elderly at an early stage. However, current systems lack the means to monitor the progression of frailty in the elderly through daily communication and to provide appropriate alerts and reminders. As a result, the progression of frailty may be overlooked, posing a risk of a decline in the quality of life for the elderly.

[0105] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0106] In this invention, the server includes means for communicating with elderly people on a daily basis using a generative AI model; means for converting voice data acquired by an interactive voice output device into text data and analyzing it; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing it with family members; means for providing reminders for healthy behaviors; means for receiving and updating feedback; means for recording and analyzing voice responses; and means for aggregating data and calculating a score. This makes it possible to continuously monitor the progress of frailty in elderly people and provide appropriate care.

[0107] "Generative artificial intelligence" is a type of artificial intelligence that uses natural language processing and machine learning models to interact with users and perform data analysis.

[0108] An "interactive voice output device" is a device that communicates with the user through voice, and smart speakers are an example of such devices.

[0109] The term "elderly person" generally refers to a person aged 65 or older.

[0110] "Frailty criteria" are standards used to assess the health status and physical decline of older adults, and are determined based on medical evaluation indicators.

[0111] "Text data" refers to data such as audio and images that are represented as a string of characters.

[0112] A "database" is a system for efficiently storing, managing, and retrieving data.

[0113] "Healthy behaviors" refer to actions that older adults should take to prevent disease and maintain their health.

[0114] A "reminder" is a notification or message sent to a user to prompt them to take a specific action.

[0115] "Feedback" refers to the information that users provide to a system regarding their actions and status.

[0116] "Speech recognition technology" is a technology that analyzes speech data and converts it into text data.

[0117] An "evaluation score" is a numerical representation of the analysis results, used to quantitatively assess the user's condition.

[0118] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze data and is trained to perform a specific task.

[0119] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on the tasks or analyses it should perform.

[0120] This invention is a system for monitoring and preventing frailty in the elderly using an interactive voice output device equipped with a generative AI model. This system operates primarily with a server, terminals, and users.

[0121] 1. Initial setup and user registration

[0122] Terminal: Provides an interface for registering the user's initial information. For example, an application installed on a tablet or smartphone is used. The application is developed in Swift for iOS devices and in Kotlin for Android® devices. The user enters basic information such as name, age, gender, and daily habits. This data is sent to the server in JSON format via the HTTPS protocol.

[0123] Server: Receives user information sent from terminals, analyzes it, and stores it in a database. Cloud platforms such as Amazon RDS and Google® Cloud Firestore are used as databases. The data collected here will later become foundational data for evaluating the health status of elderly individuals.

[0124] 2. Everyday communication

[0125] Device: A scheduled task is triggered at a specific time, such as 8 AM every morning, to broadcast a voice message saying, "Good morning, how are you feeling today?" This voice message is generated using the Google Cloud Text-to-Speech API.

[0126] User: Responds to the device's questions using voice. For example, responds, "I'm a little tired today."

[0127] Terminal: Records the user's voice response and converts the audio data into text data using the Google Cloud Speech-to-Text API. This response, for example, "I'm a little tired today," is then sent to the server.

[0128] Server: Analyzes received text data and evaluates the user's health status based on frailty criteria. Natural language processing models such as GPT-4® and BERT are used as generative AI models for the analysis. Analysis results are recorded in a database.

[0129] 3. Data accumulation and analysis

[0130] Server: Collects daily data and comprehensively evaluates the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week." Data collection and analysis are performed using Python's Pandas and NumPy.

[0131] Server: Calculates an assessment score based on frailty criteria. For example, an assessment result such as "Frailty Risk Score: 70 / 100" is shared with the user's family using email services such as SendGrid or AWS® SES.

[0132] 4. Support and Reminders

[0133] Device: Based on evaluation results, it provides elderly individuals with reminders for appropriate health behaviors. For example, an interactive voice output device might send a voice message saying, "It would be good to take a 10-minute walk today." This voice message can also be generated using the Google Cloud Text-to-Speech API.

[0134] User: Performs an action based on the reminder and reports to the device, "Yes, I went for a walk." This confirms the effectiveness of the reminder.

[0135] Terminal: Receives user feedback and sends it to the server. Feedback data is sent in JSON format and updated in the database.

[0136] Server: Based on the received feedback data, reassess the frailty risk and create the next action plan as needed.

[0137] Specific example

[0138] Here are some examples of specific prompt messages:

[0139] "Analyze the reasons why users reported feeling tired yesterday and propose an appropriate action plan."

[0140] "Based on the user's daily communication data from the past week, please calculate their frailty risk score."

[0141] "Assess the user's overall health status and generate reminders for improvement."

[0142] In this way, a system is realized that supports the maintenance of the health of the elderly by having terminals, servers, and users work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice and support.

[0143] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0144] Step 1: Start the initial setup

[0145] Terminal: Launches the application and displays the user interface. The user enters basic information such as name, age, gender, and daily habits. The input data is temporarily stored within the terminal as a JSON object (input).

[0146] Server: When the user presses the submit button, the device sends this JSON data to the server using the HTTPS protocol. The server parses the received data and saves it to an Amazon RDS database (output). This registers the user's basic information as initial settings.

[0147] Step 2: Start daily communication

[0148] Terminal: At a set time, for example, every morning at 8:00 AM, a scheduled task is triggered, generating and sending a voice message saying "Good morning, how are you feeling today?" using the Google Cloud Text-to-Speech API (input and output).

[0149] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0150] Terminal: Records the user's voice response and uses the Google Cloud Speech-to-Text API to convert the voice data into text data (data processing). For example, the text "I'm a little tired today" is generated (output). This text data is sent to the server.

[0151] Step 3: Analyzing text data

[0152] Server: Receives text data sent from the terminal and analyzes the text using a generative AI model (e.g., GPT-4) (input and data processing). For example, it analyzes the content "I'm a little tired" and evaluates the user's health status.

[0153] Server: Based on the analysis results, the server applies frailty criteria and evaluates the user's health status (data calculation). The evaluation result is recorded in the database, for example, in the form of "Frailty Risk: Medium" (output).

[0154] Step 4: Data aggregation and calculation of evaluation scores

[0155] Server: Aggregates daily accumulated evaluation data to comprehensively assess the user's frailty progress. For example, it uses Python's Pandas and NumPy to detect trends such as "feeling tired more than twice a week" from data over the past month (input and data calculation).

[0156] Server: Based on the aggregated data, it calculates an evaluation score according to the prevention criteria. For example, a result such as "Frailty Risk Score: 70 / 100" might be obtained (output). This score is then notified to the user's family via email.

[0157] Step 5: Provide a reminder

[0158] Terminal: Based on evaluation results received from the server, it generates reminders for specific health behaviors and sends them as voice messages. For example, a reminder such as "It would be good to take a 10-minute walk today" is generated using the Google Cloud Text-to-Speech API (input and output).

[0159] User: Perform an action based on a reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[0160] Terminal: Records user feedback, converts it into text data, and sends it to the server (data processing and output).

[0161] Step 6: Analyze feedback and update the database

[0162] Server: Analyzes received feedback data (using a generative AI model) and updates the database (input and data calculation). For example, information that a user has taken a walk is added to the database.

[0163] Server: Reassess frailty risk based on updated data and create the next action plan as needed. This determines the content of the next reminder or notification (output).

[0164] This clearly outlines the overall system's processing flow, enabling the implementation of a program to continuously monitor the progress of frailty in older adults and provide appropriate care.

[0165] (Application Example 1)

[0166] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0167] This invention aims to prevent the progression of frailty and health risks, and support individual health maintenance, by monitoring the health status of elderly people and workers in real time and promoting appropriate health behaviors. In particular, it addresses the challenge of efficiently managing the health of elderly workers in work environments such as factories, and providing appropriate breaks and reminders to improve work efficiency and reduce health risks.

[0168] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0169] In this invention, the server includes means for communicating with elderly people or workers on a daily basis using an interactive speaker or robot equipped with generative artificial intelligence; means for converting voice data acquired by the interactive speaker or robot into text data and analyzing it based on frailty criteria or health risk criteria; means for evaluating the progress of frailty or health risk based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family or facility manager; means for providing health behavior reminders to elderly people or workers based on the evaluation results; and means for receiving feedback from elderly people or workers and updating the database with feedback data. This enables elderly people and workers to understand their own health status on a daily basis and take appropriate action.

[0170] "Generative artificial intelligence" is artificial intelligence designed to interact with humans using technologies such as natural language processing and machine learning.

[0171] An "interactive speaker" is a speaker device that can communicate with the user using voice.

[0172] A "robot" refers to a machine that operates automatically and can interact with and perform tasks with humans.

[0173] The term "elderly" generally refers to people aged 65 and over, and often includes those living after retirement.

[0174] "Workers" refer to people who are engaged in specific tasks in a specific workplace.

[0175] "Communication" is the process by which humans and machines exchange information and understand each other.

[0176] "Audio data" refers to digital data in which audio has been recorded.

[0177] "Text data" refers to digital data recorded as character information.

[0178] "Frailty criteria" are standards used to assess the health status of elderly people.

[0179] "Health risk criteria" are standards used to assess the health status of workers and the elderly and to determine the associated risks.

[0180] "Analysis" is the process of processing data and extracting useful information.

[0181] "Evaluation" is the process of judging a specific state or situation based on the results of an analysis.

[0182] A "database" is a system for structuring, collecting, and managing data.

[0183] "User's family" refers to anyone belonging to the household of a system user.

[0184] A "facility manager" is someone responsible for the working environment and the operation of the facility.

[0185] A "reminder" is a message that notifies a user about a specific action or event.

[0186] "Feedback" refers to responses and reactions from users, and is information that is used for subsequent processing and evaluation.

[0187] This invention relates to a system that uses an interactive speaker or robot to monitor the health status of the elderly and workers in real time and promote appropriate health behaviors. This system operates primarily with a server, a terminal (interactive speaker or robot), and a user.

[0188] 1. Initial setup and user registration

[0189] Terminal (Robot): The terminal provides an interface for registering the user's initial information. For example, it prompts the user to enter the name, age, gender, role, and daily work duties of a worker in a factory.

[0190] User: Follow the interface presented by the robot and enter your information. For example, register information such as "Ichiro Sato, 70 years old, male, line operator."

[0191] Server: Receives user information sent from the terminal and stores it in the database. This collects basic data for monitoring the user's health status.

[0192] 2. Everyday communication

[0193] Terminal (Robot): At scheduled times, it sends voice messages to workers asking about their physical condition and mood. For example, before starting work, it might ask, "Before we begin work, how are you feeling today?"

[0194] User: Responds to the device's questions using voice. For example, "My shoulders are a little stiff."

[0195] Terminal (robot): Records the user's voice response, converts it to text data using the Google Cloud Speech-to-Text API, and sends it to the server.

[0196] Server: Analyzes text data and assesses the user's health status based on health risk criteria. The analysis results are recorded in a database.

[0197] 3. Data accumulation and analysis

[0198] Server: Collects daily accumulated data and comprehensively evaluates workers' health risks. The analysis system uses TENSORFLOW®.

[0199] Server: Calculates a health risk score, and this score is shared with the facility administrator. For example, it notifies the administrator via email of an evaluation result such as "Health Risk Score: 60 / 100, Caution."

[0200] 4. Support and Reminders

[0201] Terminal (Robot): Based on evaluation results, it provides workers with reminders for appropriate health behaviors. For example, it might remind them, "It would be good to do 10 minutes of stretching."

[0202] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I did my stretches."

[0203] Terminal (robot): Receives user feedback and sends it to the server. The feedback data is updated in the database.

[0204] Server: Reassess health risks based on updated data and create the next action plan as needed.

[0205] Hardware and software configuration

[0206] Hardware used: A robot equipped with an Intel Core i7 processor, built-in microphone, speaker, and high-resolution camera.

[0207] Software used: Python, TensorFlow, Google Cloud Speech-to-Text API, and the Django framework.

[0208] Specific example

[0209] Examples of prompts for a generative AI model:

[0210] Robot: "Good morning, Mr. Sato. How are you feeling today?"

[0211] User: "My shoulders are a little stiff."

[0212] In response to this prompt, the AI ​​model sends the generated text data to the server and provides post-analysis reminders and health advice.

[0213] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0214] Step 1: The terminal (robot) prompts the user (worker) to enter initial information.

[0215] Input: User's name, age, gender, role, and daily tasks.

[0216] Data processing: Organize the input information into structured data.

[0217] Output: Structured initial information data.

[0218] Specific operation: The device prompts the user via display and voice, "Please enter your name, age, gender, role, and daily tasks," and the user enters the information.

[0219] Step 2: The terminal (robot) sends initial information to the server.

[0220] Input: Structured initial information data.

[0221] Data processing: Performs data transmission processing from the terminal to the server.

[0222] Output: Initial information stored on the server.

[0223] Specific operation: The device sends information obtained from the user to the server via an API and stores it there.

[0224] Step 3: The terminal (robot) sends a voice message at scheduled times asking about the user's physical condition and mood.

[0225] Input: The prompt text for the dialogue.

[0226] Data processing: Natural language generation processing using generative AI models.

[0227] Output: A voice message asking the user a question.

[0228] Specific action: The device will say aloud, "Before we begin, how are you feeling today?"

[0229] Step 4: The user responds to the device's questions using voice.

[0230] Input: User voice response.

[0231] Data processing: Recording and saving audio data.

[0232] Output: Recorded audio data.

[0233] Specific action: The user says, "My shoulders are a little stiff," and the device starts recording the voice.

[0234] Step 5: The terminal (robot) converts the voice data into text data and sends it to the server.

[0235] Input: Recorded audio data.

[0236] Data processing: Speech recognition and text conversion using the Google Cloud Speech-to-Text API.

[0237] Output: Converted text data.

[0238] Specific operation: Use a speech recognition API to convert the spoken phrase "My shoulders are a little stiff" into text and send it to the server.

[0239] Step 6: The server analyzes the text data and assesses the user's health status based on health risk criteria.

[0240] Input: Converted text data.

[0241] Data processing: Text data analysis and health status assessment using TensorFlow.

[0242] Output: Evaluation results.

[0243] Specific operation: The server receives the text "My shoulders are a little stiff" and performs an analysis based on health risk criteria.

[0244] Step 7: The server records the analysis results in the database.

[0245] Input: Analysis results.

[0246] Data processing: Writing to a database.

[0247] Output: Evaluation results recorded in the database.

[0248] Specific action: Save the analyzed health risk score to the database.

[0249] Step 8: The terminal (robot) provides a reminder to the user based on the evaluation results.

[0250] Input: Evaluation result.

[0251] Data processing: Reminder generation, natural language generation.

[0252] Output: Reminder voice message.

[0253] Specific action: The device will remind you, "It would be good to do 10 minutes of stretching."

[0254] Step 9: The user follows the reminder, takes the action, and reports it to the device.

[0255] Input: User voice report.

[0256] Data processing: Recording and saving audio data.

[0257] Output: Recorded feedback audio data.

[0258] Specific action: The user reports "Yes, I've stretched," and the device records this.

[0259] Step 10: The terminal (robot) sends feedback data to the server.

[0260] Input: Recorded feedback audio data.

[0261] Data processing: Text conversion using speech recognition, transmission to the server.

[0262] Output: Updated feedback text data sent to the server.

[0263] Specific operation: Convert the recorded audio data into text and send it to the server.

[0264] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0265] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[0266] 1. Initial setup and user registration

[0267] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, lifestyle, etc.

[0268] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter information such as "Taro Yamada, 65 years old, male, takes a walk every day."

[0269] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[0270] 2. Everyday communication and emotion recognition

[0271] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[0272] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0273] Device: Records the user's voice response and sends the voice data to the emotion engine to recognize emotions. After voice recognition, the emotion engine analyzes the user's emotions and detects fatigue and slight anxiety from a response such as "I'm a little tired today."

[0274] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in the database.

[0275] 3. Data accumulation and analysis

[0276] Server: It aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, by analyzing data from the past month, it detects a trend where users "feel tired more than twice a week" are experiencing increased "anxiety."

[0277] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data. For example, it might send an email to the family with an assessment result such as, "Frailty risk score: 70 / 100, caution. The user has recently shown a stronger tendency to feel anxious."

[0278] 4. Support and Reminders

[0279] Device: Based on evaluation results and recognized emotional states, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0280] User: Follow the reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[0281] Terminal: Receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[0282] Server: Re-evaluate the frail risk based on the updated data and create the next action plan if necessary. For example, propose specific actions such as "Increase the frequency of walking to three times a week".

[0283] Thus, by combining the emotion engine, the system of the present invention can provide advanced health support based on the user's emotional state. As a result, family members living separately can grasp the health status of the elderly and provide appropriate support, and the elderly themselves can also recognize their health status and practice effective health behaviors.

[0284] The processing flow will be described below.

[0285] Step 1:

[0286] Terminal: Display an interface for the user to input basic information such as name, age, gender, and lifestyle habits.

[0287] Specific operation: A message "Please enter your name" and a text box are displayed on the screen.

[0288] Step 2:

[0289] User: Enter their information in the input field of the terminal and press the send button.

[0290] Specific operation: The user enters "Yamada Taro" and clicks the send button. <00009able to

[0291] Step 3:

[0292] Terminal: Send the entered information to the server.

[0293] Specific operation: The terminal sends the user's information to the server as an HTTP request in text format.

[0294] Step 4:

[0295] Server: Save the received information in the database.

[0296] Specific operation: The server records in the database as "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Take a walk every day".

[0297] Step 5:

[0298] <​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​Specific actions: Analyze emotional data and text data to assess "mild fatigue" and record it in the database.

[0309] Step 9:

[0310] Server: Aggregates the data accumulated daily and comprehensively evaluates the user's frailty progress.

[0311] Specific actions: For example, by analyzing data from the past month, it detects a trend where people "feel tired more than twice a week" and simultaneously experience increased "anxiety."

[0312] Step 10:

[0313] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data.

[0314] Specific actions: For example, an email notification is sent to the family notifying them of the assessment result, such as, "Frailty risk score: 70 / 100, caution advised. The individual has recently shown a stronger tendency to experience anxiety."

[0315] Step 11:

[0316] Device: Provides users with health behavior reminders based on evaluation results and recognized emotional states.

[0317] Specific actions: For example, a voice message could be sent reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0318] Step 12:

[0319] User: Follow the reminder and report the results to the device.

[0320] Specific actions: For example, answering, "Yes, I went for a walk."

[0321] Step 13:

[0322] Terminal: Receives user feedback and sends it to the server.

[0323] Specific operation: Feedback data is sent from the terminal to the server.

[0324] Step 14:

[0325] Server: Saves updated feedback and sentiment data to the database and recalculates the frailty risk assessment.

[0326] Specific actions: Update the database and calculate the new frailty risk score.

[0327] The above describes the specific processing flow in the system of the present invention that incorporates an emotion engine. By recognizing the user's emotions, more advanced and personalized health support becomes possible.

[0328] (Example 2)

[0329] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0330] There is a need for a system that routinely monitors the health status of the elderly and detects and prevents the progression of frailty at an early stage. However, it is difficult for the elderly to accurately understand their own health status and report it to medical institutions and family members at the appropriate time. Furthermore, there are limited means of providing advanced health support that takes emotional states into account. To solve this problem, an easy-to-use interface and accurate data collection and analysis are necessary.

[0331] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for registering the user's initial information, means for receiving basic information entered by the user and storing it in a database, means for sending an audio message to the user at a specific time asking about their physical condition and mood, means for recording the user's voice response and sending it to an emotion engine to recognize emotions, means for converting the emotion data and audio data recognized by the emotion engine into text data and analyzing it based on frailty criteria, means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, means for sharing the recorded data with the user's family, means for providing the user with health behavior reminders based on the evaluation results, and means for receiving user feedback and updating the database with feedback data. This enables accurate monitoring of the health status of the elderly and promotion of preventive behaviors.

[0332] An "interface" is a means by which a user interacts with a device or system and inputs and outputs information.

[0333] "Basic information" refers to data that includes personal attributes such as the user's name, age, gender, and lifestyle.

[0334] A "voice message" is a question or instruction delivered by voice from an electronic device.

[0335] An "emotion engine" is a combination of software or hardware used to recognize and analyze a user's emotional state from voice or text data.

[0336] "Text data" refers to data obtained by converting audio data into written text.

[0337] "Frailty criteria" are standards or indicators used to assess the health status and physical function decline of older adults.

[0338] "Frailty progress" refers to the status indicating the progression of a user's health condition and decline in physical function.

[0339] A "database" is a system or software for efficiently storing, managing, and retrieving collected data.

[0340] "Family" refers to individuals who have a close relationship with the user, and usually means the user's relatives.

[0341] A "reminder" is a notification or reminder that serves to remind a user of a specific action or appointment.

[0342] "Feedback" is data used by users to report the results and their impressions to the system after they have taken some kind of action.

[0343] "Analysis" is the process of thoroughly examining and evaluating collected data to derive useful information.

[0344] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[0345] Initial setup and user registration

[0346] The terminal provides an interface for registering the user's initial information. This interface includes forms accessible via touch panel and voice input. The user enters basic information such as name, age, gender, and lifestyle. For example, they might enter "Taro Yamada, 65 years old, male, takes a walk every day."

[0347] The server receives user information sent from the terminal and stores it in a database. This collects basic data for monitoring the health status of individual users.

[0348] Everyday communication and emotion recognition

[0349] The device will send voice messages to the user at specific times asking about their health and mood. For example, every morning at 8:00 AM, it will send a voice message asking, "Good morning, how are you feeling today?"

[0350] The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today."

[0351] The device records the user's voice responses and sends that voice data to an emotion engine to recognize emotions. The emotion engine analyzes the user's voice and detects, for example, "fatigue" and "anxiety."

[0352] The server converts the emotion data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in a database.

[0353] Data accumulation and analysis

[0354] The server aggregates the data accumulated daily and comprehensively evaluates the user's progress towards frailty. For example, by analyzing data from the past month, it detects a trend where users are experiencing fatigue more than twice a week, while simultaneously showing an increase in anxiety.

[0355] The server calculates an assessment score based on frailty criteria and notifies the user's family of the assessment score and emotional data. For example, it might send an email notification stating, "Frailty risk score: 70 / 100, caution advised. The user has recently shown a stronger tendency to feel anxious."

[0356] Support and reminders

[0357] The device provides users with health behavior reminders based on evaluation results and recognized emotional states. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0358] The user acts according to the reminder and reports the result to the device. For example, they might respond, "Yes, I went for a walk."

[0359] The device receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[0360] The server reassesss the frailty risk based on the updated data and, if necessary, create the next action plan. For example, it might suggest specific actions such as, "Increase the frequency of walks to three times a week."

[0361] Examples and prompts for generative AI models

[0362] As a concrete example, a user enters "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day" into the interface on their device, and the server saves this information to a database. Furthermore, a voice message is sent from the device every morning at 8:00 AM saying, "Good morning, how are you feeling today?" to which the user replies, "I'm a little tired today."

[0363] Example prompts for generative AI models

[0364] Please describe the details of the system that monitors the user's health status. Include the following information:

[0365] Hardware and software to be used (e.g., conversational speaker with generative artificial intelligence, emotion engine)

[0366] Data collection methods (e.g., voice responses from users)

[0367] Analysis method (Example: Analyze voice data with an emotion engine, then analyze on a server based on frailty criteria)

[0368] Support services provided (e.g., health behavior reminders)

[0369] Specific example (Example: Flow of entering and saving initial user information)

[0370] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[0371] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0372] Processing flow and specific actions

[0373] Initial setup and user registration

[0374] Step 1:

[0375] Input: User's basic information (name, age, gender, lifestyle, etc.)

[0376] Function: The device provides an interface for registering initial information. Using a touch panel or voice input form, it prompts the user to enter basic information such as "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day."

[0377] Output: Input basic information

[0378] Step 2:

[0379] Input: Basic information entered in Step 1

[0380] Operation: The terminal sends the entered basic information to the server. The data is encrypted and transmitted over the network.

[0381] Output: Basic information sent to the server

[0382] Step 3:

[0383] Input: Basic information sent to the server in Step 2

[0384] Operation: The server receives the entered basic information and stores it in the database. The stored information is used as foundational data to monitor the health status of individual users.

[0385] Output: Basic information stored in the database

[0386] Everyday communication and emotion recognition

[0387] Step 4:

[0388] Input: Set a fixed time (e.g., every morning at 8:00).

[0389] Operation: The device will send a voice message to the user at a set time, asking about their health and mood. For example, it might send a message like, "Good morning, how are you feeling today?"

[0390] Output: Sending a voice message to the user

[0391] Step 5:

[0392] Input: User response to the voice message sent in Step 4

[0393] Operation: The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today." The device records this voice response.

[0394] Output: Recorded user voice data

[0395] Step 6:

[0396] Input: User voice data recorded in Step 5

[0397] Operation: The device sends recorded audio data to the emotion engine. The emotion engine analyzes this audio data and recognizes the user's emotional state. For example, it can detect "fatigue" and "anxiety."

[0398] Output: Analyzed sentiment data

[0399] Step 7:

[0400] Input: Emotional data and voice data analyzed in Step 6

[0401] Operation: The server converts emotional and audio data into text data and analyzes it based on frailty criteria. The results are recorded in a database.

[0402] Output: Frailty analysis results stored in the database

[0403] Data accumulation and analysis

[0404] Step 8:

[0405] Input: Frailty analysis results saved in the database in Step 7

[0406] Operation: The server aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, it might detect a trend from the past month's data showing that users are experiencing fatigue more than twice a week, while simultaneously experiencing increased anxiety.

[0407] Output: Evaluation results of frailty progress

[0408] Step 9:

[0409] Input: Evaluation results of the frailty progress obtained in Step 8

[0410] Operation: The server calculates an assessment score based on frailty criteria. For example, it might assess the user as having a "frailty risk score of 70 / 100". This assessment result is then communicated to the user's family.

[0411] Output: Evaluation results and notification message

[0412] Support and reminders

[0413] Step 10:

[0414] Input: Evaluation results and perceived emotional states obtained in Step 9

[0415] Operation: The device provides the user with health behavior reminders based on the evaluation results. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0416] Output: Health behavior reminders

[0417] Step 11:

[0418] Input: User response to the reminder issued in Step 10

[0419] Operation: The user acts according to a reminder and reports the result to the device. For example, they might provide feedback such as, "Yes, I went for a walk." The device records this feedback.

[0420] Output: Recorded feedback data

[0421] Step 12:

[0422] Input: Feedback data recorded in step 11

[0423] Operation: The device sends user feedback data to the server.

[0424] Output: Feedback data sent to the server

[0425] Step 13:

[0426] Input: Feedback data sent to the server in step 12

[0427] Operation: The server updates the database with the feedback data it receives. Based on the updated data, it reassesss the risk of frailty and creates the next action plan. For example, it might suggest, "Increase the frequency of walks to three times a week."

[0428] Output: Updated data and next action plan

[0429] Through the specific processing steps described above, this system can accurately monitor the health status of elderly individuals and promote appropriate health behaviors.

[0430] (Application Example 2)

[0431] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0432] Monitoring the progression of frailty in older adults and suggesting appropriate health behaviors is crucial, but simply analyzing health status and monitoring emotional state is insufficient. In particular, nutritional status significantly impacts health in older adults, requiring comprehensive health support that includes nutritional assistance. Furthermore, not only are health behavior reminders necessary, but also means to implement specific dietary suggestions.

[0433] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for communicating with the elderly on a daily basis using an interactive speaker equipped with generating artificial intelligence; means for converting voice data acquired by the interactive speaker into text data and analyzing it based on frailty criteria; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family; means for providing the elderly with health behavior reminders based on the evaluation results; means for receiving feedback from the elderly and updating the feedback data in the database; means for generating meal suggestions that support the nutritional status of the elderly based on the analysis results and emotional state; and means for providing the meal suggestions to the elderly as specific menus in cooperation with a food delivery system. This enables comprehensive support for frailty prevention and nutrition.

[0434] "Generative artificial intelligence" is a type of artificial intelligence that learns from large amounts of data and has the ability to engage in natural conversations and make decisions like a human.

[0435] An "interactive speaker" is a speaker device that has the function of receiving voice input from the user and responding with voice output.

[0436] "Frailty criteria" are standards used to assess the progression of frailty in older adults based on their physical strength and health status.

[0437] "Analysis results" refer to conclusions and evaluation data obtained based on the analysis of voice data and emotional states.

[0438] A "database" is an information storage system for systematically accumulating and managing large amounts of data.

[0439] A "health behavior reminder" is a notification or alert designed to encourage users to engage in healthy behaviors.

[0440] "Feedback from the elderly" refers to information and opinions provided by elderly individuals regarding their behavior and physical condition.

[0441] "Meal suggestions to support nutritional status" refer to meal menus recommended based on the health status and nutritional needs of elderly individuals.

[0442] A "food delivery system" is a service system that delivers meals ordered by users to a specified location.

[0443] This invention is a system that uses an interactive speaker equipped with artificial intelligence to monitor the progress of frailty in elderly individuals and provides comprehensive health support, including nutritional support. Specific embodiments are described below.

[0444] Hardware and software to be used

[0445] Hardware: Smartphones, smart speakers

[0446] Software: Emotion engine (e.g., Microsoft® Azure® Emotion API), natural language processing engine (e.g., OpenAI® GPT-3®), food delivery API

[0447] System Configuration

[0448] 1. User Registration

[0449] The terminal provides an interface that allows the user to input basic information such as name, age, gender, and lifestyle. The information entered by the user is sent to the server and stored in a database.

[0450] 2. Daily communication and monitoring of health status

[0451] At specific times, the device sends a voice message to the user asking about their health and mood. For example, every morning at 8:00 AM, it might ask, "Good morning. How are you feeling today?" The user's voice response is recorded by an interactive speaker and sent to a server. The recorded voice data is converted to text and analyzed using an emotion engine.

[0452] 3. Accumulation and evaluation of analysis results

[0453] The converted text and sentiment data are stored on the server, and the progress of frailty is evaluated based on past data. The evaluation results are stored in a database and shared with the user's family.

[0454] 4. Generating and providing meal suggestions

[0455] Based on the analysis results and emotional state, meal suggestions are generated to support the user's nutritional status. These meal suggestions are integrated with a food delivery system and provided as specific menus. For example, if fatigue or stress levels are high, a balanced and nutritious menu will be suggested.

[0456] example

[0457] For example, if a 65-year-old male user answers "I'm a little tired today" during his daily health check, the voice data is analyzed through an emotion engine. As a result, fatigue and mild anxiety are detected. The server analyzes the data from the past month and determines that the user's fatigue is on an increasing trend. As a result, a nutritionally balanced "chicken and vegetable plate" is suggested for lunch and made available for order through a food delivery system.

[0458] Example of a prompt

[0459] Check the user's daily physical and emotional state and analyze the data weekly. Then, use an emotion engine to assess increases and decreases in fatigue and anxiety, suggest a balanced meal, and display that menu in a form that can be ordered via food delivery.

[0460] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0461] Step 1:

[0462] Input: The user enters basic information such as name, age, gender, and lifestyle into the terminal.

[0463] Operation: The terminal collects the entered information and sends it to the server.

[0464] Output: The server saves the received user information to the database.

[0465] Step 2:

[0466] Input: The device will send an audio message at a specific time asking about the elderly person's physical condition and mood.

[0467] Operation: The user responds by voice about their physical condition and mood. For example, they might say, "I'm a little tired today."

[0468] Output: The terminal records the user's voice response and sends the audio data to the server.

[0469] Step 3:

[0470] Input: The server sends the received audio data to the emotion engine.

[0471] Operation: The emotion engine analyzes audio data and identifies emotional states.

[0472] Output: The server receives the analysis results from the emotion engine and saves them to the database as text data and emotion data.

[0473] Step 4:

[0474] Input: The server aggregates and analyzes daily voice data and emotion data stored in the database.

[0475] Function: By analyzing data, the system evaluates the user's progress in frailty.

[0476] Output: The server calculates the evaluation result as a frailty score and records it in the database.

[0477] Step 5:

[0478] Input: The server analyzes the user's health trends based on frailty score and emotional state data.

[0479] Operation: Uses a natural language processing engine to generate meal suggestions to support the user's nutritional status.

[0480] Output: The server uses a generation AI model to generate individually customized meal suggestions.

[0481] Step 6:

[0482] Input: The server integrates with the food delivery system to provide the generated meal suggestions to the user.

[0483] Operation: The device notifies the user of specific menu options via voice or text.

[0484] Output: The user reviews the suggested menu and places an order through the food delivery service.

[0485] Step 7:

[0486] Input: Users input feedback into the terminal regarding the results of their meals and any changes in their physical condition.

[0487] Operation: Feedback data is sent from the terminal to the server.

[0488] Output: The server updates the database with feedback data and incorporates it into the next analysis.

[0489] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0490] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0491] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0492] [Second Embodiment]

[0493] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0494] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0495] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0496] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0497] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0498] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0499] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0500] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0501] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0502] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0503] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0504] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0505] This invention relates to a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. This system operates primarily with a server, terminals, and users.

[0506] 1. Initial setup and user registration

[0507] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, daily habits, etc.

[0508] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter "Taro Yamada, 65 years old, male, takes a walk every day."

[0509] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[0510] 2. Everyday communication

[0511] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[0512] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0513] Terminal: Records the user's voice response, converts it to text data, and sends it to the server. Using speech recognition technology, it generates text data from the voice statement, "I'm a little tired today."

[0514] Server: Analyzes text data and assesses the user's health status based on frailty criteria. The analysis results are recorded in a database.

[0515] 3. Data accumulation and analysis

[0516] Server: Aggregates data accumulated daily to comprehensively evaluate the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week."

[0517] Server: Calculates an assessment score based on frailty criteria. This score is shared with the user's family. For example, an assessment result such as "Frailty Risk Score: 70 / 100, Caution" is sent to the user's family via email.

[0518] 4. Support and Reminders

[0519] Device: Based on evaluation results, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today."

[0520] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I went for a walk."

[0521] Terminal: Receives user feedback and sends it to the server. The feedback data is updated in the database.

[0522] Server: Reassess the frailty risk based on the updated data and create the next action plan as needed.

[0523] As described above, the system of the present invention supports the maintenance of the health of the elderly by having the server, terminal, and user work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice.

[0524] The following describes the processing flow.

[0525] Step 1:

[0526] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[0527] Specific action: The screen displays the message "Please enter your name" and a text box.

[0528] Step 2:

[0529] User: Enter your information in the terminal's input field and press the submit button.

[0530] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[0531] Step 3:

[0532] Terminal: Sends the entered information to the server.

[0533] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[0534] Step 4:

[0535] Server: Stores received information in the database.

[0536] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[0537] Step 5:

[0538] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[0539] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[0540] Step 6:

[0541] User: Responds to the device's questions using voice.

[0542] Specific action: For example, the user responds, "I'm a little tired today."

[0543] Step 7:

[0544] Terminal: Records the user's voice response, converts it to text data, and sends it to the server.

[0545] Specific operation: Using speech recognition technology, the phrase "I'm a little tired today" is converted to text and sent to the server.

[0546] Step 8:

[0547] Server: Analyzes text data and evaluates the user's health status based on frailty criteria.

[0548] Specific actions: Analyze text data, evaluate it as "mild fatigue," and record it in the database.

[0549] Step 9:

[0550] Server: Collects data accumulated daily and evaluates the user's progress in frailty.

[0551] Specific actions: For example, aggregating data from the past month and detecting trends such as "feeling tired more than twice a week."

[0552] Step 10:

[0553] Server: Calculates an assessment score based on frailty criteria and notifies the user's family.

[0554] Specific actions: For example, notify family members via email of the result, "Frailty risk score: 70 / 100, caution advised."

[0555] Step 11:

[0556] Device: Provides users with health behavior reminders based on evaluation results.

[0557] Specific actions: For example, a voice reminder saying, "It would be good to take a 10-minute walk today."

[0558] Step 12:

[0559] User: Follow the reminder and report to the device whether or not the action was performed.

[0560] Specific actions: For example, answering, "Yes, I went for a walk."

[0561] Step 13:

[0562] Terminal: Receives user feedback and sends it to the server.

[0563] Specific operation: Feedback data is sent from the terminal to the server.

[0564] Step 14:

[0565] Server: Saves updated feedback data to the database and recalculates the frailty risk assessment.

[0566] Specific actions: Update the database and calculate the new frailty risk score.

[0567] The above describes the specific processing flow in the system of the present invention. This process makes it possible to continuously monitor the progress of frailty in elderly individuals and provide appropriate support.

[0568] (Example 1)

[0569] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0570] In modern society, understanding the health status of the elderly and providing appropriate care is a crucial challenge. In particular, there is a need to detect and prevent the progression of frailty in the elderly at an early stage. However, current systems lack the means to monitor the progression of frailty in the elderly through daily communication and to provide appropriate alerts and reminders. As a result, the progression of frailty may be overlooked, posing a risk of a decline in the quality of life for the elderly.

[0571] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0572] In this invention, the server includes means for communicating with elderly people on a daily basis using a generative AI model; means for converting voice data acquired by an interactive voice output device into text data and analyzing it; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing it with family members; means for providing reminders for healthy behaviors; means for receiving and updating feedback; means for recording and analyzing voice responses; and means for aggregating data and calculating a score. This makes it possible to continuously monitor the progress of frailty in elderly people and provide appropriate care.

[0573] "Generative artificial intelligence" is a type of artificial intelligence that uses natural language processing and machine learning models to interact with users and perform data analysis.

[0574] An "interactive voice output device" is a device that communicates with the user through voice, and smart speakers are an example of such devices.

[0575] The term "elderly person" generally refers to a person aged 65 or older.

[0576] "Frailty criteria" are standards used to assess the health status and physical decline of older adults, and are determined based on medical evaluation indicators.

[0577] "Text data" refers to data such as audio and images that are represented as a string of characters.

[0578] A "database" is a system for efficiently storing, managing, and retrieving data.

[0579] "Healthy behaviors" refer to actions that older adults should take to prevent disease and maintain their health.

[0580] A "reminder" is a notification or message sent to a user to prompt them to take a specific action.

[0581] "Feedback" refers to the information that users provide to a system regarding their actions and status.

[0582] "Speech recognition technology" is a technology that analyzes speech data and converts it into text data.

[0583] An "evaluation score" is a numerical representation of the analysis results, used to quantitatively assess the user's condition.

[0584] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze data and is trained to perform a specific task.

[0585] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on the tasks or analyses it should perform.

[0586] This invention is a system for monitoring and preventing frailty in the elderly using an interactive voice output device equipped with a generative AI model. This system operates primarily with a server, terminals, and users.

[0587] 1. Initial setup and user registration

[0588] Terminal: Provides an interface for registering the user's initial information. For example, an application installed on a tablet or smartphone is used. The application is developed in Swift for iOS devices and in Kotlin for Android devices, for example. The user enters basic information such as name, age, gender, and daily habits. This data is sent to the server in JSON format via the HTTPS protocol.

[0589] Server: Receives user information sent from terminals, analyzes it, and stores it in a database. Cloud platforms such as Amazon RDS and Google Cloud Firestore are used as the database. The data collected here will later serve as foundational data for evaluating the health status of elderly individuals.

[0590] 2. Everyday communication

[0591] Device: A scheduled task is triggered at a specific time, such as 8 AM every morning, to broadcast a voice message saying, "Good morning, how are you feeling today?" This voice message is generated using the Google Cloud Text-to-Speech API.

[0592] User: Responds to the device's questions using voice. For example, responds, "I'm a little tired today."

[0593] Terminal: Records the user's voice response and converts the audio data into text data using the Google Cloud Speech-to-Text API. This response, for example, "I'm a little tired today," is then sent to the server.

[0594] Server: Analyzes received text data and evaluates the user's health status based on frailty criteria. Natural language processing models such as GPT-4 and BERT are used as generative AI models for the analysis. The analysis results are recorded in a database.

[0595] 3. Data accumulation and analysis

[0596] Server: Collects daily data and comprehensively evaluates the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week." Data collection and analysis are performed using Python's Pandas and NumPy.

[0597] Server: Calculates an assessment score based on frailty criteria. For example, an assessment result such as "Frailty Risk Score: 70 / 100" is shared with the user's family using an email sending service such as SendGrid or AWS SES.

[0598] 4. Support and Reminders

[0599] Device: Based on evaluation results, it provides elderly individuals with reminders for appropriate health behaviors. For example, an interactive voice output device might send a voice message saying, "It would be good to take a 10-minute walk today." This voice message can also be generated using the Google Cloud Text-to-Speech API.

[0600] User: Performs an action based on the reminder and reports to the device, "Yes, I went for a walk." This confirms the effectiveness of the reminder.

[0601] Terminal: Receives user feedback and sends it to the server. Feedback data is sent in JSON format and updated in the database.

[0602] Server: Based on the received feedback data, reassess the frailty risk and create the next action plan as needed.

[0603] Specific example

[0604] Here are some examples of specific prompt messages:

[0605] "Analyze the reasons why users reported feeling tired yesterday and propose an appropriate action plan."

[0606] "Based on the user's daily communication data from the past week, please calculate their frailty risk score."

[0607] "Assess the user's overall health status and generate reminders for improvement."

[0608] In this way, a system is realized that supports the maintenance of the health of the elderly by having terminals, servers, and users work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice and support.

[0609] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0610] Step 1: Start the initial setup

[0611] Terminal: Launches the application and displays the user interface. The user enters basic information such as name, age, gender, and daily habits. The input data is temporarily stored within the terminal as a JSON object (input).

[0612] Server: When the user presses the submit button, the device sends this JSON data to the server using the HTTPS protocol. The server parses the received data and saves it to an Amazon RDS database (output). This registers the user's basic information as initial settings.

[0613] Step 2: Start daily communication

[0614] Terminal: At a set time, for example, every morning at 8:00 AM, a scheduled task is triggered, generating and sending a voice message saying "Good morning, how are you feeling today?" using the Google Cloud Text-to-Speech API (input and output).

[0615] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0616] Terminal: Records the user's voice response and uses the Google Cloud Speech-to-Text API to convert the voice data into text data (data processing). For example, the text "I'm a little tired today" is generated (output). This text data is sent to the server.

[0617] Step 3: Analyzing text data

[0618] Server: Receives text data sent from the terminal and analyzes the text using a generative AI model (e.g., GPT-4) (input and data processing). For example, it analyzes the content "I'm a little tired" and evaluates the user's health status.

[0619] Server: Based on the analysis results, the server applies frailty criteria and evaluates the user's health status (data calculation). The evaluation result is recorded in the database, for example, in the form of "Frailty Risk: Medium" (output).

[0620] Step 4: Data aggregation and calculation of evaluation scores

[0621] Server: Aggregates daily accumulated evaluation data to comprehensively assess the user's frailty progress. For example, it uses Python's Pandas and NumPy to detect trends such as "feeling tired more than twice a week" from data over the past month (input and data calculation).

[0622] Server: Based on the aggregated data, it calculates an evaluation score according to the prevention criteria. For example, a result such as "Frailty Risk Score: 70 / 100" might be obtained (output). This score is then notified to the user's family via email.

[0623] Step 5: Provide a reminder

[0624] Terminal: Based on evaluation results received from the server, it generates reminders for specific health behaviors and sends them as voice messages. For example, a reminder such as "It would be good to take a 10-minute walk today" is generated using the Google Cloud Text-to-Speech API (input and output).

[0625] User: Perform an action based on a reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[0626] Terminal: Records user feedback, converts it into text data, and sends it to the server (data processing and output).

[0627] Step 6: Analyze feedback and update the database

[0628] Server: Analyzes received feedback data (using a generative AI model) and updates the database (input and data calculation). For example, information that a user has taken a walk is added to the database.

[0629] Server: Reassess frailty risk based on updated data and create the next action plan as needed. This determines the content of the next reminder or notification (output).

[0630] This clearly outlines the overall system's processing flow, enabling the implementation of a program to continuously monitor the progress of frailty in older adults and provide appropriate care.

[0631] (Application Example 1)

[0632] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0633] This invention aims to prevent the progression of frailty and health risks, and support individual health maintenance, by monitoring the health status of elderly people and workers in real time and promoting appropriate health behaviors. In particular, it addresses the challenge of efficiently managing the health of elderly workers in work environments such as factories, and providing appropriate breaks and reminders to improve work efficiency and reduce health risks.

[0634] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0635] In this invention, the server includes means for communicating with elderly people or workers on a daily basis using an interactive speaker or robot equipped with generative artificial intelligence; means for converting voice data acquired by the interactive speaker or robot into text data and analyzing it based on frailty criteria or health risk criteria; means for evaluating the progress of frailty or health risk based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family or facility manager; means for providing health behavior reminders to elderly people or workers based on the evaluation results; and means for receiving feedback from elderly people or workers and updating the database with feedback data. This enables elderly people and workers to understand their own health status on a daily basis and take appropriate action.

[0636] "Generative artificial intelligence" is artificial intelligence designed to interact with humans using technologies such as natural language processing and machine learning.

[0637] An "interactive speaker" is a speaker device that can communicate with the user using voice.

[0638] A "robot" refers to a machine that operates automatically and can interact with and perform tasks with humans.

[0639] The term "elderly" generally refers to people aged 65 and over, and often includes those living after retirement.

[0640] "Workers" refer to people who are engaged in specific tasks in a specific workplace.

[0641] "Communication" is the process by which humans and machines exchange information and understand each other.

[0642] "Audio data" refers to digital data in which audio has been recorded.

[0643] "Text data" refers to digital data recorded as character information.

[0644] "Frailty criteria" are standards used to assess the health status of elderly people.

[0645] "Health risk criteria" are standards used to assess the health status of workers and the elderly and to determine the associated risks.

[0646] "Analysis" is the process of processing data and extracting useful information.

[0647] "Evaluation" is the process of judging a specific state or situation based on the results of an analysis.

[0648] A "database" is a system for structuring, collecting, and managing data.

[0649] "User's family" refers to anyone belonging to the household of a system user.

[0650] A "facility manager" is someone responsible for the working environment and the operation of the facility.

[0651] A "reminder" is a message that notifies a user about a specific action or event.

[0652] "Feedback" refers to responses and reactions from users, and is information that is used for subsequent processing and evaluation.

[0653] This invention relates to a system that uses an interactive speaker or robot to monitor the health status of the elderly and workers in real time and promote appropriate health behaviors. This system operates primarily with a server, a terminal (interactive speaker or robot), and a user.

[0654] 1. Initial setup and user registration

[0655] Terminal (Robot): The terminal provides an interface for registering the user's initial information. For example, it prompts the user to enter the name, age, gender, role, and daily work duties of a worker in a factory.

[0656] User: Follow the interface presented by the robot and enter your information. For example, register information such as "Ichiro Sato, 70 years old, male, line operator."

[0657] Server: Receives user information sent from the terminal and stores it in the database. This collects basic data for monitoring the user's health status.

[0658] 2. Everyday communication

[0659] Terminal (Robot): At scheduled times, it sends voice messages to workers asking about their physical condition and mood. For example, before starting work, it might ask, "Before we begin work, how are you feeling today?"

[0660] User: Responds to the device's questions using voice. For example, "My shoulders are a little stiff."

[0661] Terminal (robot): Records the user's voice response, converts it to text data using the Google Cloud Speech-to-Text API, and sends it to the server.

[0662] Server: Analyzes text data and assesses the user's health status based on health risk criteria. The analysis results are recorded in a database.

[0663] 3. Data accumulation and analysis

[0664] Server: Collects daily accumulated data and comprehensively evaluates workers' health risks. TensorFlow is used for the analysis system.

[0665] Server: Calculates a health risk score, and this score is shared with the facility administrator. For example, it notifies the administrator via email of an evaluation result such as "Health Risk Score: 60 / 100, Caution."

[0666] 4. Support and Reminders

[0667] Terminal (Robot): Based on evaluation results, it provides workers with reminders for appropriate health behaviors. For example, it might remind them, "It would be good to do 10 minutes of stretching."

[0668] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I did my stretches."

[0669] Terminal (robot): Receives user feedback and sends it to the server. The feedback data is updated in the database.

[0670] Server: Reassess health risks based on updated data and create the next action plan as needed.

[0671] Hardware and software configuration

[0672] Hardware used: A robot equipped with an Intel Core i7 processor, built-in microphone, speaker, and high-resolution camera.

[0673] Software used: Python, TensorFlow, Google Cloud Speech-to-Text API, and the Django framework.

[0674] Specific example

[0675] Examples of prompts for a generative AI model:

[0676] Robot: "Good morning, Mr. Sato. How are you feeling today?"

[0677] User: "My shoulders are a little stiff."

[0678] In response to this prompt, the AI ​​model sends the generated text data to the server and provides post-analysis reminders and health advice.

[0679] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0680] Step 1: The terminal (robot) prompts the user (worker) to enter initial information.

[0681] Input: User's name, age, gender, role, and daily tasks.

[0682] Data processing: Organize the input information into structured data.

[0683] Output: Structured initial information data.

[0684] Specific operation: The device prompts the user via display and voice, "Please enter your name, age, gender, role, and daily tasks," and the user enters the information.

[0685] Step 2: The terminal (robot) sends initial information to the server.

[0686] Input: Structured initial information data.

[0687] Data processing: Performs data transmission processing from the terminal to the server.

[0688] Output: Initial information stored on the server.

[0689] Specific operation: The device sends information obtained from the user to the server via an API and stores it there.

[0690] Step 3: The terminal (robot) sends a voice message at scheduled times asking about the user's physical condition and mood.

[0691] Input: The prompt text for the dialogue.

[0692] Data processing: Natural language generation processing using generative AI models.

[0693] Output: A voice message asking the user a question.

[0694] Specific action: The device will say aloud, "Before we begin, how are you feeling today?"

[0695] Step 4: The user responds to the device's questions using voice.

[0696] Input: User voice response.

[0697] Data processing: Recording and saving audio data.

[0698] Output: Recorded audio data.

[0699] Specific action: The user says, "My shoulders are a little stiff," and the device starts recording the voice.

[0700] Step 5: The terminal (robot) converts the voice data into text data and sends it to the server.

[0701] Input: Recorded audio data.

[0702] Data processing: Speech recognition and text conversion using the Google Cloud Speech-to-Text API.

[0703] Output: Converted text data.

[0704] Specific operation: Use a speech recognition API to convert the spoken phrase "My shoulders are a little stiff" into text and send it to the server.

[0705] Step 6: The server analyzes the text data and assesses the user's health status based on health risk criteria.

[0706] Input: Converted text data.

[0707] Data processing: Text data analysis and health status assessment using TensorFlow.

[0708] Output: Evaluation results.

[0709] Specific operation: The server receives the text "My shoulders are a little stiff" and performs an analysis based on health risk criteria.

[0710] Step 7: The server records the analysis results in the database.

[0711] Input: Analysis results.

[0712] Data processing: Writing to a database.

[0713] Output: Evaluation results recorded in the database.

[0714] Specific action: Save the analyzed health risk score to the database.

[0715] Step 8: The terminal (robot) provides a reminder to the user based on the evaluation results.

[0716] Input: Evaluation result.

[0717] Data processing: Reminder generation, natural language generation.

[0718] Output: Reminder voice message.

[0719] Specific action: The device will remind you, "It would be good to do 10 minutes of stretching."

[0720] Step 9: The user follows the reminder, takes the action, and reports it to the device.

[0721] Input: User voice report.

[0722] Data processing: Recording and saving audio data.

[0723] Output: Recorded feedback audio data.

[0724] Specific action: The user reports "Yes, I've stretched," and the device records this.

[0725] Step 10: The terminal (robot) sends feedback data to the server.

[0726] Input: Recorded feedback audio data.

[0727] Data processing: Text conversion using speech recognition, transmission to the server.

[0728] Output: Updated feedback text data sent to the server.

[0729] Specific operation: Convert the recorded audio data into text and send it to the server.

[0730] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0731] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[0732] 1. Initial setup and user registration

[0733] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, lifestyle, etc.

[0734] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter information such as "Taro Yamada, 65 years old, male, takes a walk every day."

[0735] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[0736] 2. Everyday communication and emotion recognition

[0737] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[0738] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0739] Device: Records the user's voice response and sends the voice data to the emotion engine to recognize emotions. After voice recognition, the emotion engine analyzes the user's emotions and detects fatigue and slight anxiety from a response such as "I'm a little tired today."

[0740] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in the database.

[0741] 3. Data accumulation and analysis

[0742] Server: It aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, by analyzing data from the past month, it detects a trend where users "feel tired more than twice a week" are experiencing increased "anxiety."

[0743] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data. For example, it might send an email to the family with an assessment result such as, "Frailty risk score: 70 / 100, caution. The user has recently shown a stronger tendency to feel anxious."

[0744] 4. Support and Reminders

[0745] Device: Based on evaluation results and recognized emotional states, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0746] User: Follow the reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[0747] Terminal: Receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[0748] Server: Based on the updated data, reassess the risk of frailty and create the next action plan as needed. For example, it might suggest specific actions such as "increase the frequency of walks to three times a week."

[0749] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[0750] The following describes the processing flow.

[0751] Step 1:

[0752] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[0753] Specific action: The screen displays the message "Please enter your name" and a text box.

[0754] Step 2:

[0755] User: Enter your information in the terminal's input field and press the submit button.

[0756] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[0757] Step 3:

[0758] Terminal: Sends the entered information to the server.

[0759] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[0760] Step 4:

[0761] Server: Stores received information in the database.

[0762] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[0763] Step 5:

[0764] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[0765] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[0766] Step 6:

[0767] User: Responds to the device's questions using voice.

[0768] Specific action: For example, the user responds, "I'm a little tired today."

[0769] Step 7:

[0770] Terminal: Records the user's voice response and sends that voice data to the emotion engine to recognize emotions.

[0771] Specific operation: Voice data is sent to the emotion engine, and the emotion recognition algorithm detects the emotions of "fatigue" and "slight anxiety."

[0772] Step 8:

[0773] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria.

[0774] Specific actions: Analyze emotional data and text data to assess "mild fatigue" and record it in the database.

[0775] Step 9:

[0776] Server: Aggregates the data accumulated daily and comprehensively evaluates the user's frailty progress.

[0777] Specific actions: For example, by analyzing data from the past month, it detects a trend where people "feel tired more than twice a week" and simultaneously experience increased "anxiety."

[0778] Step 10:

[0779] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data.

[0780] Specific actions: For example, an email notification is sent to the family notifying them of the assessment result, such as, "Frailty risk score: 70 / 100, caution advised. The individual has recently shown a stronger tendency to experience anxiety."

[0781] Step 11:

[0782] Device: Provides users with health behavior reminders based on evaluation results and recognized emotional states.

[0783] Specific actions: For example, a voice message could be sent reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0784] Step 12:

[0785] User: Follow the reminder and report the results to the device.

[0786] Specific actions: For example, answering, "Yes, I went for a walk."

[0787] Step 13:

[0788] Terminal: Receives user feedback and sends it to the server.

[0789] Specific operation: Feedback data is sent from the terminal to the server.

[0790] Step 14:

[0791] Server: Saves updated feedback and sentiment data to the database and recalculates the frailty risk assessment.

[0792] Specific actions: Update the database and calculate the new frailty risk score.

[0793] The above describes the specific processing flow in the system of the present invention that incorporates an emotion engine. By recognizing the user's emotions, more advanced and personalized health support becomes possible.

[0794] (Example 2)

[0795] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0796] There is a need for a system that routinely monitors the health status of the elderly and detects and prevents the progression of frailty at an early stage. However, it is difficult for the elderly to accurately understand their own health status and report it to medical institutions and family members at the appropriate time. Furthermore, there are limited means of providing advanced health support that takes emotional states into account. To solve this problem, an easy-to-use interface and accurate data collection and analysis are necessary.

[0797] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for registering the user's initial information, means for receiving basic information entered by the user and storing it in a database, means for sending an audio message to the user at a specific time asking about their physical condition and mood, means for recording the user's voice response and sending it to an emotion engine to recognize emotions, means for converting the emotion data and audio data recognized by the emotion engine into text data and analyzing it based on frailty criteria, means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, means for sharing the recorded data with the user's family, means for providing the user with health behavior reminders based on the evaluation results, and means for receiving user feedback and updating the database with feedback data. This enables accurate monitoring of the health status of the elderly and promotion of preventive behaviors.

[0798] An "interface" is a means by which a user interacts with a device or system and inputs and outputs information.

[0799] "Basic information" refers to data that includes personal attributes such as the user's name, age, gender, and lifestyle.

[0800] A "voice message" is a question or instruction delivered by voice from an electronic device.

[0801] An "emotion engine" is a combination of software or hardware used to recognize and analyze a user's emotional state from voice or text data.

[0802] "Text data" refers to data obtained by converting audio data into written text.

[0803] "Frailty criteria" are standards or indicators used to assess the health status and physical function decline of older adults.

[0804] "Frailty progress" refers to the status indicating the progression of a user's health condition and decline in physical function.

[0805] A "database" is a system or software for efficiently storing, managing, and retrieving collected data.

[0806] "Family" refers to individuals who have a close relationship with the user, and usually means the user's relatives.

[0807] A "reminder" is a notification or reminder that serves to remind a user of a specific action or appointment.

[0808] "Feedback" is data used by users to report the results and their impressions to the system after they have taken some kind of action.

[0809] "Analysis" is the process of thoroughly examining and evaluating collected data to derive useful information.

[0810] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[0811] Initial setup and user registration

[0812] The terminal provides an interface for registering the user's initial information. This interface includes forms accessible via touch panel and voice input. The user enters basic information such as name, age, gender, and lifestyle. For example, they might enter "Taro Yamada, 65 years old, male, takes a walk every day."

[0813] The server receives user information sent from the terminal and stores it in a database. This collects basic data for monitoring the health status of individual users.

[0814] Everyday communication and emotion recognition

[0815] The device will send voice messages to the user at specific times asking about their health and mood. For example, every morning at 8:00 AM, it will send a voice message asking, "Good morning, how are you feeling today?"

[0816] The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today."

[0817] The device records the user's voice responses and sends that voice data to an emotion engine to recognize emotions. The emotion engine analyzes the user's voice and detects, for example, "fatigue" and "anxiety."

[0818] The server converts the emotion data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in a database.

[0819] Data accumulation and analysis

[0820] The server aggregates the data accumulated daily and comprehensively evaluates the user's progress towards frailty. For example, by analyzing data from the past month, it detects a trend where users are experiencing fatigue more than twice a week, while simultaneously showing an increase in anxiety.

[0821] The server calculates an assessment score based on frailty criteria and notifies the user's family of the assessment score and emotional data. For example, it might send an email notification stating, "Frailty risk score: 70 / 100, caution advised. The user has recently shown a stronger tendency to feel anxious."

[0822] Support and reminders

[0823] The device provides users with health behavior reminders based on evaluation results and recognized emotional states. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0824] The user acts according to the reminder and reports the result to the device. For example, they might respond, "Yes, I went for a walk."

[0825] The device receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[0826] The server reassesss the frailty risk based on the updated data and, if necessary, create the next action plan. For example, it might suggest specific actions such as, "Increase the frequency of walks to three times a week."

[0827] Examples and prompts for generative AI models

[0828] As a concrete example, a user enters "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day" into the interface on their device, and the server saves this information to a database. Furthermore, a voice message is sent from the device every morning at 8:00 AM saying, "Good morning, how are you feeling today?" to which the user replies, "I'm a little tired today."

[0829] Example prompts for generative AI models

[0830] Please describe the details of the system that monitors the user's health status. Include the following information:

[0831] Hardware and software to be used (e.g., conversational speaker with generative artificial intelligence, emotion engine)

[0832] Data collection methods (e.g., voice responses from users)

[0833] Analysis method (Example: Analyze voice data with an emotion engine, then analyze on a server based on frailty criteria)

[0834] Support services provided (e.g., health behavior reminders)

[0835] Specific example (Example: Flow of entering and saving initial user information)

[0836] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[0837] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0838] Processing flow and specific actions

[0839] Initial setup and user registration

[0840] Step 1:

[0841] Input: User's basic information (name, age, gender, lifestyle, etc.)

[0842] Function: The device provides an interface for registering initial information. Using a touch panel or voice input form, it prompts the user to enter basic information such as "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day."

[0843] Output: Input basic information

[0844] Step 2:

[0845] Input: Basic information entered in Step 1

[0846] Operation: The terminal sends the entered basic information to the server. The data is encrypted and transmitted over the network.

[0847] Output: Basic information sent to the server

[0848] Step 3:

[0849] Input: Basic information sent to the server in Step 2

[0850] Operation: The server receives the entered basic information and stores it in the database. The stored information is used as foundational data to monitor the health status of individual users.

[0851] Output: Basic information stored in the database

[0852] Everyday communication and emotion recognition

[0853] Step 4:

[0854] Input: Set a fixed time (e.g., every morning at 8:00).

[0855] Operation: The device will send a voice message to the user at a set time, asking about their health and mood. For example, it might send a message like, "Good morning, how are you feeling today?"

[0856] Output: Sending a voice message to the user

[0857] Step 5:

[0858] Input: User response to the voice message sent in Step 4

[0859] Operation: The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today." The device records this voice response.

[0860] Output: Recorded user voice data

[0861] Step 6:

[0862] Input: User voice data recorded in Step 5

[0863] Operation: The device sends recorded audio data to the emotion engine. The emotion engine analyzes this audio data and recognizes the user's emotional state. For example, it can detect "fatigue" and "anxiety."

[0864] Output: Analyzed sentiment data

[0865] Step 7:

[0866] Input: Emotional data and voice data analyzed in Step 6

[0867] Operation: The server converts emotional and audio data into text data and analyzes it based on frailty criteria. The results are recorded in a database.

[0868] Output: Frailty analysis results stored in the database

[0869] Data accumulation and analysis

[0870] Step 8:

[0871] Input: Frailty analysis results saved in the database in Step 7

[0872] Operation: The server aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, it might detect a trend from the past month's data showing that users are experiencing fatigue more than twice a week, while simultaneously experiencing increased anxiety.

[0873] Output: Evaluation results of frailty progress

[0874] Step 9:

[0875] Input: Evaluation results of the frailty progress obtained in Step 8

[0876] Operation: The server calculates an assessment score based on frailty criteria. For example, it might assess the user as having a "frailty risk score of 70 / 100". This assessment result is then communicated to the user's family.

[0877] Output: Evaluation results and notification message

[0878] Support and reminders

[0879] Step 10:

[0880] Input: Evaluation results and perceived emotional states obtained in Step 9

[0881] Operation: The device provides the user with health behavior reminders based on the evaluation results. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[0882] Output: Health behavior reminders

[0883] Step 11:

[0884] Input: User response to the reminder issued in Step 10

[0885] Operation: The user acts according to a reminder and reports the result to the device. For example, they might provide feedback such as, "Yes, I went for a walk." The device records this feedback.

[0886] Output: Recorded feedback data

[0887] Step 12:

[0888] Input: Feedback data recorded in step 11

[0889] Operation: The device sends user feedback data to the server.

[0890] Output: Feedback data sent to the server

[0891] Step 13:

[0892] Input: Feedback data sent to the server in step 12

[0893] Operation: The server updates the database with the feedback data it receives. Based on the updated data, it reassesss the risk of frailty and creates the next action plan. For example, it might suggest, "Increase the frequency of walks to three times a week."

[0894] Output: Updated data and next action plan

[0895] Through the specific processing steps described above, this system can accurately monitor the health status of elderly individuals and promote appropriate health behaviors.

[0896] (Application Example 2)

[0897] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0898] Monitoring the progression of frailty in older adults and suggesting appropriate health behaviors is crucial, but simply analyzing health status and monitoring emotional state is insufficient. In particular, nutritional status significantly impacts health in older adults, requiring comprehensive health support that includes nutritional assistance. Furthermore, not only are health behavior reminders necessary, but also means to implement specific dietary suggestions.

[0899] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for communicating with the elderly on a daily basis using an interactive speaker equipped with generating artificial intelligence; means for converting voice data acquired by the interactive speaker into text data and analyzing it based on frailty criteria; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family; means for providing the elderly with health behavior reminders based on the evaluation results; means for receiving feedback from the elderly and updating the feedback data in the database; means for generating meal suggestions that support the nutritional status of the elderly based on the analysis results and emotional state; and means for providing the meal suggestions to the elderly as specific menus in cooperation with a food delivery system. This enables comprehensive support for frailty prevention and nutrition.

[0900] "Generative artificial intelligence" is a type of artificial intelligence that learns from large amounts of data and has the ability to engage in natural conversations and make decisions like a human.

[0901] An "interactive speaker" is a speaker device that has the function of receiving voice input from the user and responding with voice output.

[0902] "Frailty criteria" are standards used to assess the progression of frailty in older adults based on their physical strength and health status.

[0903] "Analysis results" refer to conclusions and evaluation data obtained based on the analysis of voice data and emotional states.

[0904] A "database" is an information storage system for systematically accumulating and managing large amounts of data.

[0905] A "health behavior reminder" is a notification or alert designed to encourage users to engage in healthy behaviors.

[0906] "Feedback from the elderly" refers to information and opinions provided by elderly individuals regarding their behavior and physical condition.

[0907] "Meal suggestions to support nutritional status" refer to meal menus recommended based on the health status and nutritional needs of elderly individuals.

[0908] A "food delivery system" is a service system that delivers meals ordered by users to a specified location.

[0909] This invention is a system that uses an interactive speaker equipped with artificial intelligence to monitor the progress of frailty in elderly individuals and provides comprehensive health support, including nutritional support. Specific embodiments are described below.

[0910] Hardware and software to be used

[0911] Hardware: Smartphones, smart speakers

[0912] Software: Emotion engine (e.g., Microsoft Azure Emotion API), Natural Language Processing engine (e.g., OpenAI's GPT-3), Food Delivery API

[0913] System Configuration

[0914] 1. User Registration

[0915] The terminal provides an interface that allows the user to input basic information such as name, age, gender, and lifestyle. The information entered by the user is sent to the server and stored in a database.

[0916] 2. Daily communication and monitoring of health status

[0917] At specific times, the device sends a voice message to the user asking about their health and mood. For example, every morning at 8:00 AM, it might ask, "Good morning. How are you feeling today?" The user's voice response is recorded by an interactive speaker and sent to a server. The recorded voice data is converted to text and analyzed using an emotion engine.

[0918] 3. Accumulation and evaluation of analysis results

[0919] The converted text and sentiment data are stored on the server, and the progress of frailty is evaluated based on past data. The evaluation results are stored in a database and shared with the user's family.

[0920] 4. Generating and providing meal suggestions

[0921] Based on the analysis results and emotional state, meal suggestions are generated to support the user's nutritional status. These meal suggestions are integrated with a food delivery system and provided as specific menus. For example, if fatigue or stress levels are high, a balanced and nutritious menu will be suggested.

[0922] example

[0923] For example, if a 65-year-old male user answers "I'm a little tired today" during his daily health check, the voice data is analyzed through an emotion engine. As a result, fatigue and mild anxiety are detected. The server analyzes the data from the past month and determines that the user's fatigue is on an increasing trend. As a result, a nutritionally balanced "chicken and vegetable plate" is suggested for lunch and made available for order through a food delivery system.

[0924] Example of a prompt

[0925] Check the user's daily physical and emotional state and analyze the data weekly. Then, use an emotion engine to assess increases and decreases in fatigue and anxiety, suggest a balanced meal, and display that menu in a form that can be ordered via food delivery.

[0926] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0927] Step 1:

[0928] Input: The user enters basic information such as name, age, gender, and lifestyle into the terminal.

[0929] Operation: The terminal collects the entered information and sends it to the server.

[0930] Output: The server saves the received user information to the database.

[0931] Step 2:

[0932] Input: The device will send an audio message at a specific time asking about the elderly person's physical condition and mood.

[0933] Operation: The user responds by voice about their physical condition and mood. For example, they might say, "I'm a little tired today."

[0934] Output: The terminal records the user's voice response and sends the audio data to the server.

[0935] Step 3:

[0936] Input: The server sends the received audio data to the emotion engine.

[0937] Operation: The emotion engine analyzes audio data and identifies emotional states.

[0938] Output: The server receives the analysis results from the emotion engine and saves them to the database as text data and emotion data.

[0939] Step 4:

[0940] Input: The server aggregates and analyzes daily voice data and emotion data stored in the database.

[0941] Function: By analyzing data, the system evaluates the user's progress in frailty.

[0942] Output: The server calculates the evaluation result as a frailty score and records it in the database.

[0943] Step 5:

[0944] Input: The server analyzes the user's health trends based on frailty score and emotional state data.

[0945] Operation: Uses a natural language processing engine to generate meal suggestions to support the user's nutritional status.

[0946] Output: The server uses a generation AI model to generate individually customized meal suggestions.

[0947] Step 6:

[0948] Input: The server integrates with the food delivery system to provide the generated meal suggestions to the user.

[0949] Operation: The device notifies the user of specific menu options via voice or text.

[0950] Output: The user reviews the suggested menu and places an order through the food delivery service.

[0951] Step 7:

[0952] Input: Users input feedback into the terminal regarding the results of their meals and any changes in their physical condition.

[0953] Operation: Feedback data is sent from the terminal to the server.

[0954] Output: The server updates the database with feedback data and incorporates it into the next analysis.

[0955] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0956] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0957] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0958] [Third Embodiment]

[0959] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0960] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0961] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0962] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0963] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0965] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0966] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0967] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0968] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0969] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0970] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0971] This invention relates to a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. This system operates primarily with a server, terminals, and users.

[0972] 1. Initial setup and user registration

[0973] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, daily habits, etc.

[0974] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter "Taro Yamada, 65 years old, male, takes a walk every day."

[0975] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[0976] 2. Everyday communication

[0977] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[0978] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[0979] Terminal: Records the user's voice response, converts it to text data, and sends it to the server. Using speech recognition technology, it generates text data from the voice statement, "I'm a little tired today."

[0980] Server: Analyzes text data and assesses the user's health status based on frailty criteria. The analysis results are recorded in a database.

[0981] 3. Data accumulation and analysis

[0982] Server: Aggregates data accumulated daily to comprehensively evaluate the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week."

[0983] Server: Calculates an assessment score based on frailty criteria. This score is shared with the user's family. For example, an assessment result such as "Frailty Risk Score: 70 / 100, Caution" is sent to the user's family via email.

[0984] 4. Support and Reminders

[0985] Device: Based on evaluation results, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today."

[0986] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I went for a walk."

[0987] Terminal: Receives user feedback and sends it to the server. The feedback data is updated in the database.

[0988] Server: Reassess the frailty risk based on the updated data and create the next action plan as needed.

[0989] As described above, the system of the present invention supports the maintenance of the health of the elderly by having the server, terminal, and user work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice.

[0990] The following describes the processing flow.

[0991] Step 1:

[0992] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[0993] Specific action: The screen displays the message "Please enter your name" and a text box.

[0994] Step 2:

[0995] User: Enter your information in the terminal's input field and press the submit button.

[0996] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[0997] Step 3:

[0998] Terminal: Sends the entered information to the server.

[0999] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[1000] Step 4:

[1001] Server: Stores received information in the database.

[1002] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[1003] Step 5:

[1004] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[1005] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[1006] Step 6:

[1007] User: Responds to the device's questions using voice.

[1008] Specific action: For example, the user responds, "I'm a little tired today."

[1009] Step 7:

[1010] Terminal: Records the user's voice response, converts it to text data, and sends it to the server.

[1011] Specific operation: Using speech recognition technology, the phrase "I'm a little tired today" is converted to text and sent to the server.

[1012] Step 8:

[1013] Server: Analyzes text data and evaluates the user's health status based on frailty criteria.

[1014] Specific actions: Analyze text data, evaluate it as "mild fatigue," and record it in the database.

[1015] Step 9:

[1016] Server: Collects data accumulated daily and evaluates the user's progress in frailty.

[1017] Specific actions: For example, aggregating data from the past month and detecting trends such as "feeling tired more than twice a week."

[1018] Step 10:

[1019] Server: Calculates an assessment score based on frailty criteria and notifies the user's family.

[1020] Specific actions: For example, notify family members via email of the result, "Frailty risk score: 70 / 100, caution advised."

[1021] Step 11:

[1022] Device: Provides users with health behavior reminders based on evaluation results.

[1023] Specific actions: For example, a voice reminder saying, "It would be good to take a 10-minute walk today."

[1024] Step 12:

[1025] User: Follow the reminder and report to the device whether or not the action was performed.

[1026] Specific actions: For example, answering, "Yes, I went for a walk."

[1027] Step 13:

[1028] Terminal: Receives user feedback and sends it to the server.

[1029] Specific operation: Feedback data is sent from the terminal to the server.

[1030] Step 14:

[1031] Server: Saves updated feedback data to the database and recalculates the frailty risk assessment.

[1032] Specific actions: Update the database and calculate the new frailty risk score.

[1033] The above describes the specific processing flow in the system of the present invention. This process makes it possible to continuously monitor the progress of frailty in elderly individuals and provide appropriate support.

[1034] (Example 1)

[1035] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1036] In modern society, understanding the health status of the elderly and providing appropriate care is a crucial challenge. In particular, there is a need to detect and prevent the progression of frailty in the elderly at an early stage. However, current systems lack the means to monitor the progression of frailty in the elderly through daily communication and to provide appropriate alerts and reminders. As a result, the progression of frailty may be overlooked, posing a risk of a decline in the quality of life for the elderly.

[1037] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1038] In this invention, the server includes means for communicating with elderly people on a daily basis using a generative AI model; means for converting voice data acquired by an interactive voice output device into text data and analyzing it; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing it with family members; means for providing reminders for healthy behaviors; means for receiving and updating feedback; means for recording and analyzing voice responses; and means for aggregating data and calculating a score. This makes it possible to continuously monitor the progress of frailty in elderly people and provide appropriate care.

[1039] "Generative artificial intelligence" is a type of artificial intelligence that uses natural language processing and machine learning models to interact with users and perform data analysis.

[1040] An "interactive voice output device" is a device that communicates with the user through voice, and smart speakers are an example of such devices.

[1041] The term "elderly person" generally refers to a person aged 65 or older.

[1042] "Frailty criteria" are standards used to assess the health status and physical decline of older adults, and are determined based on medical evaluation indicators.

[1043] "Text data" refers to data such as audio and images that are represented as a string of characters.

[1044] A "database" is a system for efficiently storing, managing, and retrieving data.

[1045] "Healthy behaviors" refer to actions that older adults should take to prevent disease and maintain their health.

[1046] A "reminder" is a notification or message sent to a user to prompt them to take a specific action.

[1047] "Feedback" refers to the information that users provide to a system regarding their actions and status.

[1048] "Speech recognition technology" is a technology that analyzes speech data and converts it into text data.

[1049] An "evaluation score" is a numerical representation of the analysis results, used to quantitatively assess the user's condition.

[1050] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze data and is trained to perform a specific task.

[1051] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on the tasks or analyses it should perform.

[1052] This invention is a system for monitoring and preventing frailty in the elderly using an interactive voice output device equipped with a generative AI model. This system operates primarily with a server, terminals, and users.

[1053] 1. Initial setup and user registration

[1054] Terminal: Provides an interface for registering the user's initial information. For example, an application installed on a tablet or smartphone is used. The application is developed in Swift for iOS devices and in Kotlin for Android devices, for example. The user enters basic information such as name, age, gender, and daily habits. This data is sent to the server in JSON format via the HTTPS protocol.

[1055] Server: Receives user information sent from terminals, analyzes it, and stores it in a database. Cloud platforms such as Amazon RDS and Google Cloud Firestore are used as the database. The data collected here will later serve as foundational data for evaluating the health status of elderly individuals.

[1056] 2. Everyday communication

[1057] Device: A scheduled task is triggered at a specific time, such as 8 AM every morning, to broadcast a voice message saying, "Good morning, how are you feeling today?" This voice message is generated using the Google Cloud Text-to-Speech API.

[1058] User: Responds to the device's questions using voice. For example, responds, "I'm a little tired today."

[1059] Terminal: Records the user's voice response and converts the audio data into text data using the Google Cloud Speech-to-Text API. This response, for example, "I'm a little tired today," is then sent to the server.

[1060] Server: Analyzes received text data and evaluates the user's health status based on frailty criteria. Natural language processing models such as GPT-4 and BERT are used as generative AI models for the analysis. The analysis results are recorded in a database.

[1061] 3. Data accumulation and analysis

[1062] Server: Collects daily data and comprehensively evaluates the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week." Data collection and analysis are performed using Python's Pandas and NumPy.

[1063] Server: Calculates an assessment score based on frailty criteria. For example, an assessment result such as "Frailty Risk Score: 70 / 100" is shared with the user's family using an email sending service such as SendGrid or AWS SES.

[1064] 4. Support and Reminders

[1065] Device: Based on evaluation results, it provides elderly individuals with reminders for appropriate health behaviors. For example, an interactive voice output device might send a voice message saying, "It would be good to take a 10-minute walk today." This voice message can also be generated using the Google Cloud Text-to-Speech API.

[1066] User: Performs an action based on the reminder and reports to the device, "Yes, I went for a walk." This confirms the effectiveness of the reminder.

[1067] Terminal: Receives user feedback and sends it to the server. Feedback data is sent in JSON format and updated in the database.

[1068] Server: Based on the received feedback data, reassess the frailty risk and create the next action plan as needed.

[1069] Specific example

[1070] Here are some examples of specific prompt messages:

[1071] "Analyze the reasons why users reported feeling tired yesterday and propose an appropriate action plan."

[1072] "Based on the user's daily communication data from the past week, please calculate their frailty risk score."

[1073] "Assess the user's overall health status and generate reminders for improvement."

[1074] In this way, a system is realized that supports the maintenance of the health of the elderly by having terminals, servers, and users work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice and support.

[1075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1076] Step 1: Start the initial setup

[1077] Terminal: Launches the application and displays the user interface. The user enters basic information such as name, age, gender, and daily habits. The input data is temporarily stored within the terminal as a JSON object (input).

[1078] Server: When the user presses the submit button, the device sends this JSON data to the server using the HTTPS protocol. The server parses the received data and saves it to an Amazon RDS database (output). This registers the user's basic information as initial settings.

[1079] Step 2: Start daily communication

[1080] Terminal: At a set time, for example, every morning at 8:00 AM, a scheduled task is triggered, generating and sending a voice message saying "Good morning, how are you feeling today?" using the Google Cloud Text-to-Speech API (input and output).

[1081] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[1082] Terminal: Records the user's voice response and uses the Google Cloud Speech-to-Text API to convert the voice data into text data (data processing). For example, the text "I'm a little tired today" is generated (output). This text data is sent to the server.

[1083] Step 3: Analyzing text data

[1084] Server: Receives text data sent from the terminal and analyzes the text using a generative AI model (e.g., GPT-4) (input and data processing). For example, it analyzes the content "I'm a little tired" and evaluates the user's health status.

[1085] Server: Based on the analysis results, the server applies frailty criteria and evaluates the user's health status (data calculation). The evaluation result is recorded in the database, for example, in the form of "Frailty Risk: Medium" (output).

[1086] Step 4: Data aggregation and calculation of evaluation scores

[1087] Server: Aggregates daily accumulated evaluation data to comprehensively assess the user's frailty progress. For example, it uses Python's Pandas and NumPy to detect trends such as "feeling tired more than twice a week" from data over the past month (input and data calculation).

[1088] Server: Based on the aggregated data, it calculates an evaluation score according to the prevention criteria. For example, a result such as "Frailty Risk Score: 70 / 100" might be obtained (output). This score is then notified to the user's family via email.

[1089] Step 5: Provide a reminder

[1090] Terminal: Based on evaluation results received from the server, it generates reminders for specific health behaviors and sends them as voice messages. For example, a reminder such as "It would be good to take a 10-minute walk today" is generated using the Google Cloud Text-to-Speech API (input and output).

[1091] User: Perform an action based on a reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[1092] Terminal: Records user feedback, converts it into text data, and sends it to the server (data processing and output).

[1093] Step 6: Analyze feedback and update the database

[1094] Server: Analyzes received feedback data (using a generative AI model) and updates the database (input and data calculation). For example, information that a user has taken a walk is added to the database.

[1095] Server: Reassess frailty risk based on updated data and create the next action plan as needed. This determines the content of the next reminder or notification (output).

[1096] This clearly outlines the overall system's processing flow, enabling the implementation of a program to continuously monitor the progress of frailty in older adults and provide appropriate care.

[1097] (Application Example 1)

[1098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1099] This invention aims to prevent the progression of frailty and health risks, and support individual health maintenance, by monitoring the health status of elderly people and workers in real time and promoting appropriate health behaviors. In particular, it addresses the challenge of efficiently managing the health of elderly workers in work environments such as factories, and providing appropriate breaks and reminders to improve work efficiency and reduce health risks.

[1100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1101] In this invention, the server includes means for communicating with elderly people or workers on a daily basis using an interactive speaker or robot equipped with generative artificial intelligence; means for converting voice data acquired by the interactive speaker or robot into text data and analyzing it based on frailty criteria or health risk criteria; means for evaluating the progress of frailty or health risk based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family or facility manager; means for providing health behavior reminders to elderly people or workers based on the evaluation results; and means for receiving feedback from elderly people or workers and updating the database with feedback data. This enables elderly people and workers to understand their own health status on a daily basis and take appropriate action.

[1102] "Generative artificial intelligence" is artificial intelligence designed to interact with humans using technologies such as natural language processing and machine learning.

[1103] An "interactive speaker" is a speaker device that can communicate with the user using voice.

[1104] A "robot" refers to a machine that operates automatically and can interact with and perform tasks with humans.

[1105] The term "elderly" generally refers to people aged 65 and over, and often includes those living after retirement.

[1106] "Workers" refer to people who are engaged in specific tasks in a specific workplace.

[1107] "Communication" is the process by which humans and machines exchange information and understand each other.

[1108] "Audio data" refers to digital data in which audio has been recorded.

[1109] "Text data" refers to digital data recorded as character information.

[1110] "Frailty criteria" are standards used to assess the health status of elderly people.

[1111] "Health risk criteria" are standards used to assess the health status of workers and the elderly and to determine the associated risks.

[1112] "Analysis" is the process of processing data and extracting useful information.

[1113] "Evaluation" is the process of judging a specific state or situation based on the results of an analysis.

[1114] A "database" is a system for structuring, collecting, and managing data.

[1115] "User's family" refers to anyone belonging to the household of a system user.

[1116] A "facility manager" is someone responsible for the working environment and the operation of the facility.

[1117] A "reminder" is a message that notifies a user about a specific action or event.

[1118] "Feedback" refers to responses and reactions from users, and is information that is used for subsequent processing and evaluation.

[1119] This invention relates to a system that uses an interactive speaker or robot to monitor the health status of the elderly and workers in real time and promote appropriate health behaviors. This system operates primarily with a server, a terminal (interactive speaker or robot), and a user.

[1120] 1. Initial setup and user registration

[1121] Terminal (Robot): The terminal provides an interface for registering the user's initial information. For example, it prompts the user to enter the name, age, gender, role, and daily work duties of a worker in a factory.

[1122] User: Follow the interface presented by the robot and enter your information. For example, register information such as "Ichiro Sato, 70 years old, male, line operator."

[1123] Server: Receives user information sent from the terminal and stores it in the database. This collects basic data for monitoring the user's health status.

[1124] 2. Everyday communication

[1125] Terminal (Robot): At scheduled times, it sends voice messages to workers asking about their physical condition and mood. For example, before starting work, it might ask, "Before we begin work, how are you feeling today?"

[1126] User: Responds to the device's questions using voice. For example, "My shoulders are a little stiff."

[1127] Terminal (robot): Records the user's voice response, converts it to text data using the Google Cloud Speech-to-Text API, and sends it to the server.

[1128] Server: Analyzes text data and assesses the user's health status based on health risk criteria. The analysis results are recorded in a database.

[1129] 3. Data accumulation and analysis

[1130] Server: Collects daily accumulated data and comprehensively evaluates workers' health risks. TensorFlow is used for the analysis system.

[1131] Server: Calculates a health risk score, and this score is shared with the facility administrator. For example, it notifies the administrator via email of an evaluation result such as "Health Risk Score: 60 / 100, Caution."

[1132] 4. Support and Reminders

[1133] Terminal (Robot): Based on evaluation results, it provides workers with reminders for appropriate health behaviors. For example, it might remind them, "It would be good to do 10 minutes of stretching."

[1134] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I did my stretches."

[1135] Terminal (robot): Receives user feedback and sends it to the server. The feedback data is updated in the database.

[1136] Server: Reassess health risks based on updated data and create the next action plan as needed.

[1137] Hardware and software configuration

[1138] Hardware used: A robot equipped with an Intel Core i7 processor, built-in microphone, speaker, and high-resolution camera.

[1139] Software used: Python, TensorFlow, Google Cloud Speech-to-Text API, and the Django framework.

[1140] Specific example

[1141] Examples of prompts for a generative AI model:

[1142] Robot: "Good morning, Mr. Sato. How are you feeling today?"

[1143] User: "My shoulders are a little stiff."

[1144] In response to this prompt, the AI ​​model sends the generated text data to the server and provides post-analysis reminders and health advice.

[1145] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1146] Step 1: The terminal (robot) prompts the user (worker) to enter initial information.

[1147] Input: User's name, age, gender, role, and daily tasks.

[1148] Data processing: Organize the input information into structured data.

[1149] Output: Structured initial information data.

[1150] Specific operation: The device prompts the user via display and voice, "Please enter your name, age, gender, role, and daily tasks," and the user enters the information.

[1151] Step 2: The terminal (robot) sends initial information to the server.

[1152] Input: Structured initial information data.

[1153] Data processing: Performs data transmission processing from the terminal to the server.

[1154] Output: Initial information stored on the server.

[1155] Specific operation: The device sends information obtained from the user to the server via an API and stores it there.

[1156] Step 3: The terminal (robot) sends a voice message at scheduled times asking about the user's physical condition and mood.

[1157] Input: The prompt text for the dialogue.

[1158] Data processing: Natural language generation processing using generative AI models.

[1159] Output: A voice message asking the user a question.

[1160] Specific action: The device will say aloud, "Before we begin, how are you feeling today?"

[1161] Step 4: The user responds to the device's questions using voice.

[1162] Input: User voice response.

[1163] Data processing: Recording and saving audio data.

[1164] Output: Recorded audio data.

[1165] Specific action: The user says, "My shoulders are a little stiff," and the device starts recording the voice.

[1166] Step 5: The terminal (robot) converts the voice data into text data and sends it to the server.

[1167] Input: Recorded audio data.

[1168] Data processing: Speech recognition and text conversion using the Google Cloud Speech-to-Text API.

[1169] Output: Converted text data.

[1170] Specific operation: Use a speech recognition API to convert the spoken phrase "My shoulders are a little stiff" into text and send it to the server.

[1171] Step 6: The server analyzes the text data and assesses the user's health status based on health risk criteria.

[1172] Input: Converted text data.

[1173] Data processing: Text data analysis and health status assessment using TensorFlow.

[1174] Output: Evaluation results.

[1175] Specific operation: The server receives the text "My shoulders are a little stiff" and performs an analysis based on health risk criteria.

[1176] Step 7: The server records the analysis results in the database.

[1177] Input: Analysis results.

[1178] Data processing: Writing to a database.

[1179] Output: Evaluation results recorded in the database.

[1180] Specific action: Save the analyzed health risk score to the database.

[1181] Step 8: The terminal (robot) provides a reminder to the user based on the evaluation results.

[1182] Input: Evaluation result.

[1183] Data processing: Reminder generation, natural language generation.

[1184] Output: Reminder voice message.

[1185] Specific action: The device will remind you, "It would be good to do 10 minutes of stretching."

[1186] Step 9: The user follows the reminder, takes the action, and reports it to the device.

[1187] Input: User voice report.

[1188] Data processing: Recording and saving audio data.

[1189] Output: Recorded feedback audio data.

[1190] Specific action: The user reports "Yes, I've stretched," and the device records this.

[1191] Step 10: The terminal (robot) sends feedback data to the server.

[1192] Input: Recorded feedback audio data.

[1193] Data processing: Text conversion using speech recognition, transmission to the server.

[1194] Output: Updated feedback text data sent to the server.

[1195] Specific operation: Convert the recorded audio data into text and send it to the server.

[1196] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1197] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[1198] 1. Initial setup and user registration

[1199] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, lifestyle, etc.

[1200] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter information such as "Taro Yamada, 65 years old, male, takes a walk every day."

[1201] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[1202] 2. Everyday communication and emotion recognition

[1203] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[1204] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[1205] Device: Records the user's voice response and sends the voice data to the emotion engine to recognize emotions. After voice recognition, the emotion engine analyzes the user's emotions and detects fatigue and slight anxiety from a response such as "I'm a little tired today."

[1206] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in the database.

[1207] 3. Data accumulation and analysis

[1208] Server: It aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, by analyzing data from the past month, it detects a trend where users "feel tired more than twice a week" are experiencing increased "anxiety."

[1209] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data. For example, it might send an email to the family with an assessment result such as, "Frailty risk score: 70 / 100, caution. The user has recently shown a stronger tendency to feel anxious."

[1210] 4. Support and Reminders

[1211] Device: Based on evaluation results and recognized emotional states, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1212] User: Follow the reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[1213] Terminal: Receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[1214] Server: Based on the updated data, reassess the risk of frailty and create the next action plan as needed. For example, it might suggest specific actions such as "increase the frequency of walks to three times a week."

[1215] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[1216] The following describes the processing flow.

[1217] Step 1:

[1218] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[1219] Specific action: The screen displays the message "Please enter your name" and a text box.

[1220] Step 2:

[1221] User: Enter your information in the terminal's input field and press the submit button.

[1222] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[1223] Step 3:

[1224] Terminal: Sends the entered information to the server.

[1225] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[1226] Step 4:

[1227] Server: Stores received information in the database.

[1228] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[1229] Step 5:

[1230] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[1231] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[1232] Step 6:

[1233] User: Responds to the device's questions using voice.

[1234] Specific action: For example, the user responds, "I'm a little tired today."

[1235] Step 7:

[1236] Terminal: Records the user's voice response and sends that voice data to the emotion engine to recognize emotions.

[1237] Specific operation: Voice data is sent to the emotion engine, and the emotion recognition algorithm detects the emotions of "fatigue" and "slight anxiety."

[1238] Step 8:

[1239] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria.

[1240] Specific actions: Analyze emotional data and text data to assess "mild fatigue" and record it in the database.

[1241] Step 9:

[1242] Server: Aggregates the data accumulated daily and comprehensively evaluates the user's frailty progress.

[1243] Specific actions: For example, by analyzing data from the past month, it detects a trend where people "feel tired more than twice a week" and simultaneously experience increased "anxiety."

[1244] Step 10:

[1245] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data.

[1246] Specific actions: For example, an email notification is sent to the family notifying them of the assessment result, such as, "Frailty risk score: 70 / 100, caution advised. The individual has recently shown a stronger tendency to experience anxiety."

[1247] Step 11:

[1248] Device: Provides users with health behavior reminders based on evaluation results and recognized emotional states.

[1249] Specific actions: For example, a voice message could be sent reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1250] Step 12:

[1251] User: Follow the reminder and report the results to the device.

[1252] Specific actions: For example, answering, "Yes, I went for a walk."

[1253] Step 13:

[1254] Terminal: Receives user feedback and sends it to the server.

[1255] Specific operation: Feedback data is sent from the terminal to the server.

[1256] Step 14:

[1257] Server: Saves updated feedback and sentiment data to the database and recalculates the frailty risk assessment.

[1258] Specific actions: Update the database and calculate the new frailty risk score.

[1259] The above describes the specific processing flow in the system of the present invention that incorporates an emotion engine. By recognizing the user's emotions, more advanced and personalized health support becomes possible.

[1260] (Example 2)

[1261] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1262] There is a need for a system that routinely monitors the health status of the elderly and detects and prevents the progression of frailty at an early stage. However, it is difficult for the elderly to accurately understand their own health status and report it to medical institutions and family members at the appropriate time. Furthermore, there are limited means of providing advanced health support that takes emotional states into account. To solve this problem, an easy-to-use interface and accurate data collection and analysis are necessary.

[1263] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for registering the user's initial information, means for receiving basic information entered by the user and storing it in a database, means for sending an audio message to the user at a specific time asking about their physical condition and mood, means for recording the user's voice response and sending it to an emotion engine to recognize emotions, means for converting the emotion data and audio data recognized by the emotion engine into text data and analyzing it based on frailty criteria, means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, means for sharing the recorded data with the user's family, means for providing the user with health behavior reminders based on the evaluation results, and means for receiving user feedback and updating the database with feedback data. This enables accurate monitoring of the health status of the elderly and promotion of preventive behaviors.

[1264] An "interface" is a means by which a user interacts with a device or system and inputs and outputs information.

[1265] "Basic information" refers to data that includes personal attributes such as the user's name, age, gender, and lifestyle.

[1266] A "voice message" is a question or instruction delivered by voice from an electronic device.

[1267] An "emotion engine" is a combination of software or hardware used to recognize and analyze a user's emotional state from voice or text data.

[1268] "Text data" refers to data obtained by converting audio data into written text.

[1269] "Frailty criteria" are standards or indicators used to assess the health status and physical function decline of older adults.

[1270] "Frailty progress" refers to the status indicating the progression of a user's health condition and decline in physical function.

[1271] A "database" is a system or software for efficiently storing, managing, and retrieving collected data.

[1272] "Family" refers to individuals who have a close relationship with the user, and usually means the user's relatives.

[1273] A "reminder" is a notification or reminder that serves to remind a user of a specific action or appointment.

[1274] "Feedback" is data used by users to report the results and their impressions to the system after they have taken some kind of action.

[1275] "Analysis" is the process of thoroughly examining and evaluating collected data to derive useful information.

[1276] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[1277] Initial setup and user registration

[1278] The terminal provides an interface for registering the user's initial information. This interface includes forms accessible via touch panel and voice input. The user enters basic information such as name, age, gender, and lifestyle. For example, they might enter "Taro Yamada, 65 years old, male, takes a walk every day."

[1279] The server receives user information sent from the terminal and stores it in a database. This collects basic data for monitoring the health status of individual users.

[1280] Everyday communication and emotion recognition

[1281] The device will send voice messages to the user at specific times asking about their health and mood. For example, every morning at 8:00 AM, it will send a voice message asking, "Good morning, how are you feeling today?"

[1282] The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today."

[1283] The device records the user's voice responses and sends that voice data to an emotion engine to recognize emotions. The emotion engine analyzes the user's voice and detects, for example, "fatigue" and "anxiety."

[1284] The server converts the emotion data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in a database.

[1285] Data accumulation and analysis

[1286] The server aggregates the data accumulated daily and comprehensively evaluates the user's progress towards frailty. For example, by analyzing data from the past month, it detects a trend where users are experiencing fatigue more than twice a week, while simultaneously showing an increase in anxiety.

[1287] The server calculates an assessment score based on frailty criteria and notifies the user's family of the assessment score and emotional data. For example, it might send an email notification stating, "Frailty risk score: 70 / 100, caution advised. The user has recently shown a stronger tendency to feel anxious."

[1288] Support and reminders

[1289] The device provides users with health behavior reminders based on evaluation results and recognized emotional states. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1290] The user acts according to the reminder and reports the result to the device. For example, they might respond, "Yes, I went for a walk."

[1291] The device receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[1292] The server reassesss the frailty risk based on the updated data and, if necessary, create the next action plan. For example, it might suggest specific actions such as, "Increase the frequency of walks to three times a week."

[1293] Examples and prompts for generative AI models

[1294] As a concrete example, a user enters "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day" into the interface on their device, and the server saves this information to a database. Furthermore, a voice message is sent from the device every morning at 8:00 AM saying, "Good morning, how are you feeling today?" to which the user replies, "I'm a little tired today."

[1295] Example prompts for generative AI models

[1296] Please describe the details of the system that monitors the user's health status. Include the following information:

[1297] Hardware and software to be used (e.g., conversational speaker with generative artificial intelligence, emotion engine)

[1298] Data collection methods (e.g., voice responses from users)

[1299] Analysis method (Example: Analyze voice data with an emotion engine, then analyze on a server based on frailty criteria)

[1300] Support services provided (e.g., health behavior reminders)

[1301] Specific example (Example: Flow of entering and saving initial user information)

[1302] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[1303] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1304] Processing flow and specific actions

[1305] Initial setup and user registration

[1306] Step 1:

[1307] Input: User's basic information (name, age, gender, lifestyle, etc.)

[1308] Function: The device provides an interface for registering initial information. Using a touch panel or voice input form, it prompts the user to enter basic information such as "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day."

[1309] Output: Input basic information

[1310] Step 2:

[1311] Input: Basic information entered in Step 1

[1312] Operation: The terminal sends the entered basic information to the server. The data is encrypted and transmitted over the network.

[1313] Output: Basic information sent to the server

[1314] Step 3:

[1315] Input: Basic information sent to the server in Step 2

[1316] Operation: The server receives the entered basic information and stores it in the database. The stored information is used as foundational data to monitor the health status of individual users.

[1317] Output: Basic information stored in the database

[1318] Everyday communication and emotion recognition

[1319] Step 4:

[1320] Input: Set a fixed time (e.g., every morning at 8:00).

[1321] Operation: The device will send a voice message to the user at a set time, asking about their health and mood. For example, it might send a message like, "Good morning, how are you feeling today?"

[1322] Output: Sending a voice message to the user

[1323] Step 5:

[1324] Input: User response to the voice message sent in Step 4

[1325] Operation: The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today." The device records this voice response.

[1326] Output: Recorded user voice data

[1327] Step 6:

[1328] Input: User voice data recorded in Step 5

[1329] Operation: The device sends recorded audio data to the emotion engine. The emotion engine analyzes this audio data and recognizes the user's emotional state. For example, it can detect "fatigue" and "anxiety."

[1330] Output: Analyzed sentiment data

[1331] Step 7:

[1332] Input: Emotional data and voice data analyzed in Step 6

[1333] Operation: The server converts emotional and audio data into text data and analyzes it based on frailty criteria. The results are recorded in a database.

[1334] Output: Frailty analysis results stored in the database

[1335] Data accumulation and analysis

[1336] Step 8:

[1337] Input: Frailty analysis results saved in the database in Step 7

[1338] Operation: The server aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, it might detect a trend from the past month's data showing that users are experiencing fatigue more than twice a week, while simultaneously experiencing increased anxiety.

[1339] Output: Evaluation results of frailty progress

[1340] Step 9:

[1341] Input: Evaluation results of the frailty progress obtained in Step 8

[1342] Operation: The server calculates an assessment score based on frailty criteria. For example, it might assess the user as having a "frailty risk score of 70 / 100". This assessment result is then communicated to the user's family.

[1343] Output: Evaluation results and notification message

[1344] Support and reminders

[1345] Step 10:

[1346] Input: Evaluation results and perceived emotional states obtained in Step 9

[1347] Operation: The device provides the user with health behavior reminders based on the evaluation results. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1348] Output: Health behavior reminders

[1349] Step 11:

[1350] Input: User response to the reminder issued in Step 10

[1351] Operation: The user acts according to a reminder and reports the result to the device. For example, they might provide feedback such as, "Yes, I went for a walk." The device records this feedback.

[1352] Output: Recorded feedback data

[1353] Step 12:

[1354] Input: Feedback data recorded in step 11

[1355] Operation: The device sends user feedback data to the server.

[1356] Output: Feedback data sent to the server

[1357] Step 13:

[1358] Input: Feedback data sent to the server in step 12

[1359] Operation: The server updates the database with the feedback data it receives. Based on the updated data, it reassesss the risk of frailty and creates the next action plan. For example, it might suggest, "Increase the frequency of walks to three times a week."

[1360] Output: Updated data and next action plan

[1361] Through the specific processing steps described above, this system can accurately monitor the health status of elderly individuals and promote appropriate health behaviors.

[1362] (Application Example 2)

[1363] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1364] Monitoring the progression of frailty in older adults and suggesting appropriate health behaviors is crucial, but simply analyzing health status and monitoring emotional state is insufficient. In particular, nutritional status significantly impacts health in older adults, requiring comprehensive health support that includes nutritional assistance. Furthermore, not only are health behavior reminders necessary, but also means to implement specific dietary suggestions.

[1365] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for communicating with the elderly on a daily basis using an interactive speaker equipped with generating artificial intelligence; means for converting voice data acquired by the interactive speaker into text data and analyzing it based on frailty criteria; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family; means for providing the elderly with health behavior reminders based on the evaluation results; means for receiving feedback from the elderly and updating the feedback data in the database; means for generating meal suggestions that support the nutritional status of the elderly based on the analysis results and emotional state; and means for providing the meal suggestions to the elderly as specific menus in cooperation with a food delivery system. This enables comprehensive support for frailty prevention and nutrition.

[1366] "Generative artificial intelligence" is a type of artificial intelligence that learns from large amounts of data and has the ability to engage in natural conversations and make decisions like a human.

[1367] An "interactive speaker" is a speaker device that has the function of receiving voice input from the user and responding with voice output.

[1368] "Frailty criteria" are standards used to assess the progression of frailty in older adults based on their physical strength and health status.

[1369] "Analysis results" refer to conclusions and evaluation data obtained based on the analysis of voice data and emotional states.

[1370] A "database" is an information storage system for systematically accumulating and managing large amounts of data.

[1371] A "health behavior reminder" is a notification or alert designed to encourage users to engage in healthy behaviors.

[1372] "Feedback from the elderly" refers to information and opinions provided by elderly individuals regarding their behavior and physical condition.

[1373] "Meal suggestions to support nutritional status" refer to meal menus recommended based on the health status and nutritional needs of elderly individuals.

[1374] A "food delivery system" is a service system that delivers meals ordered by users to a specified location.

[1375] This invention is a system that uses an interactive speaker equipped with artificial intelligence to monitor the progress of frailty in elderly individuals and provides comprehensive health support, including nutritional support. Specific embodiments are described below.

[1376] Hardware and software to be used

[1377] Hardware: Smartphones, smart speakers

[1378] Software: Emotion engine (e.g., Microsoft Azure Emotion API), Natural Language Processing engine (e.g., OpenAI's GPT-3), Food Delivery API

[1379] System Configuration

[1380] 1. User Registration

[1381] The terminal provides an interface that allows the user to input basic information such as name, age, gender, and lifestyle. The information entered by the user is sent to the server and stored in a database.

[1382] 2. Daily communication and monitoring of health status

[1383] At specific times, the device sends a voice message to the user asking about their health and mood. For example, every morning at 8:00 AM, it might ask, "Good morning. How are you feeling today?" The user's voice response is recorded by an interactive speaker and sent to a server. The recorded voice data is converted to text and analyzed using an emotion engine.

[1384] 3. Accumulation and evaluation of analysis results

[1385] The converted text and sentiment data are stored on the server, and the progress of frailty is evaluated based on past data. The evaluation results are stored in a database and shared with the user's family.

[1386] 4. Generating and providing meal suggestions

[1387] Based on the analysis results and emotional state, meal suggestions are generated to support the user's nutritional status. These meal suggestions are integrated with a food delivery system and provided as specific menus. For example, if fatigue or stress levels are high, a balanced and nutritious menu will be suggested.

[1388] example

[1389] For example, if a 65-year-old male user answers "I'm a little tired today" during his daily health check, the voice data is analyzed through an emotion engine. As a result, fatigue and mild anxiety are detected. The server analyzes the data from the past month and determines that the user's fatigue is on an increasing trend. As a result, a nutritionally balanced "chicken and vegetable plate" is suggested for lunch and made available for order through a food delivery system.

[1390] Example of a prompt

[1391] Check the user's daily physical and emotional state and analyze the data weekly. Then, use an emotion engine to assess increases and decreases in fatigue and anxiety, suggest a balanced meal, and display that menu in a form that can be ordered via food delivery.

[1392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1393] Step 1:

[1394] Input: The user enters basic information such as name, age, gender, and lifestyle into the terminal.

[1395] Operation: The terminal collects the entered information and sends it to the server.

[1396] Output: The server saves the received user information to the database.

[1397] Step 2:

[1398] Input: The device will send an audio message at a specific time asking about the elderly person's physical condition and mood.

[1399] Operation: The user responds by voice about their physical condition and mood. For example, they might say, "I'm a little tired today."

[1400] Output: The terminal records the user's voice response and sends the audio data to the server.

[1401] Step 3:

[1402] Input: The server sends the received audio data to the emotion engine.

[1403] Operation: The emotion engine analyzes audio data and identifies emotional states.

[1404] Output: The server receives the analysis results from the emotion engine and saves them to the database as text data and emotion data.

[1405] Step 4:

[1406] Input: The server aggregates and analyzes daily voice data and emotion data stored in the database.

[1407] Function: By analyzing data, the system evaluates the user's progress in frailty.

[1408] Output: The server calculates the evaluation result as a frailty score and records it in the database.

[1409] Step 5:

[1410] Input: The server analyzes the user's health trends based on frailty score and emotional state data.

[1411] Operation: Uses a natural language processing engine to generate meal suggestions to support the user's nutritional status.

[1412] Output: The server uses a generation AI model to generate individually customized meal suggestions.

[1413] Step 6:

[1414] Input: The server integrates with the food delivery system to provide the generated meal suggestions to the user.

[1415] Operation: The device notifies the user of specific menu options via voice or text.

[1416] Output: The user reviews the suggested menu and places an order through the food delivery service.

[1417] Step 7:

[1418] Input: Users input feedback into the terminal regarding the results of their meals and any changes in their physical condition.

[1419] Operation: Feedback data is sent from the terminal to the server.

[1420] Output: The server updates the database with feedback data and incorporates it into the next analysis.

[1421] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1422] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1423] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1424] [Fourth Embodiment]

[1425] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1426] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1427] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1428] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1429] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1430] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1431] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1432] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1433] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1434] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1435] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1436] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1437] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1438] This invention relates to a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. This system operates primarily with a server, terminals, and users.

[1439] 1. Initial setup and user registration

[1440] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, daily habits, etc.

[1441] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter "Taro Yamada, 65 years old, male, takes a walk every day."

[1442] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[1443] 2. Everyday communication

[1444] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[1445] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[1446] Terminal: Records the user's voice response, converts it to text data, and sends it to the server. Using speech recognition technology, it generates text data from the voice statement, "I'm a little tired today."

[1447] Server: Analyzes text data and assesses the user's health status based on frailty criteria. The analysis results are recorded in a database.

[1448] 3. Data accumulation and analysis

[1449] Server: Aggregates data accumulated daily to comprehensively evaluate the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week."

[1450] Server: Calculates an assessment score based on frailty criteria. This score is shared with the user's family. For example, an assessment result such as "Frailty Risk Score: 70 / 100, Caution" is sent to the user's family via email.

[1451] 4. Support and Reminders

[1452] Device: Based on evaluation results, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today."

[1453] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I went for a walk."

[1454] Terminal: Receives user feedback and sends it to the server. The feedback data is updated in the database.

[1455] Server: Reassess the frailty risk based on the updated data and create the next action plan as needed.

[1456] As described above, the system of the present invention supports the maintenance of the health of the elderly by having the server, terminal, and user work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice.

[1457] The following describes the processing flow.

[1458] Step 1:

[1459] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[1460] Specific action: The screen displays the message "Please enter your name" and a text box.

[1461] Step 2:

[1462] User: Enter your information in the terminal's input field and press the submit button.

[1463] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[1464] Step 3:

[1465] Terminal: Sends the entered information to the server.

[1466] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[1467] Step 4:

[1468] Server: Stores received information in the database.

[1469] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[1470] Step 5:

[1471] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[1472] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[1473] Step 6:

[1474] User: Responds to the device's questions using voice.

[1475] Specific action: For example, the user responds, "I'm a little tired today."

[1476] Step 7:

[1477] Terminal: Records the user's voice response, converts it to text data, and sends it to the server.

[1478] Specific operation: Using speech recognition technology, the phrase "I'm a little tired today" is converted to text and sent to the server.

[1479] Step 8:

[1480] Server: Analyzes text data and evaluates the user's health status based on frailty criteria.

[1481] Specific actions: Analyze text data, evaluate it as "mild fatigue," and record it in the database.

[1482] Step 9:

[1483] Server: Collects data accumulated daily and evaluates the user's progress in frailty.

[1484] Specific actions: For example, aggregating data from the past month and detecting trends such as "feeling tired more than twice a week."

[1485] Step 10:

[1486] Server: Calculates an assessment score based on frailty criteria and notifies the user's family.

[1487] Specific actions: For example, notify family members via email of the result, "Frailty risk score: 70 / 100, caution advised."

[1488] Step 11:

[1489] Device: Provides users with health behavior reminders based on evaluation results.

[1490] Specific actions: For example, a voice reminder saying, "It would be good to take a 10-minute walk today."

[1491] Step 12:

[1492] User: Follow the reminder and report to the device whether or not the action was performed.

[1493] Specific actions: For example, answering, "Yes, I went for a walk."

[1494] Step 13:

[1495] Terminal: Receives user feedback and sends it to the server.

[1496] Specific operation: Feedback data is sent from the terminal to the server.

[1497] Step 14:

[1498] Server: Saves updated feedback data to the database and recalculates the frailty risk assessment.

[1499] Specific actions: Update the database and calculate the new frailty risk score.

[1500] The above describes the specific processing flow in the system of the present invention. This process makes it possible to continuously monitor the progress of frailty in elderly individuals and provide appropriate support.

[1501] (Example 1)

[1502] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1503] In modern society, understanding the health status of the elderly and providing appropriate care is a crucial challenge. In particular, there is a need to detect and prevent the progression of frailty in the elderly at an early stage. However, current systems lack the means to monitor the progression of frailty in the elderly through daily communication and to provide appropriate alerts and reminders. As a result, the progression of frailty may be overlooked, posing a risk of a decline in the quality of life for the elderly.

[1504] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1505] In this invention, the server includes means for communicating with elderly people on a daily basis using a generative AI model; means for converting voice data acquired by an interactive voice output device into text data and analyzing it; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing it with family members; means for providing reminders for healthy behaviors; means for receiving and updating feedback; means for recording and analyzing voice responses; and means for aggregating data and calculating a score. This makes it possible to continuously monitor the progress of frailty in elderly people and provide appropriate care.

[1506] "Generative artificial intelligence" is a type of artificial intelligence that uses natural language processing and machine learning models to interact with users and perform data analysis.

[1507] An "interactive voice output device" is a device that communicates with the user through voice, and smart speakers are an example of such devices.

[1508] The term "elderly person" generally refers to a person aged 65 or older.

[1509] "Frailty criteria" are standards used to assess the health status and physical decline of older adults, and are determined based on medical evaluation indicators.

[1510] "Text data" refers to data such as audio and images that are represented as a string of characters.

[1511] A "database" is a system for efficiently storing, managing, and retrieving data.

[1512] "Healthy behaviors" refer to actions that older adults should take to prevent disease and maintain their health.

[1513] A "reminder" is a notification or message sent to a user to prompt them to take a specific action.

[1514] "Feedback" refers to the information that users provide to a system regarding their actions and status.

[1515] "Speech recognition technology" is a technology that analyzes speech data and converts it into text data.

[1516] An "evaluation score" is a numerical representation of the analysis results, used to quantitatively assess the user's condition.

[1517] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to analyze data and is trained to perform a specific task.

[1518] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on the tasks or analyses it should perform.

[1519] This invention is a system for monitoring and preventing frailty in the elderly using an interactive voice output device equipped with a generative AI model. This system operates primarily with a server, terminals, and users.

[1520] 1. Initial setup and user registration

[1521] Terminal: Provides an interface for registering the user's initial information. For example, an application installed on a tablet or smartphone is used. The application is developed in Swift for iOS devices and in Kotlin for Android devices, for example. The user enters basic information such as name, age, gender, and daily habits. This data is sent to the server in JSON format via the HTTPS protocol.

[1522] Server: Receives user information sent from terminals, analyzes it, and stores it in a database. Cloud platforms such as Amazon RDS and Google Cloud Firestore are used as the database. The data collected here will later serve as foundational data for evaluating the health status of elderly individuals.

[1523] 2. Everyday communication

[1524] Device: A scheduled task is triggered at a specific time, such as 8 AM every morning, to broadcast a voice message saying, "Good morning, how are you feeling today?" This voice message is generated using the Google Cloud Text-to-Speech API.

[1525] User: Responds to the device's questions using voice. For example, responds, "I'm a little tired today."

[1526] Terminal: Records the user's voice response and converts the audio data into text data using the Google Cloud Speech-to-Text API. This response, for example, "I'm a little tired today," is then sent to the server.

[1527] Server: Analyzes received text data and evaluates the user's health status based on frailty criteria. Natural language processing models such as GPT-4 and BERT are used as generative AI models for the analysis. The analysis results are recorded in a database.

[1528] 3. Data accumulation and analysis

[1529] Server: Collects daily data and comprehensively evaluates the user's frailty progress. For example, it analyzes data from the past month to detect trends such as "feeling tired more than twice a week." Data collection and analysis are performed using Python's Pandas and NumPy.

[1530] Server: Calculates an assessment score based on frailty criteria. For example, an assessment result such as "Frailty Risk Score: 70 / 100" is shared with the user's family using an email sending service such as SendGrid or AWS SES.

[1531] 4. Support and Reminders

[1532] Device: Based on evaluation results, it provides elderly individuals with reminders for appropriate health behaviors. For example, an interactive voice output device might send a voice message saying, "It would be good to take a 10-minute walk today." This voice message can also be generated using the Google Cloud Text-to-Speech API.

[1533] User: Performs an action based on the reminder and reports to the device, "Yes, I went for a walk." This confirms the effectiveness of the reminder.

[1534] Terminal: Receives user feedback and sends it to the server. Feedback data is sent in JSON format and updated in the database.

[1535] Server: Based on the received feedback data, reassess the frailty risk and create the next action plan as needed.

[1536] Specific example

[1537] Here are some examples of specific prompt messages:

[1538] "Analyze the reasons why users reported feeling tired yesterday and propose an appropriate action plan."

[1539] "Based on the user's daily communication data from the past week, please calculate their frailty risk score."

[1540] "Assess the user's overall health status and generate reminders for improvement."

[1541] In this way, a system is realized that supports the maintenance of the health of the elderly by having terminals, servers, and users work together to continuously monitor the progress of frailty in the elderly and provide appropriate advice and support.

[1542] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1543] Step 1: Start the initial setup

[1544] Terminal: Launches the application and displays the user interface. The user enters basic information such as name, age, gender, and daily habits. The input data is temporarily stored within the terminal as a JSON object (input).

[1545] Server: When the user presses the submit button, the device sends this JSON data to the server using the HTTPS protocol. The server parses the received data and saves it to an Amazon RDS database (output). This registers the user's basic information as initial settings.

[1546] Step 2: Start daily communication

[1547] Terminal: At a set time, for example, every morning at 8:00 AM, a scheduled task is triggered, generating and sending a voice message saying "Good morning, how are you feeling today?" using the Google Cloud Text-to-Speech API (input and output).

[1548] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[1549] Terminal: Records the user's voice response and uses the Google Cloud Speech-to-Text API to convert the voice data into text data (data processing). For example, the text "I'm a little tired today" is generated (output). This text data is sent to the server.

[1550] Step 3: Analyzing text data

[1551] Server: Receives text data sent from the terminal and analyzes the text using a generative AI model (e.g., GPT-4) (input and data processing). For example, it analyzes the content "I'm a little tired" and evaluates the user's health status.

[1552] Server: Based on the analysis results, the server applies frailty criteria and evaluates the user's health status (data calculation). The evaluation result is recorded in the database, for example, in the form of "Frailty Risk: Medium" (output).

[1553] Step 4: Data aggregation and calculation of evaluation scores

[1554] Server: Aggregates daily accumulated evaluation data to comprehensively assess the user's frailty progress. For example, it uses Python's Pandas and NumPy to detect trends such as "feeling tired more than twice a week" from data over the past month (input and data calculation).

[1555] Server: Based on the aggregated data, it calculates an evaluation score according to the prevention criteria. For example, a result such as "Frailty Risk Score: 70 / 100" might be obtained (output). This score is then notified to the user's family via email.

[1556] Step 5: Provide a reminder

[1557] Terminal: Based on evaluation results received from the server, it generates reminders for specific health behaviors and sends them as voice messages. For example, a reminder such as "It would be good to take a 10-minute walk today" is generated using the Google Cloud Text-to-Speech API (input and output).

[1558] User: Perform an action based on a reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[1559] Terminal: Records user feedback, converts it into text data, and sends it to the server (data processing and output).

[1560] Step 6: Analyze feedback and update the database

[1561] Server: Analyzes received feedback data (using a generative AI model) and updates the database (input and data calculation). For example, information that a user has taken a walk is added to the database.

[1562] Server: Reassess frailty risk based on updated data and create the next action plan as needed. This determines the content of the next reminder or notification (output).

[1563] This clearly outlines the overall system's processing flow, enabling the implementation of a program to continuously monitor the progress of frailty in older adults and provide appropriate care.

[1564] (Application Example 1)

[1565] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1566] This invention aims to prevent the progression of frailty and health risks, and support individual health maintenance, by monitoring the health status of elderly people and workers in real time and promoting appropriate health behaviors. In particular, it addresses the challenge of efficiently managing the health of elderly workers in work environments such as factories, and providing appropriate breaks and reminders to improve work efficiency and reduce health risks.

[1567] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1568] In this invention, the server includes means for communicating with elderly people or workers on a daily basis using an interactive speaker or robot equipped with generative artificial intelligence; means for converting voice data acquired by the interactive speaker or robot into text data and analyzing it based on frailty criteria or health risk criteria; means for evaluating the progress of frailty or health risk based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family or facility manager; means for providing health behavior reminders to elderly people or workers based on the evaluation results; and means for receiving feedback from elderly people or workers and updating the database with feedback data. This enables elderly people and workers to understand their own health status on a daily basis and take appropriate action.

[1569] "Generative artificial intelligence" is artificial intelligence designed to interact with humans using technologies such as natural language processing and machine learning.

[1570] An "interactive speaker" is a speaker device that can communicate with the user using voice.

[1571] A "robot" refers to a machine that operates automatically and can interact with and perform tasks with humans.

[1572] The term "elderly" generally refers to people aged 65 and over, and often includes those living after retirement.

[1573] "Workers" refer to people who are engaged in specific tasks in a specific workplace.

[1574] "Communication" is the process by which humans and machines exchange information and understand each other.

[1575] "Audio data" refers to digital data in which audio has been recorded.

[1576] "Text data" refers to digital data recorded as character information.

[1577] "Frailty criteria" are standards used to assess the health status of elderly people.

[1578] "Health risk criteria" are standards used to assess the health status of workers and the elderly and to determine the associated risks.

[1579] "Analysis" is the process of processing data and extracting useful information.

[1580] "Evaluation" is the process of judging a specific state or situation based on the results of an analysis.

[1581] A "database" is a system for structuring, collecting, and managing data.

[1582] "User's family" refers to anyone belonging to the household of a system user.

[1583] A "facility manager" is someone responsible for the working environment and the operation of the facility.

[1584] A "reminder" is a message that notifies a user about a specific action or event.

[1585] "Feedback" refers to responses and reactions from users, and is information that is used for subsequent processing and evaluation.

[1586] This invention relates to a system that uses an interactive speaker or robot to monitor the health status of the elderly and workers in real time and promote appropriate health behaviors. This system operates primarily with a server, a terminal (interactive speaker or robot), and a user.

[1587] 1. Initial setup and user registration

[1588] Terminal (Robot): The terminal provides an interface for registering the user's initial information. For example, it prompts the user to enter the name, age, gender, role, and daily work duties of a worker in a factory.

[1589] User: Follow the interface presented by the robot and enter your information. For example, register information such as "Ichiro Sato, 70 years old, male, line operator."

[1590] Server: Receives user information sent from the terminal and stores it in the database. This collects basic data for monitoring the user's health status.

[1591] 2. Everyday communication

[1592] Terminal (Robot): At scheduled times, it sends voice messages to workers asking about their physical condition and mood. For example, before starting work, it might ask, "Before we begin work, how are you feeling today?"

[1593] User: Responds to the device's questions using voice. For example, "My shoulders are a little stiff."

[1594] Terminal (robot): Records the user's voice response, converts it to text data using the Google Cloud Speech-to-Text API, and sends it to the server.

[1595] Server: Analyzes text data and assesses the user's health status based on health risk criteria. The analysis results are recorded in a database.

[1596] 3. Data accumulation and analysis

[1597] Server: Collects daily accumulated data and comprehensively evaluates workers' health risks. TensorFlow is used for the analysis system.

[1598] Server: Calculates a health risk score, and this score is shared with the facility administrator. For example, it notifies the administrator via email of an evaluation result such as "Health Risk Score: 60 / 100, Caution."

[1599] 4. Support and Reminders

[1600] Terminal (Robot): Based on evaluation results, it provides workers with reminders for appropriate health behaviors. For example, it might remind them, "It would be good to do 10 minutes of stretching."

[1601] User: Follow the reminder and report to the device whether the action was performed. For example, "Yes, I did my stretches."

[1602] Terminal (robot): Receives user feedback and sends it to the server. The feedback data is updated in the database.

[1603] Server: Reassess health risks based on updated data and create the next action plan as needed.

[1604] Hardware and software configuration

[1605] Hardware used: A robot equipped with an Intel Core i7 processor, built-in microphone, speaker, and high-resolution camera.

[1606] Software used: Python, TensorFlow, Google Cloud Speech-to-Text API, and the Django framework.

[1607] Specific example

[1608] Examples of prompts for a generative AI model:

[1609] Robot: "Good morning, Mr. Sato. How are you feeling today?"

[1610] User: "My shoulders are a little stiff."

[1611] In response to this prompt, the AI ​​model sends the generated text data to the server and provides post-analysis reminders and health advice.

[1612] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1613] Step 1: The terminal (robot) prompts the user (worker) to enter initial information.

[1614] Input: User's name, age, gender, role, and daily tasks.

[1615] Data processing: Organize the input information into structured data.

[1616] Output: Structured initial information data.

[1617] Specific operation: The device prompts the user via display and voice, "Please enter your name, age, gender, role, and daily tasks," and the user enters the information.

[1618] Step 2: The terminal (robot) sends initial information to the server.

[1619] Input: Structured initial information data.

[1620] Data processing: Performs data transmission processing from the terminal to the server.

[1621] Output: Initial information stored on the server.

[1622] Specific operation: The device sends information obtained from the user to the server via an API and stores it there.

[1623] Step 3: The terminal (robot) sends a voice message at scheduled times asking about the user's physical condition and mood.

[1624] Input: The prompt text for the dialogue.

[1625] Data processing: Natural language generation processing using generative AI models.

[1626] Output: A voice message asking the user a question.

[1627] Specific action: The device will say aloud, "Before we begin, how are you feeling today?"

[1628] Step 4: The user responds to the device's questions using voice.

[1629] Input: User voice response.

[1630] Data processing: Recording and saving audio data.

[1631] Output: Recorded audio data.

[1632] Specific action: The user says, "My shoulders are a little stiff," and the device starts recording the voice.

[1633] Step 5: The terminal (robot) converts the voice data into text data and sends it to the server.

[1634] Input: Recorded audio data.

[1635] Data processing: Speech recognition and text conversion using the Google Cloud Speech-to-Text API.

[1636] Output: Converted text data.

[1637] Specific operation: Use a speech recognition API to convert the spoken phrase "My shoulders are a little stiff" into text and send it to the server.

[1638] Step 6: The server analyzes the text data and assesses the user's health status based on health risk criteria.

[1639] Input: Converted text data.

[1640] Data processing: Text data analysis and health status assessment using TensorFlow.

[1641] Output: Evaluation results.

[1642] Specific operation: The server receives the text "My shoulders are a little stiff" and performs an analysis based on health risk criteria.

[1643] Step 7: The server records the analysis results in the database.

[1644] Input: Analysis results.

[1645] Data processing: Writing to a database.

[1646] Output: Evaluation results recorded in the database.

[1647] Specific action: Save the analyzed health risk score to the database.

[1648] Step 8: The terminal (robot) provides a reminder to the user based on the evaluation results.

[1649] Input: Evaluation result.

[1650] Data processing: Reminder generation, natural language generation.

[1651] Output: Reminder voice message.

[1652] Specific action: The device will remind you, "It would be good to do 10 minutes of stretching."

[1653] Step 9: The user follows the reminder, takes the action, and reports it to the device.

[1654] Input: User voice report.

[1655] Data processing: Recording and saving audio data.

[1656] Output: Recorded feedback audio data.

[1657] Specific action: The user reports "Yes, I've stretched," and the device records this.

[1658] Step 10: The terminal (robot) sends feedback data to the server.

[1659] Input: Recorded feedback audio data.

[1660] Data processing: Text conversion using speech recognition, transmission to the server.

[1661] Output: Updated feedback text data sent to the server.

[1662] Specific operation: Convert the recorded audio data into text and send it to the server.

[1663] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1664] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[1665] 1. Initial setup and user registration

[1666] Terminal: Provides an interface for registering the user's initial information. The terminal prompts the user to enter their name, age, gender, lifestyle, etc.

[1667] User: Follow the interface displayed on the terminal and enter your basic information. For example, enter information such as "Taro Yamada, 65 years old, male, takes a walk every day."

[1668] Server: Receives user information sent from terminals and stores it in the database. This collects basic data for monitoring the health status of individual users.

[1669] 2. Everyday communication and emotion recognition

[1670] Device: Sends a voice message to the user at a specific time, asking about their physical condition and mood. For example, it might send a voice message every morning at 8:00 AM asking, "Good morning, how are you feeling today?"

[1671] User: Responds to the device's questions using voice. For example, "I'm a little tired today."

[1672] Device: Records the user's voice response and sends the voice data to the emotion engine to recognize emotions. After voice recognition, the emotion engine analyzes the user's emotions and detects fatigue and slight anxiety from a response such as "I'm a little tired today."

[1673] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in the database.

[1674] 3. Data accumulation and analysis

[1675] Server: It aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, by analyzing data from the past month, it detects a trend where users "feel tired more than twice a week" are experiencing increased "anxiety."

[1676] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data. For example, it might send an email to the family with an assessment result such as, "Frailty risk score: 70 / 100, caution. The user has recently shown a stronger tendency to feel anxious."

[1677] 4. Support and Reminders

[1678] Device: Based on evaluation results and recognized emotional states, it provides users with reminders for healthy behaviors. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1679] User: Follow the reminder and report the result to the device. For example, respond with "Yes, I went for a walk."

[1680] Terminal: Receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[1681] Server: Based on the updated data, reassess the risk of frailty and create the next action plan as needed. For example, it might suggest specific actions such as "increase the frequency of walks to three times a week."

[1682] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[1683] The following describes the processing flow.

[1684] Step 1:

[1685] Terminal: Displays an interface for the user to enter basic information such as name, age, gender, and lifestyle.

[1686] Specific action: The screen displays the message "Please enter your name" and a text box.

[1687] Step 2:

[1688] User: Enter your information in the terminal's input field and press the submit button.

[1689] Specific action: The user enters "Taro Yamada" and clicks the submit button.

[1690] Step 3:

[1691] Terminal: Sends the entered information to the server.

[1692] Specific action: The terminal sends user information to the server as an HTTP request in text format.

[1693] Step 4:

[1694] Server: Stores received information in the database.

[1695] Specific action: The server records the following in the database: "ID: 1, Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day".

[1696] Step 5:

[1697] Device: Sends an audio message to the user at a specific time, asking about their physical condition and mood.

[1698] Specific actions: For example, every morning at 8:00 AM, a voice message will be sent asking, "Good morning, how are you feeling today?"

[1699] Step 6:

[1700] User: Responds to the device's questions using voice.

[1701] Specific action: For example, the user responds, "I'm a little tired today."

[1702] Step 7:

[1703] Terminal: Records the user's voice response and sends that voice data to the emotion engine to recognize emotions.

[1704] Specific operation: Voice data is sent to the emotion engine, and the emotion recognition algorithm detects the emotions of "fatigue" and "slight anxiety."

[1705] Step 8:

[1706] Server: Converts emotional data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria.

[1707] Specific actions: Analyze emotional data and text data to assess "mild fatigue" and record it in the database.

[1708] Step 9:

[1709] Server: Aggregates the data accumulated daily and comprehensively evaluates the user's frailty progress.

[1710] Specific actions: For example, by analyzing data from the past month, it detects a trend where people "feel tired more than twice a week" and simultaneously experience increased "anxiety."

[1711] Step 10:

[1712] Server: Calculates an assessment score based on frailty criteria. Notifies the user's family of this assessment score and emotional data.

[1713] Specific actions: For example, an email notification is sent to the family notifying them of the assessment result, such as, "Frailty risk score: 70 / 100, caution advised. The individual has recently shown a stronger tendency to experience anxiety."

[1714] Step 11:

[1715] Device: Provides users with health behavior reminders based on evaluation results and recognized emotional states.

[1716] Specific actions: For example, a voice message could be sent reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1717] Step 12:

[1718] User: Follow the reminder and report the results to the device.

[1719] Specific actions: For example, answering, "Yes, I went for a walk."

[1720] Step 13:

[1721] Terminal: Receives user feedback and sends it to the server.

[1722] Specific operation: Feedback data is sent from the terminal to the server.

[1723] Step 14:

[1724] Server: Saves updated feedback and sentiment data to the database and recalculates the frailty risk assessment.

[1725] Specific actions: Update the database and calculate the new frailty risk score.

[1726] The above describes the specific processing flow in the system of the present invention that incorporates an emotion engine. By recognizing the user's emotions, more advanced and personalized health support becomes possible.

[1727] (Example 2)

[1728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1729] There is a need for a system that routinely monitors the health status of the elderly and detects and prevents the progression of frailty at an early stage. However, it is difficult for the elderly to accurately understand their own health status and report it to medical institutions and family members at the appropriate time. Furthermore, there are limited means of providing advanced health support that takes emotional states into account. To solve this problem, an easy-to-use interface and accurate data collection and analysis are necessary.

[1730] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for providing an interface for registering the user's initial information, means for receiving basic information entered by the user and storing it in a database, means for sending an audio message to the user at a specific time asking about their physical condition and mood, means for recording the user's voice response and sending it to an emotion engine to recognize emotions, means for converting the emotion data and audio data recognized by the emotion engine into text data and analyzing it based on frailty criteria, means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, means for sharing the recorded data with the user's family, means for providing the user with health behavior reminders based on the evaluation results, and means for receiving user feedback and updating the database with feedback data. This enables accurate monitoring of the health status of the elderly and promotion of preventive behaviors.

[1731] An "interface" is a means by which a user interacts with a device or system and inputs and outputs information.

[1732] "Basic information" refers to data that includes personal attributes such as the user's name, age, gender, and lifestyle.

[1733] A "voice message" is a question or instruction delivered by voice from an electronic device.

[1734] An "emotion engine" is a combination of software or hardware used to recognize and analyze a user's emotional state from voice or text data.

[1735] "Text data" refers to data obtained by converting audio data into written text.

[1736] "Frailty criteria" are standards or indicators used to assess the health status and physical function decline of older adults.

[1737] "Frailty progress" refers to the status indicating the progression of a user's health condition and decline in physical function.

[1738] A "database" is a system or software for efficiently storing, managing, and retrieving collected data.

[1739] "Family" refers to individuals who have a close relationship with the user, and usually means the user's relatives.

[1740] A "reminder" is a notification or reminder that serves to remind a user of a specific action or appointment.

[1741] "Feedback" is data used by users to report the results and their impressions to the system after they have taken some kind of action.

[1742] "Analysis" is the process of thoroughly examining and evaluating collected data to derive useful information.

[1743] This invention is a system for monitoring and preventing frailty in the elderly using an interactive speaker equipped with generative artificial intelligence. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it provides more accurate health support. Specific embodiments are described below.

[1744] Initial setup and user registration

[1745] The terminal provides an interface for registering the user's initial information. This interface includes forms accessible via touch panel and voice input. The user enters basic information such as name, age, gender, and lifestyle. For example, they might enter "Taro Yamada, 65 years old, male, takes a walk every day."

[1746] The server receives user information sent from the terminal and stores it in a database. This collects basic data for monitoring the health status of individual users.

[1747] Everyday communication and emotion recognition

[1748] The device will send voice messages to the user at specific times asking about their health and mood. For example, every morning at 8:00 AM, it will send a voice message asking, "Good morning, how are you feeling today?"

[1749] The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today."

[1750] The device records the user's voice responses and sends that voice data to an emotion engine to recognize emotions. The emotion engine analyzes the user's voice and detects, for example, "fatigue" and "anxiety."

[1751] The server converts the emotion data and voice data recognized by the emotion engine into text data and analyzes it based on frailty criteria. The analysis results are recorded in a database.

[1752] Data accumulation and analysis

[1753] The server aggregates the data accumulated daily and comprehensively evaluates the user's progress towards frailty. For example, by analyzing data from the past month, it detects a trend where users are experiencing fatigue more than twice a week, while simultaneously showing an increase in anxiety.

[1754] The server calculates an assessment score based on frailty criteria and notifies the user's family of the assessment score and emotional data. For example, it might send an email notification stating, "Frailty risk score: 70 / 100, caution advised. The user has recently shown a stronger tendency to feel anxious."

[1755] Support and reminders

[1756] The device provides users with health behavior reminders based on evaluation results and recognized emotional states. For example, it might send a voice message reminding them, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1757] The user acts according to the reminder and reports the result to the device. For example, they might respond, "Yes, I went for a walk."

[1758] The device receives user feedback and sends it to the server. The feedback data and the sentiment data associated with it are updated in the database.

[1759] The server reassesss the frailty risk based on the updated data and, if necessary, create the next action plan. For example, it might suggest specific actions such as, "Increase the frequency of walks to three times a week."

[1760] Examples and prompts for generative AI models

[1761] As a concrete example, a user enters "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day" into the interface on their device, and the server saves this information to a database. Furthermore, a voice message is sent from the device every morning at 8:00 AM saying, "Good morning, how are you feeling today?" to which the user replies, "I'm a little tired today."

[1762] Example prompts for generative AI models

[1763] Please describe the details of the system that monitors the user's health status. Include the following information:

[1764] Hardware and software to be used (e.g., conversational speaker with generative artificial intelligence, emotion engine)

[1765] Data collection methods (e.g., voice responses from users)

[1766] Analysis method (Example: Analyze voice data with an emotion engine, then analyze on a server based on frailty criteria)

[1767] Support services provided (e.g., health behavior reminders)

[1768] Specific example (Example: Flow of entering and saving initial user information)

[1769] Thus, by combining the system of the present invention with an emotion engine, it is possible to provide advanced health support based on the user's emotional state. This enables family members living separately to understand the health status of elderly people and provide appropriate support, and also allows elderly people themselves to recognize their own health status and practice effective health behaviors.

[1770] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1771] Processing flow and specific actions

[1772] Initial setup and user registration

[1773] Step 1:

[1774] Input: User's basic information (name, age, gender, lifestyle, etc.)

[1775] Function: The device provides an interface for registering initial information. Using a touch panel or voice input form, it prompts the user to enter basic information such as "Name: Taro Yamada, Age: 65, Gender: Male, Lifestyle: Takes a walk every day."

[1776] Output: Input basic information

[1777] Step 2:

[1778] Input: Basic information entered in Step 1

[1779] Operation: The terminal sends the entered basic information to the server. The data is encrypted and transmitted over the network.

[1780] Output: Basic information sent to the server

[1781] Step 3:

[1782] Input: Basic information sent to the server in Step 2

[1783] Operation: The server receives the entered basic information and stores it in the database. The stored information is used as foundational data to monitor the health status of individual users.

[1784] Output: Basic information stored in the database

[1785] Everyday communication and emotion recognition

[1786] Step 4:

[1787] Input: Set a fixed time (e.g., every morning at 8:00).

[1788] Operation: The device will send a voice message to the user at a set time, asking about their health and mood. For example, it might send a message like, "Good morning, how are you feeling today?"

[1789] Output: Sending a voice message to the user

[1790] Step 5:

[1791] Input: User response to the voice message sent in Step 4

[1792] Operation: The user responds to the device's questions using voice. For example, they might say, "I'm a little tired today." The device records this voice response.

[1793] Output: Recorded user voice data

[1794] Step 6:

[1795] Input: User voice data recorded in Step 5

[1796] Operation: The device sends recorded audio data to the emotion engine. The emotion engine analyzes this audio data and recognizes the user's emotional state. For example, it can detect "fatigue" and "anxiety."

[1797] Output: Analyzed sentiment data

[1798] Step 7:

[1799] Input: Emotional data and voice data analyzed in Step 6

[1800] Operation: The server converts emotional and audio data into text data and analyzes it based on frailty criteria. The results are recorded in a database.

[1801] Output: Frailty analysis results stored in the database

[1802] Data accumulation and analysis

[1803] Step 8:

[1804] Input: Frailty analysis results saved in the database in Step 7

[1805] Operation: The server aggregates data accumulated daily and comprehensively evaluates the user's frailty progress. For example, it might detect a trend from the past month's data showing that users are experiencing fatigue more than twice a week, while simultaneously experiencing increased anxiety.

[1806] Output: Evaluation results of frailty progress

[1807] Step 9:

[1808] Input: Evaluation results of the frailty progress obtained in Step 8

[1809] Operation: The server calculates an assessment score based on frailty criteria. For example, it might assess the user as having a "frailty risk score of 70 / 100". This assessment result is then communicated to the user's family.

[1810] Output: Evaluation results and notification message

[1811] Support and reminders

[1812] Step 10:

[1813] Input: Evaluation results and perceived emotional states obtained in Step 9

[1814] Operation: The device provides the user with health behavior reminders based on the evaluation results. For example, it might send a voice message reminding the user, "It would be good to take a 10-minute walk today. You seem to get tired easily lately, so please don't overexert yourself."

[1815] Output: Health behavior reminders

[1816] Step 11:

[1817] Input: User response to the reminder issued in Step 10

[1818] Operation: The user acts according to a reminder and reports the result to the device. For example, they might provide feedback such as, "Yes, I went for a walk." The device records this feedback.

[1819] Output: Recorded feedback data

[1820] Step 12:

[1821] Input: Feedback data recorded in step 11

[1822] Operation: The device sends user feedback data to the server.

[1823] Output: Feedback data sent to the server

[1824] Step 13:

[1825] Input: Feedback data sent to the server in step 12

[1826] Operation: The server updates the database with the feedback data it receives. Based on the updated data, it reassesss the risk of frailty and creates the next action plan. For example, it might suggest, "Increase the frequency of walks to three times a week."

[1827] Output: Updated data and next action plan

[1828] Through the specific processing steps described above, this system can accurately monitor the health status of elderly individuals and promote appropriate health behaviors.

[1829] (Application Example 2)

[1830] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1831] Monitoring the progression of frailty in older adults and suggesting appropriate health behaviors is crucial, but simply analyzing health status and monitoring emotional state is insufficient. In particular, nutritional status significantly impacts health in older adults, requiring comprehensive health support that includes nutritional assistance. Furthermore, not only are health behavior reminders necessary, but also means to implement specific dietary suggestions.

[1832] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for communicating with the elderly on a daily basis using an interactive speaker equipped with generating artificial intelligence; means for converting voice data acquired by the interactive speaker into text data and analyzing it based on frailty criteria; means for evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database; means for sharing the recorded data with the user's family; means for providing the elderly with health behavior reminders based on the evaluation results; means for receiving feedback from the elderly and updating the feedback data in the database; means for generating meal suggestions that support the nutritional status of the elderly based on the analysis results and emotional state; and means for providing the meal suggestions to the elderly as specific menus in cooperation with a food delivery system. This enables comprehensive support for frailty prevention and nutrition.

[1833] "Generative artificial intelligence" is a type of artificial intelligence that learns from large amounts of data and has the ability to engage in natural conversations and make decisions like a human.

[1834] An "interactive speaker" is a speaker device that has the function of receiving voice input from the user and responding with voice output.

[1835] "Frailty criteria" are standards used to assess the progression of frailty in older adults based on their physical strength and health status.

[1836] "Analysis results" refer to conclusions and evaluation data obtained based on the analysis of voice data and emotional states.

[1837] A "database" is an information storage system for systematically accumulating and managing large amounts of data.

[1838] A "health behavior reminder" is a notification or alert designed to encourage users to engage in healthy behaviors.

[1839] "Feedback from the elderly" refers to information and opinions provided by elderly individuals regarding their behavior and physical condition.

[1840] "Meal suggestions to support nutritional status" refer to meal menus recommended based on the health status and nutritional needs of elderly individuals.

[1841] A "food delivery system" is a service system that delivers meals ordered by users to a specified location.

[1842] This invention is a system that uses an interactive speaker equipped with artificial intelligence to monitor the progress of frailty in elderly individuals and provides comprehensive health support, including nutritional support. Specific embodiments are described below.

[1843] Hardware and software to be used

[1844] Hardware: Smartphones, smart speakers

[1845] Software: Emotion engine (e.g., Microsoft Azure Emotion API), Natural Language Processing engine (e.g., OpenAI's GPT-3), Food Delivery API

[1846] System Configuration

[1847] 1. User Registration

[1848] The terminal provides an interface that allows the user to input basic information such as name, age, gender, and lifestyle. The information entered by the user is sent to the server and stored in a database.

[1849] 2. Daily communication and monitoring of health status

[1850] At specific times, the device sends a voice message to the user asking about their health and mood. For example, every morning at 8:00 AM, it might ask, "Good morning. How are you feeling today?" The user's voice response is recorded by an interactive speaker and sent to a server. The recorded voice data is converted to text and analyzed using an emotion engine.

[1851] 3. Accumulation and evaluation of analysis results

[1852] The converted text and sentiment data are stored on the server, and the progress of frailty is evaluated based on past data. The evaluation results are stored in a database and shared with the user's family.

[1853] 4. Generating and providing meal suggestions

[1854] Based on the analysis results and emotional state, meal suggestions are generated to support the user's nutritional status. These meal suggestions are integrated with a food delivery system and provided as specific menus. For example, if fatigue or stress levels are high, a balanced and nutritious menu will be suggested.

[1855] example

[1856] For example, if a 65-year-old male user answers "I'm a little tired today" during his daily health check, the voice data is analyzed through an emotion engine. As a result, fatigue and mild anxiety are detected. The server analyzes the data from the past month and determines that the user's fatigue is on an increasing trend. As a result, a nutritionally balanced "chicken and vegetable plate" is suggested for lunch and made available for order through a food delivery system.

[1857] Example of a prompt

[1858] Check the user's daily physical and emotional state and analyze the data weekly. Then, use an emotion engine to assess increases and decreases in fatigue and anxiety, suggest a balanced meal, and display that menu in a form that can be ordered via food delivery.

[1859] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1860] Step 1:

[1861] Input: The user enters basic information such as name, age, gender, and lifestyle into the terminal.

[1862] Operation: The terminal collects the entered information and sends it to the server.

[1863] Output: The server saves the received user information to the database.

[1864] Step 2:

[1865] Input: The device will send an audio message at a specific time asking about the elderly person's physical condition and mood.

[1866] Operation: The user responds by voice about their physical condition and mood. For example, they might say, "I'm a little tired today."

[1867] Output: The terminal records the user's voice response and sends the audio data to the server.

[1868] Step 3:

[1869] Input: The server sends the received audio data to the emotion engine.

[1870] Operation: The emotion engine analyzes audio data and identifies emotional states.

[1871] Output: The server receives the analysis results from the emotion engine and saves them to the database as text data and emotion data.

[1872] Step 4:

[1873] Input: The server aggregates and analyzes daily voice data and emotion data stored in the database.

[1874] Function: By analyzing data, the system evaluates the user's progress in frailty.

[1875] Output: The server calculates the evaluation result as a frailty score and records it in the database.

[1876] Step 5:

[1877] Input: The server analyzes the user's health trends based on frailty score and emotional state data.

[1878] Operation: Uses a natural language processing engine to generate meal suggestions to support the user's nutritional status.

[1879] Output: The server uses a generation AI model to generate individually customized meal suggestions.

[1880] Step 6:

[1881] Input: The server integrates with the food delivery system to provide the generated meal suggestions to the user.

[1882] Operation: The device notifies the user of specific menu options via voice or text.

[1883] Output: The user reviews the suggested menu and places an order through the food delivery service.

[1884] Step 7:

[1885] Input: Users input feedback into the terminal regarding the results of their meals and any changes in their physical condition.

[1886] Operation: Feedback data is sent from the terminal to the server.

[1887] Output: The server updates the database with feedback data and incorporates it into the next analysis.

[1888] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1889] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1890] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1891] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1892] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1893] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1894] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1895] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1896] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1897] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1898] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1899] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1900] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1901] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1902] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1903] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1904] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1905] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1906] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1907] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1908] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1909] The following is further disclosed regarding the embodiments described above.

[1910] (Claim 1)

[1911] A means of communicating with the elderly on a daily basis using an interactive speaker equipped with generative artificial intelligence,

[1912] A means for converting audio data acquired by the aforementioned interactive speaker into text data and analyzing it based on frailty criteria,

[1913] A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database,

[1914] A means of sharing the recorded data with the user's family,

[1915] Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly,

[1916] A means of receiving feedback from elderly people and updating the feedback data in a database.

[1917] A system that includes this.

[1918] (Claim 2)

[1919] The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person and generating alerts based on the results of a frailty assessment.

[1920] (Claim 3)

[1921] The system according to claim 1, further comprising means for recording the daily activities of an elderly person by voice input through the interactive speaker and transmitting the information to a server.

[1922] "Example 1"

[1923] (Claim 1)

[1924] A means of communicating with the elderly on a daily basis using an interactive voice output device equipped with generative artificial intelligence,

[1925] A means for converting voice data acquired by the aforementioned interactive voice output device into text data and analyzing it based on frailty criteria,

[1926] A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database,

[1927] A means of sharing the recorded data with the user's family,

[1928] Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly,

[1929] A means of receiving feedback from elderly people and updating the feedback data in a database,

[1930] A means for recording the user's voice response, converting it into text data using speech recognition technology, and analyzing it,

[1931] A means of aggregating daily data, calculating an evaluation score based on prevention criteria, and notifying the family of the results,

[1932] A system that includes this.

[1933] (Claim 2)

[1934] The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person and generating alerts based on the results of a frailty assessment.

[1935] (Claim 3)

[1936] The system according to claim 1, further comprising means for recording the daily activities of an elderly person by voice input through the interactive voice output device and transmitting the information to a server.

[1937] "Application Example 1"

[1938] Claims based on a new invention

[1939] (Claim 1)

[1940] A means of communicating on a daily basis with the elderly or workers using an interactive speaker or robot equipped with generative artificial intelligence,

[1941] A means for converting voice data acquired by the aforementioned interactive speaker or robot into text data and analyzing it based on frailty criteria or health risk criteria,

[1942] A means of evaluating the progress of frailty or health risk based on the analysis results and recording the evaluation results in a database,

[1943] A means of sharing the recorded data with the user's family or facility manager,

[1944] Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly or workers,

[1945] A means of receiving feedback from elderly people or workers and updating the feedback data in a database.

[1946] A system that includes this.

[1947] (Claim 2)

[1948] The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person or worker and generating alerts based on frailty assessment results or health risk assessment results.

[1949] (Claim 3)

[1950] The system according to claim 1, further comprising means for recording the daily activities of an elderly person or worker by voice input through the interactive speaker or robot and transmitting the information to a server.

[1951] "Example 2 of combining an emotion engine"

[1952] (Claim 1)

[1953] A means of providing an interface for registering the user's initial information,

[1954] A means of receiving basic information entered by the user and saving it to a database,

[1955] A method for sending voice messages to users at specific times to ask about their physical condition and mood,

[1956] A means of recording the user's voice response and sending it to an emotion engine to recognize emotions,

[1957] A means for converting emotional data and voice data recognized by an emotion engine into text data and analyzing it based on frailty criteria,

[1958] A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database,

[1959] A means of sharing the recorded data with the user's family,

[1960] Based on the aforementioned evaluation results, a means for providing users with reminders for healthy behaviors,

[1961] A means of receiving user feedback and updating the feedback data in a database,

[1962] A system that includes this.

[1963] (Claim 2)

[1964] The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person and generating alerts based on the results of a frailty assessment.

[1965] (Claim 3)

[1966] The system according to claim 1, further comprising means for recording the daily activities of an elderly person by voice input through the interactive speaker and transmitting the information to a server.

[1967] "Application example 2 when combining with an emotional engine"

[1968] (Claim 1)

[1969] A means of communicating with the elderly on a daily basis using an interactive speaker equipped with generative artificial intelligence,

[1970] A means for converting audio data acquired by the aforementioned interactive speaker into text data and analyzing it based on frailty criteria,

[1971] A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database,

[1972] A means of sharing the recorded data with the user's family,

[1973] Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly,

[1974] A means of receiving feedback from elderly people and updating the feedback data in a database,

[1975] A means for generating meal suggestions to support the nutritional status of elderly people based on analysis results and emotional state,

[1976] A means of providing the aforementioned meal suggestions to the elderly as specific menus in conjunction with a food delivery system.

[1977] A system that includes this.

[1978] (Claim 2)

[1979] The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person and generating alerts based on the results of a frailty assessment.

[1980] (Claim 3)

[1981] The system according to claim 1, further comprising means for recording the daily activities of an elderly person by voice input through the interactive speaker and transmitting the information to a server. [Explanation of Symbols]

[1982] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of communicating with the elderly on a daily basis using an interactive speaker equipped with generative artificial intelligence, A means for converting audio data acquired by the aforementioned interactive speaker into text data and analyzing it based on frailty criteria, A means of evaluating the progress of frailty based on the analysis results and recording the evaluation results in a database, A means of sharing the recorded data with the user's family, Based on the aforementioned evaluation results, a means for providing health behavior reminders to the elderly, A means of receiving feedback from elderly people and updating the feedback data in a database. A system that includes this.

2. The system according to claim 1, further comprising means for continuously monitoring the health status of an elderly person and generating alerts based on the results of a frailty assessment.

3. The system according to claim 1, further comprising means for recording the daily activities of an elderly person by voice input through the aforementioned interactive speaker and transmitting the information to a server.

Images