system

Abstract

We provide the system. [Solution] A means of analyzing health data using AI algorithms and evaluating specific health risks, A means of safely transporting users to their destination using a mobility assistance device equipped with an autonomous driving function, A means of detecting abnormalities and issuing alarms according to the health status, A means of managing schedules and providing reminders based on user instructions, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an aging society, it is an issue to eliminate the physical and psychological burdens, time constraints, and lack of expertise related to caregiving, and improve the quality of life of the elderly. Also, it is necessary to quickly grasp changes in the health condition and enable appropriate responses. By smoothly supporting the safe movement of users and enabling remote communication with family members, it is also required to reduce feelings of loneliness and anxiety and provide a rich life.

Means for Solving the Problems

[0005] This invention provides a means to analyze a user's health data in real time using an AI algorithm and to immediately assess health risks. It also adds a means to enable users to move safely inside and outside facilities using a mobility support device with autonomous driving capabilities. Furthermore, it constructs a system that generates alarms based on health data and anomaly detection, notifying care staff and family members. It also includes a means to manage the user's schedule and provide reminders to help them reliably perform important daily tasks. This reduces the burden of caregiving and improves the user's quality of life.

[0006] An "AI algorithm" refers to a computational procedure designed using artificial intelligence technology for the purpose of analyzing data and solving problems.

[0007] "Health data" refers to numerical values ​​and data that indicate the user's physical health status, such as heart rate, blood pressure, and body temperature.

[0008] "Health risk" refers to the results of an assessment of the likelihood of disease or disability occurring in the user's health condition.

[0009] "Autonomous driving function" refers to technology that enables vehicles and equipment to perceive their environment and move and operate autonomously without requiring human intervention.

[0010] A "mobility assistance device" refers to a machine that helps elderly people and people with disabilities reach their destination safely.

[0011] An "alarm" refers to a notification or warning issued by a system when it detects an anomaly, prompting a quick response.

[0012] "Schedule management" refers to the act of planning and monitoring events and activities based on predetermined timeframes.

[0013] A "reminder" refers to a notice or warning given in advance to encourage the completion of an important appointment or task.

[0014] "Care staff" refers to professionals who provide life support for the elderly and those requiring care.

[0015] "Notification" refers to the communication and information sent from the system to users and related parties.

Brief Explanation of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. <( [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Modes for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory where information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] To implement this invention, it is first necessary to provide the elderly user with a smart device. This device has the function of continuously monitoring and collecting health data such as heart rate, blood pressure, and body temperature in real time. This data is also transmitted to a terminal via a communication means. The terminal organizes the received health data and transfers it to a server in an appropriate format.

[0038] The server runs a program that includes an AI algorithm to analyze the received data. When the server analyzes the data and evaluates and detects abnormal values ​​or health risks, it generates an alarm. For example, if a user's blood pressure rises to a dangerous level, the server immediately notifies the care staff and the user's terminal. This enables a rapid response.

[0039] Furthermore, users can safely move around the facility using a mobility assistance device with autonomous driving capabilities. Users specify their desired destination via voice input, and the terminal transmits this information to the robot, which then guides them along the optimal route to their destination.

[0040] The device also has a function to update and manage the user's schedule on a daily basis. When the user records an appointment using voice commands, the device sends it to the server and notifies the user at the set time as a reminder. In this way, the worry of forgetting important data or appointments is reduced, improving the quality of daily life.

[0041] As a concrete example, suppose a user registers the time to take their medication the following day on their device. The device sends this information to a server, which then uses that information to provide the user with a voice reminder at the designated time. This ensures that the medication is taken on time.

[0042] The system of this invention allows users to enjoy a safer and more controlled living environment, significantly improving their quality of life.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] The user wears a smart device to measure health data. The device continuously measures heart rate, blood pressure, and body temperature, generating data in real time.

[0046] Step 2:

[0047] The terminal receives health data from smart devices. It then checks for data inconsistencies or omissions and records the results.

[0048] Step 3:

[0049] The device converts the verified health data into a format suitable for the database and sends it to the server. A timestamp is attached to the transmitted data.

[0050] Step 4:

[0051] The server analyzes health data received from the terminal in real time. An AI algorithm on the server processes this data and evaluates any anomalies or health risks.

[0052] Step 5:

[0053] If the server detects an anomaly, it immediately generates an alarm. The alarm includes specific anomaly parameters and is sent to the terminals of both care staff and users.

[0054] Step 6:

[0055] The user instructs the autonomous mobility assistance device on its destination. The voice instructions are registered on a terminal and then transmitted from the terminal to the robot.

[0056] Step 7:

[0057] The terminal transmits destination information to the robot. The robot uses a map of the facility to calculate the optimal route and begins moving automatically.

[0058] Step 8:

[0059] The user enters their schedule into the device using voice or touch input. The device then sends this schedule data to the server.

[0060] Step 9:

[0061] The server receives the schedule data and stores it in the database. The server generates reminders according to the set time and prepares to send them to the user's device.

[0062] Step 10:

[0063] The device receives reminders from the server and notifies the user at the specified time. This ensures that the user can reliably complete important appointments and tasks.

[0064] (Example 1)

[0065] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0066] In modern society, with its aging population, managing the health of the elderly and ensuring safe mobility are crucial issues. Traditional methods fail to adequately provide real-time monitoring of individual health information, rapid response to emergencies, and support for daily activities. Therefore, a comprehensive support system is needed to enable the elderly to live safely and comfortably.

[0067] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0068] In this invention, the server includes means for acquiring biometric data from a measuring device for detecting health information, means for converting the acquired biometric data into a data structure and transmitting it to an information processing device, and means for analyzing the received biometric data using a generative AI model and evaluating the risk of specific health conditions. This enables real-time monitoring of the user's health status, improving safety and quality of life.

[0069] A "measuring device" is a device that measures biological data and monitors the user's health status in real time.

[0070] "Biometric data" refers to information that represents the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0071] "Data structure" refers to the format or style used to convert biometric data into a format that an information processing device can recognize.

[0072] An "information processing device" is a device that has the function of receiving biological data, analyzing it, or transmitting it to other devices.

[0073] A "generative AI model" is a machine learning-based algorithm used to analyze received data and recognize patterns.

[0074] "Health risk" refers to factors that may affect a specific physical condition, based on the user's health data.

[0075] A "mobility assistance device" is a device equipped with an autonomous driving function that supports users in safely traveling to their destination.

[0076] "Voice commands" refer to commands or requests that a user inputs into a device by speaking.

[0077] A "notification" is a message sent to inform a user or a third party of pre-configured information or alerts.

[0078] This invention is a system that supports the health management and safe mobility of the elderly. The following hardware and software are required to implement this invention.

[0079] The user wears a measuring device that measures heart rate, blood pressure, body temperature, etc. This device incorporates heart rate and blood pressure sensors, allowing for real-time collection of biometric data. This data is transmitted from the measuring device to the terminal using communication methods such as Bluetooth or Wi-Fi.

[0080] The device converts the received biometric data into a standardized data structure and sends it to the server. The server analyzes the received data using an analysis algorithm based on a generative AI model. Specifically, it detects anomalies and assesses the risk of specific health conditions. If an anomaly is detected, the server immediately generates an alarm and notifies remote care staff or family members. This notification is sent via email or a dedicated app.

[0081] Furthermore, users can utilize mobility assistance devices equipped with autonomous driving capabilities through voice commands. The terminal recognizes the user's voice commands and transmits destination information to the mobility assistance device. The mobility assistance device calculates the optimal route within the facility and safely guides the user to their destination. In addition, users can register their daily schedules on the terminal using voice commands. Registered schedules are stored on the server and notified to the user as reminders at the specified time.

[0082] As a concrete example, consider a scenario where a user registers the time to take their medication the following day by voice on their device. The device sends this information to a server, which then performs a process of providing the user with a voice and visual reminder at the designated time. This helps users manage their health and contributes to a safer and more comfortable living environment.

[0083] Examples of prompt statements include the following:

[0084] "Please explain how this system monitors the user's blood pressure and detects abnormalities."

[0085] "Please explain the process of how the vehicle autonomously guides users to their desired location."

[0086] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0087] Step 1:

[0088] The user wears a measuring device that detects biometric data such as heart rate, blood pressure, and body temperature. The input is the user's biological state, and the output is various biometric data. The measuring device collects this data in real time using built-in sensors and transmits it to the terminal in digital format via Bluetooth.

[0089] Step 2:

[0090] The terminal organizes the biometric data received from the measuring device and converts it into a data structure. The input is the biometric data from the measuring device, and the output is the formatted data structure. For example, the terminal converts the biometric data into JSON format, formatting it so that the server can easily analyze it. This process ensures data consistency.

[0091] Step 3:

[0092] The terminal sends the formatted data structure to the server. The input is the converted data structure, and the output is the biometric data received by the server. Using TCP / IP as the communication protocol ensures reliable data transfer.

[0093] Step 4:

[0094] The server applies a generative AI model to analyze the received biometric data. The input is biometric data sent from the terminal, and the output is the analysis result. The generative AI model helps to detect anomalies by comparing them with past data and assessing the risk of specific health conditions. For example, if the server determines that a blood pressure value exceeds the standard value, it flags it as an anomaly.

[0095] Step 5:

[0096] Based on the analysis results, the server generates an alarm if an anomaly is detected and notifies remote care staff or family members. The input is the analysis results, and the output is a notification message. The notification is sent via SMS or a dedicated app to quickly inform recipients of the situation.

[0097] Step 6:

[0098] The user specifies the destination to the mobility assistance device using voice commands. The input is the user's voice command, and the output is data as an instruction to the device. The terminal recognizes and analyzes these voice commands and transmits the destination data to the mobility assistance device.

[0099] Step 7:

[0100] Mobility assistance devices calculate the optimal route based on destination information transmitted from a terminal and guide the user safely. The input is destination information, and the output is physical movement and guidance. The device uses a recognition system installed within the facility to navigate to the destination while avoiding obstacles.

[0101] Step 8:

[0102] The user registers schedules and sets reminders on the device via voice commands. The input is voice data from the user, and the output is schedule data. The device analyzes the user's commands, sends the schedule to the server, and prepares to notify the user at the necessary time.

[0103] These processing steps enable the system to function effectively, supporting the health management and safe mobility of the elderly.

[0104] (Application Example 1)

[0105] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0106] In recent years, the importance of health management and lifestyle support for the elderly has increased. In particular, there is a need for technologies that efficiently collect and analyze health data, provide mobility support, and manage schedules. However, current systems lack the visual support necessary for the elderly to access information in real time, limiting their ability to ensure a safe and smooth life for them.

[0107] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0108] In this invention, the server includes means for analyzing health-related data using an AI algorithm and evaluating specific health risks, means for providing route information using a visual information display device, and means for detecting abnormalities according to the health status and issuing alarms. This enables elderly people to live independently and engage in safe and smooth daily activities.

[0109] An "AI algorithm" is a set of computational procedures that use artificial intelligence to analyze data and detect patterns and anomalies.

[0110] "Health data" refers to a collection of information that indicates an individual's health status, such as heart rate, blood pressure, and body temperature.

[0111] A "means for assessing health risks" is a function that determines the possibility of potential health problems based on collected health data.

[0112] A "mobility support device with autonomous driving capabilities" is a device that has autonomous driving functions to safely transport users to their destination.

[0113] A "visual information display device" is a device that provides users with visual information in real time.

[0114] "Means for detecting abnormalities and issuing alarms" refers to a function that automatically generates and notifies an alarm when a preset threshold value is exceeded.

[0115] "A means of managing schedules and providing reminders" refers to a function that records the user's schedule and sends notifications at specified times.

[0116] To implement this invention, first, a smart device for monitoring health data is provided to the user. The smart device measures data such as heart rate, blood pressure, and body temperature and transmits it to the terminal in real time. The terminal organizes this data and sends it to a server. The server analyzes the received health data using an AI algorithm and issues an alarm if it detects any abnormal values. The alarm is notified to the user and their family or care staff who are located remotely.

[0117] Furthermore, users can safely travel to their destination using a mobility assistance device equipped with autonomous driving capabilities. This device provides route information to a visual information display device based on the destination specified by the user via voice input, and provides real-time navigation. When a user enters their schedule by voice through an application installed on their smartphone, the device manages it as a schedule and provides reminders at the specified time.

[0118] Specifically, the server analyzes the collected data using AI algorithms such as Firebase ML Kit and TENSORFLOW® to detect anomalies. The data is stored in databases such as Firestore and SQLite. Google® Glass® and other similar devices are used as visual information display devices. The notification function uses Firebase Cloud Messaging to send information to users and care staff in real time.

[0119] For example, when a user leaves home to go to a cafe, route information is visually displayed on their glasses, allowing them to travel safely and with peace of mind. If a user records the time for their medication the next day in the app, they will receive a reminder notification at the specified time, so they don't have to worry about forgetting to take their medication.

[0120] As an example of using a generative AI model, consider the following prompt message: "Analyze health monitoring data and generate a prompt that warns if the heart rate is higher than normal."

[0121] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0122] Step 1:

[0123] The user wears a smart device on their body to collect data such as heart rate, blood pressure, and body temperature. The input is health data from the sensors, and the output is the transmission of this data to the device.

[0124] Step 2:

[0125] The terminal stores the received health data in a database, standardizes the format, and then transfers it to the server. The input is raw data received from the smart device, and the output is health data in a standardized format.

[0126] Step 3:

[0127] The server analyzes received health data using an AI algorithm. Data processing includes anomaly detection, distinguishing between normal and abnormal values. The input is organized health data, and the output consists of data analysis results and anomaly detection information.

[0128] Step 4:

[0129] The server generates an alarm if it detects an anomaly based on the analysis results. The generated alarm is sent to care staff or family members located remotely. The input is anomaly detection information, and the output is the alarm notification.

[0130] Step 5:

[0131] The user specifies their desired destination by voice via their smartphone. The input is the user's voice command, and the output is destination information.

[0132] Step 6:

[0133] The terminal transmits destination information to a mobility assistance device equipped with an autonomous driving function, which calculates the optimal route and provides real-time navigation information to a visual information display device. The input is destination information, and the output is route information and navigation instructions.

[0134] Step 7:

[0135] The user registers their schedule using voice input on the device. The input is the user's voice-generated schedule information, and the output is the recorded schedule information.

[0136] Step 8:

[0137] The device stores schedule information, generates a reminder at the specified time, and notifies the user. The input is schedule information, and the output is the reminder notification.

[0138] Step 9:

[0139] The server utilizes a generative AI model to generate prompt messages related to anomaly detection in health data and schedule management. Inputs are analytical data and schedule information, while output is the prompt messages.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] This invention provides a system incorporating an emotion engine to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. First, the user is provided with a terminal equipped with a voice recognition device and a camera, through which voice and facial expression data are collected. This data is analyzed by the terminal to recognize the user's emotional state in real time. The emotion engine uses algorithms to identify emotions such as joy, anxiety, anger, and sadness, and based on this, health risks and quality of life are assessed.

[0142] The terminal is responsible for transmitting emotional data collected from users to a server. The server integrates and analyzes the emotional data and health data to predict health risks if stress and anxiety persist for a long period. Based on this analysis, it generates alarms and reminders and sends them to users and care staff. It also provides a function to automatically adjust care plans and schedules accordingly if the user is in a specific emotional state.

[0143] As a concrete example, consider a scenario where a user utilizes the system: If, one afternoon, the user displays a sad expression while watching television, the terminal instantly analyzes this using an emotion engine and sends the data to the server. The server then combines this data with the user's health data for the day, and if it determines that the user may already be experiencing prolonged stress, it sends a notification to the care staff suggesting stress management activities. This process allows care staff to take swift action and support the user's mental well-being.

[0144] The system incorporating the emotion engine of this invention is a means of improving the overall quality of life of users by focusing not only on health management but also on emotional care. This system provides a form that can address the emotional challenges that users face in their daily lives.

[0145] The following describes the processing flow.

[0146] Step 1:

[0147] Users go about their daily lives using a device equipped with a voice recognition device and a camera. The device collects not only the user's voice commands but also image data of their facial expressions in real time.

[0148] Step 2:

[0149] The device analyzes the user's emotional state using an emotion engine based on collected voice and facial expression data. It identifies emotions such as joy, anxiety, anger, and sadness, and records their changes.

[0150] Step 3:

[0151] The device sends the results of its emotion analysis to the server. The transmitted information includes the type and intensity of the emotion, along with other relevant time information.

[0152] Step 4:

[0153] The server integrates the received emotional data with existing health data to assess the user's overall health status. This assessment pays particular attention to the effects of persistent stress and anxiety on the body.

[0154] Step 5:

[0155] If the server detects a severe stress level, it will generate an alert for care staff and family members and send a notification to prompt necessary intervention.

[0156] Step 6:

[0157] The server further adjusts the user's schedule based on their emotions and health status, and offers suggestions to reduce stress. This adjustment is sent to the user's device for them to review.

[0158] Step 7:

[0159] The device notifies the user of appropriate reminders and changes in care plans, helping to optimize daily activities for health and emotional well-being.

[0160] (Example 2)

[0161] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0162] In managing the health of the elderly, traditional methods often focus solely on physical health, leaving insufficient assessment of health risks associated with changes in emotional state and inadequate support for daily living. Furthermore, there is a lack of measures to prevent health problems arising from the accumulation of long-term stress and anxiety.

[0163] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0164] This invention includes a server that uses an emotion engine to analyze emotional states from voice data and facial expression data and evaluates health risks based on specific emotions; means for integrating the analyzed emotional data and health data to predict long-term stress risks; and means for automatically adjusting care plans and schedules according to the emotional data. This enables rapid response to changes in emotional states and allows for comprehensive health risk assessment and lifestyle support.

[0165] An "emotion engine" is a part of a system that includes algorithms for identifying a user's emotional state by analyzing voice data and facial expression data.

[0166] "Health risk" is an indicator that shows the possibility of future health problems or abnormalities based on the user's emotional state and health data.

[0167] "Emotional data" refers to information about a user's emotional state obtained through the analysis of voice data and facial expression data.

[0168] "Health data" refers to information that indicates a user's physical health status, and generally includes physiological indicators such as blood pressure and heart rate.

[0169] A "care plan" is a plan of support and care that is formulated based on the user's emotional state and health condition.

[0170] "Automatic schedule adjustment" is a process that automatically modifies or optimizes existing activities and appointments based on the user's emotional state.

[0171] An "alert" is a notification sent to supporters to alert them when a user's health or emotional state exceeds predetermined criteria.

[0172] A "reminder" is a notification that reminds users or supporters of scheduled events or necessary activities.

[0173] This invention utilizes a system combining an emotion engine and health data analysis functions to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. A voice recognition device and a camera-equipped terminal are used to collect user voice and facial expression data. The terminal acquires data in real time and performs analysis using a locally installed emotion engine. The emotion engine implements machine learning algorithms to identify emotions such as joy, anxiety, anger, and sadness.

[0174] The device sends the analysis results to the server. The server combines emotional data with accumulated health data to predict health risks resulting from prolonged stress and anxiety. This typically utilizes physiological data such as the user's blood pressure and heart rate. Based on the analysis results, alarms and reminders are automatically generated and sent to the user and caregivers. Furthermore, if the user's emotional state meets certain conditions, care plans and schedules are automatically adjusted.

[0175] For example, if a user displays a sad expression while watching television one afternoon, the device instantly analyzes this using its emotion engine and sends the data to the server. The server integrates this data with the user's health data for the day, and if it determines that the user may be experiencing ongoing stress, it sends a notification to a support worker suggesting stress management activities. This allows support workers to take action quickly.

[0176] An example of a prompt for a generative AI model might be, "We want to design a system that analyzes the emotions of elderly people in real time and predicts their health risks." By using this system, users can respond flexibly to emotional challenges in their daily lives and improve their overall quality of life.

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The device collects the user's voice and facial expression data in real time using a voice recognition device and a camera. The input data consists of audio of everyday conversations and video of facial expressions, which are used to prepare for the identification of specific emotional states. The output is raw audio and video data formatted in a way that is necessary for analysis.

[0180] Step 2:

[0181] The device analyzes collected voice and facial expression data using an emotion engine. The emotion engine uses machine learning algorithms to evaluate voice intonation and facial features, identifying emotions such as joy, anxiety, anger, and sadness from the input data. Specifically, it quantifies human emotions using voice waveform analysis and image processing techniques. The output obtained at this stage is emotion data representing the user's identified emotions.

[0182] Step 3:

[0183] The terminal transmits the analyzed sentiment data to the server via the network. The input is the sentiment data identified in the previous step, and the output is the transmitted sentiment data. Metadata such as timestamps and location information are also sent during this process and used for subsequent analysis.

[0184] Step 4:

[0185] The server integrates and analyzes received emotional data with existing health data. Inputs include emotional data and physiological data from health monitoring devices. Based on this, the server performs statistical modeling to predict long-term stress and health risks. The output is a risk assessment result, which is notified to support staff or users as needed.

[0186] Step 5:

[0187] The server generates and sends alarms and reminders to users and support staff based on the results of the risk assessment. The input is the risk assessment results, and the output is alarms and reminders for management. Specifically, it sends a warning notification to the support staff's smartphone and displays a message on the user's device.

[0188] Step 6:

[0189] The server automatically adjusts care plans and schedules based on the emotional data obtained. Inputs are the individual user's emotional state and health risk assessment results. Outputs are new activity proposals and adjustment plans incorporated into the user's schedule. This process determines the optimal care policy by comparing it with the user's past behavioral history.

[0190] (Application Example 2)

[0191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0192] In managing the health of the elderly, it is necessary to understand not only their physical health but also their emotional state in real time and provide mental support. Conventional systems make it difficult to provide comprehensive health management that takes emotional states into account, and there is a problem in that it is difficult to respond quickly and appropriately to maintain the mental health of the elderly.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0194] This invention includes a server that uses an AI algorithm to analyze health data and assess specific health risks, an emotion recognition engine to analyze the user's emotional state in real time and assess their mental health, and a means to automatically adjust the care support plan according to the user's emotional state. This enables the integration and analysis of emotional and health data to provide appropriate care support quickly.

[0195] An "AI algorithm" is a method of analyzing data using artificial intelligence technology and mimicking human intellectual work.

[0196] "Health data" refers to various measurements and observations that indicate an individual's physical and mental state.

[0197] "Health risk" refers to potential dangers that could affect an individual's health.

[0198] An "emotion recognition engine" is a technology that analyzes voice and facial expression data to identify the user's inner emotional state.

[0199] "Real-time" refers to the temporal characteristic of being able to process and use data immediately.

[0200] "Mental health" refers to a state in which an individual's emotional and psychological state is good and well-adjusted.

[0201] A "care support plan" is a specific action plan formulated to provide appropriate care based on an individual's health and emotional state.

[0202] "Automatic adjustment" refers to the characteristic of a system that can change its settings to an optimal state on its own without external instructions or intervention.

[0203] This system consists of a device worn by the user and a server in the cloud. The device is equipped with a microphone and camera to collect the user's voice and facial expression data in real time. Based on this data, an AI algorithm analyzes the user's emotional state in an emotion recognition engine. This analysis utilizes software specifically designed for emotion recognition, such as Microsoft® Azure® Face API and Google Cloud Vision API.

[0204] The collected data is sent to a server where it is integrated with health data and analyzed. The server uses AI algorithms to identify health risks. Furthermore, it automatically adjusts care plans in response to changes in emotional state, providing appropriate support to users and care staff. This function makes it possible to prevent mental health problems such as long-term stress and feelings of isolation.

[0205] For example, if a user shows a sad expression while watching television in the living room, the device immediately sends that data to a server. The server analyzes the data and notifies care staff of the possibility of an emotional change. Based on this information, the staff can suggest appropriate conversations and activities for the user, thereby strengthening their emotional support.

[0206] An example of a prompt in a generative AI model is, "If the user is showing a sad expression, what kind of caregiving response would you suggest?" By using this prompt, the AI ​​model can come up with caregiving responses that are appropriate to the user's emotional state.

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The device collects the user's voice and facial expressions using a microphone and camera. It acquires real-time audio and video data as input, converts it to a digital format, and prepares it for transmission to the server.

[0210] Step 2:

[0211] The server receives audio and video data transmitted from the terminal. The input data is passed to an AI algorithm for speech recognition and image analysis. This algorithm extracts features from the voice and facial expressions and performs calculations to identify the user's emotional state. As output, it generates digital data representing the emotion.

[0212] Step 3:

[0213] The server analyzes the user's health status along with health data based on emotional data identified by the emotion recognition engine. It integrates the input emotional and health data to assess health risks such as stress and anxiety. As output, it generates a digital report regarding the presence or absence of health risks.

[0214] Step 4:

[0215] The server determines the necessary actions based on the analysis results. Based on the entered health risk information, it automatically adjusts the care support plan as needed. As output, it creates an adjusted care plan to present to the care staff.

[0216] Step 5:

[0217] The server sends the adjusted care plan and necessary notifications to the care staff. Using the adjusted care plan and notification data as input, it generates prompts through a generative AI model to suggest appropriate care responses. As output, it delivers notifications to the staff's smart devices.

[0218] Step 6:

[0219] Users receive support from care staff as needed. The input is the maintenance of mental and physical health through care and activities provided by staff. The output is the confirmation of users' stable emotional state and improved quality of life.

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] To implement this invention, it is first necessary to provide the elderly user with a smart device. This device has the function of continuously monitoring and collecting health data such as heart rate, blood pressure, and body temperature in real time. This data is also transmitted to a terminal via a communication means. The terminal organizes the received health data and transfers it to a server in an appropriate format.

[0237] The server runs a program that includes an AI algorithm to analyze the received data. When the server analyzes the data and evaluates and detects abnormal values ​​or health risks, it generates an alarm. For example, if a user's blood pressure rises to a dangerous level, the server immediately notifies the care staff and the user's terminal. This enables a rapid response.

[0238] Furthermore, users can safely move around the facility using a mobility assistance device with autonomous driving capabilities. Users specify their desired destination via voice input, and the terminal transmits this information to the robot, which then guides them along the optimal route to their destination.

[0239] The device also has a function to update and manage the user's schedule on a daily basis. When the user records an appointment using voice commands, the device sends it to the server and notifies the user at the set time as a reminder. In this way, the worry of forgetting important data or appointments is reduced, improving the quality of daily life.

[0240] As a concrete example, suppose a user registers the time to take their medication the following day on their device. The device sends this information to a server, which then uses that information to provide the user with a voice reminder at the designated time. This ensures that the medication is taken on time.

[0241] The system of this invention allows users to enjoy a safer and more controlled living environment, significantly improving their quality of life.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The user wears a smart device to measure health data. The device continuously measures heart rate, blood pressure, and body temperature, generating data in real time.

[0245] Step 2:

[0246] The terminal receives health data from smart devices. It then checks for data inconsistencies or omissions and records the results.

[0247] Step 3:

[0248] The device converts the verified health data into a format suitable for the database and sends it to the server. A timestamp is attached to the transmitted data.

[0249] Step 4:

[0250] The server analyzes health data received from the terminal in real time. An AI algorithm on the server processes this data and evaluates any anomalies or health risks.

[0251] Step 5:

[0252] If the server detects an anomaly, it immediately generates an alarm. The alarm includes specific anomaly parameters and is sent to the terminals of both care staff and users.

[0253] Step 6:

[0254] The user instructs the autonomous mobility assistance device on its destination. The voice instructions are registered on a terminal and then transmitted from the terminal to the robot.

[0255] Step 7:

[0256] The terminal transmits destination information to the robot. The robot uses a map of the facility to calculate the optimal route and begins moving automatically.

[0257] Step 8:

[0258] The user enters their schedule into the device using voice or touch input. The device then sends this schedule data to the server.

[0259] Step 9:

[0260] The server receives the schedule data and stores it in the database. The server generates reminders according to the set time and prepares to send them to the user's device.

[0261] Step 10:

[0262] The device receives reminders from the server and notifies the user at the specified time. This ensures that the user can reliably complete important appointments and tasks.

[0263] (Example 1)

[0264] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0265] In modern society, with its aging population, managing the health of the elderly and ensuring safe mobility are crucial issues. Traditional methods fail to adequately provide real-time monitoring of individual health information, rapid response to emergencies, and support for daily activities. Therefore, a comprehensive support system is needed to enable the elderly to live safely and comfortably.

[0266] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0267] In this invention, the server includes means for acquiring biometric data from a measuring device for detecting health information, means for converting the acquired biometric data into a data structure and transmitting it to an information processing device, and means for analyzing the received biometric data using a generative AI model and evaluating the risk of specific health conditions. This enables real-time monitoring of the user's health status, improving safety and quality of life.

[0268] A "measuring device" is a device that measures biological data and monitors the user's health status in real time.

[0269] "Biometric data" refers to information that represents the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0270] "Data structure" refers to the format or style used to convert biometric data into a format that an information processing device can recognize.

[0271] An "information processing device" is a device that has the function of receiving biological data, analyzing it, or transmitting it to other devices.

[0272] A "generative AI model" is a machine learning-based algorithm used to analyze received data and recognize patterns.

[0273] "Health risk" refers to factors that may affect a specific physical condition, based on the user's health data.

[0274] A "mobility assistance device" is a device equipped with an autonomous driving function that supports users in safely traveling to their destination.

[0275] "Voice commands" refer to commands or requests that a user inputs into a device by speaking.

[0276] A "notification" is a message sent to inform a user or a third party of pre-configured information or alerts.

[0277] This invention is a system that supports the health management and safe mobility of the elderly. The following hardware and software are required to implement this invention.

[0278] The user wears a measuring device that measures heart rate, blood pressure, body temperature, etc. This device incorporates heart rate and blood pressure sensors, allowing for real-time collection of biometric data. This data is transmitted from the measuring device to the terminal using communication methods such as Bluetooth or Wi-Fi.

[0279] The terminal formats and converts the received biological data into a standardized data structure and transmits it to the server. The server analyzes the received data using an analysis algorithm based on a generative AI model. Specifically, it detects outliers and evaluates the risks of specific health conditions. If an anomaly is detected, the server immediately generates an alert and notifies the remote care staff and family. This notification is made via email or a dedicated app.

[0280] Also, the user can utilize a mobility assistance device equipped with an autonomous driving function through voice instructions. The terminal recognizes the user's voice instructions and transmits the destination information to the mobility assistance device. The mobility assistance device calculates the optimal route within the facility and safely guides the user to the destination. Additionally, the user can register their daily schedule with the terminal via voice instructions. The registered schedule is saved on the server and notified to the user as a reminder at the designated time.

[0281] As a specific example, consider a scenario where the user registers the medication intake time for the next day with the terminal via voice. The terminal transmits this information to the server, and the server implements a process to provide the user with reminders both audibly and visually at the determined time. This helps in the user's health management and realizes a safer and more comfortable living environment.

[0282] Examples of prompt texts may include the following:

[0283] "This system monitors the user's blood pressure. Please explain how it detects abnormalities."

[0284] "Please explain the process of guiding the user to the desired location via autonomous driving."

[0285] The flow of the specific process in Example 1 will be described using FIG. 11.

[0286] Step 1:

[0287] The user wears a measuring device that detects biometric data such as heart rate, blood pressure, and body temperature. The input is the user's biological state, and the output is various biometric data. The measuring device collects this data in real time using built-in sensors and transmits it to the terminal in digital format via Bluetooth.

[0288] Step 2:

[0289] The terminal organizes the biometric data received from the measuring device and converts it into a data structure. The input is the biometric data from the measuring device, and the output is the formatted data structure. For example, the terminal converts the biometric data into JSON format, formatting it so that the server can easily analyze it. This process ensures data consistency.

[0290] Step 3:

[0291] The terminal sends the formatted data structure to the server. The input is the converted data structure, and the output is the biometric data received by the server. Using TCP / IP as the communication protocol ensures reliable data transfer.

[0292] Step 4:

[0293] The server applies a generative AI model to analyze the received biometric data. The input is biometric data sent from the terminal, and the output is the analysis result. The generative AI model helps to detect anomalies by comparing them with past data and assessing the risk of specific health conditions. For example, if the server determines that a blood pressure value exceeds the standard value, it flags it as an anomaly.

[0294] Step 5:

[0295] Based on the analysis results, the server generates an alarm if an anomaly is detected and notifies remote care staff or family members. The input is the analysis results, and the output is a notification message. The notification is sent via SMS or a dedicated app to quickly inform recipients of the situation.

[0296] Step 6:

[0297] The user specifies the destination to the mobility assistance device using voice commands. The input is the user's voice command, and the output is data as an instruction to the device. The terminal recognizes and analyzes these voice commands and transmits the destination data to the mobility assistance device.

[0298] Step 7:

[0299] Mobility assistance devices calculate the optimal route based on destination information transmitted from a terminal and guide the user safely. The input is destination information, and the output is physical movement and guidance. The device uses a recognition system installed within the facility to navigate to the destination while avoiding obstacles.

[0300] Step 8:

[0301] The user registers schedules and sets reminders on the device via voice commands. The input is voice data from the user, and the output is schedule data. The device analyzes the user's commands, sends the schedule to the server, and prepares to notify the user at the necessary time.

[0302] These processing steps enable the system to function effectively, supporting the health management and safe mobility of the elderly.

[0303] (Application Example 1)

[0304] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0305] In recent years, the importance of health management and life support for the elderly has been increasing. In particular, technologies for efficiently collecting and analyzing health data, providing mobility support, and managing schedules are in demand. However, in current systems, there is a lack of visual support that allows the elderly to obtain information in real time, so there are limitations in achieving a safe and smooth life.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0307] In this invention, the server includes means for analyzing health-related data using an AI algorithm and evaluating specific health risks, means for providing route information using a visual information display device, and means for detecting abnormalities according to the health condition and issuing an alarm. As a result, while the elderly can lead an independent life, safe and smooth daily activities become possible.

[0308] An "AI algorithm" is a computational procedure for analyzing data by artificial intelligence to detect patterns and abnormalities.

[0309] "Health-related data" is a collection of information indicating an individual's health status, such as heart rate, blood pressure, and body temperature.

[0310] "Means for evaluating health risks" is a function for determining the possibility of potential health problems based on the collected health data.

[0311] A "mobility support device with an automatic driving function" is a device having an autonomous driving function for safely transporting a user to a destination.

[0312] A "visual information display device" is a device for providing visual information to a user in real time.

[0313] "Means for detecting abnormalities and issuing an alarm" is a function for automatically generating and notifying an alarm when a preset reference value is exceeded.

[0314] "A means of managing schedules and providing reminders" refers to a function that records the user's schedule and sends notifications at specified times.

[0315] To implement this invention, first, a smart device for monitoring health data is provided to the user. The smart device measures data such as heart rate, blood pressure, and body temperature and transmits it to the terminal in real time. The terminal organizes this data and sends it to a server. The server analyzes the received health data using an AI algorithm and issues an alarm if it detects any abnormal values. The alarm is notified to the user and their family or care staff who are located remotely.

[0316] Furthermore, users can safely travel to their destination using a mobility assistance device equipped with autonomous driving capabilities. This device provides route information to a visual information display device based on the destination specified by the user via voice input, and provides real-time navigation. When a user enters their schedule by voice through an application installed on their smartphone, the device manages it as a schedule and provides reminders at the specified time.

[0317] Specifically, the server analyzes the collected data using AI algorithms such as Firebase ML Kit and TensorFlow to detect anomalies. The data is stored in databases such as Firestore and SQLite. Google Glass and other similar devices are used as visual information display devices. The notification function uses Firebase Cloud Messaging to send information to users and care staff in real time.

[0318] For example, when a user leaves home to go to a cafe, route information is visually displayed on their glasses, allowing them to travel safely and with peace of mind. If a user records the time for their medication the next day in the app, they will receive a reminder notification at the specified time, so they don't have to worry about forgetting to take their medication.

[0319] As an example of using a generative AI model, consider the following prompt message: "Analyze health monitoring data and generate a prompt that warns if the heart rate is higher than normal."

[0320] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0321] Step 1:

[0322] The user wears a smart device on their body to collect data such as heart rate, blood pressure, and body temperature. The input is health data from the sensors, and the output is the transmission of this data to the device.

[0323] Step 2:

[0324] The terminal stores the received health data in a database, standardizes the format, and then transfers it to the server. The input is raw data received from the smart device, and the output is health data in a standardized format.

[0325] Step 3:

[0326] The server analyzes received health data using an AI algorithm. Data processing includes anomaly detection, distinguishing between normal and abnormal values. The input is organized health data, and the output consists of data analysis results and anomaly detection information.

[0327] Step 4:

[0328] The server generates an alarm if it detects an anomaly based on the analysis results. The generated alarm is sent to care staff or family members located remotely. The input is anomaly detection information, and the output is the alarm notification.

[0329] Step 5:

[0330] The user specifies their desired destination by voice via their smartphone. The input is the user's voice command, and the output is destination information.

[0331] Step 6:

[0332] The terminal transmits destination information to a mobility assistance device equipped with an autonomous driving function, which calculates the optimal route and provides real-time navigation information to a visual information display device. The input is destination information, and the output is route information and navigation instructions.

[0333] Step 7:

[0334] The user registers their schedule using voice input on the device. The input is the user's voice-generated schedule information, and the output is the recorded schedule information.

[0335] Step 8:

[0336] The device stores schedule information, generates a reminder at the specified time, and notifies the user. The input is schedule information, and the output is the reminder notification.

[0337] Step 9:

[0338] The server utilizes a generative AI model to generate prompt messages related to anomaly detection in health data and schedule management. Inputs are analytical data and schedule information, while output is the prompt messages.

[0339] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0340] This invention provides a system incorporating an emotion engine to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. First, the user is provided with a terminal equipped with a voice recognition device and a camera, through which voice and facial expression data are collected. This data is analyzed by the terminal to recognize the user's emotional state in real time. The emotion engine uses algorithms to identify emotions such as joy, anxiety, anger, and sadness, and based on this, health risks and quality of life are assessed.

[0341] The terminal is responsible for transmitting emotional data collected from users to a server. The server integrates and analyzes the emotional data and health data to predict health risks if stress and anxiety persist for a long period. Based on this analysis, it generates alarms and reminders and sends them to users and care staff. It also provides a function to automatically adjust care plans and schedules accordingly if the user is in a specific emotional state.

[0342] As a concrete example, consider a scenario where a user utilizes the system: If, one afternoon, the user displays a sad expression while watching television, the terminal instantly analyzes this using an emotion engine and sends the data to the server. The server then combines this data with the user's health data for the day, and if it determines that the user may already be experiencing prolonged stress, it sends a notification to the care staff suggesting stress management activities. This process allows care staff to take swift action and support the user's mental well-being.

[0343] The system incorporating the emotion engine of this invention is a means of improving the overall quality of life of users by focusing not only on health management but also on emotional care. This system provides a form that can address the emotional challenges that users face in their daily lives.

[0344] The following describes the processing flow.

[0345] Step 1:

[0346] Users go about their daily lives using a device equipped with a voice recognition device and a camera. The device collects not only the user's voice commands but also image data of their facial expressions in real time.

[0347] Step 2:

[0348] The device analyzes the user's emotional state using an emotion engine based on collected voice and facial expression data. It identifies emotions such as joy, anxiety, anger, and sadness, and records their changes.

[0349] Step 3:

[0350] The device sends the results of its emotion analysis to the server. The transmitted information includes the type and intensity of the emotion, along with other relevant time information.

[0351] Step 4:

[0352] The server integrates the received emotional data with existing health data to assess the user's overall health status. This assessment pays particular attention to the effects of persistent stress and anxiety on the body.

[0353] Step 5:

[0354] If the server detects a severe stress level, it will generate an alert for care staff and family members and send a notification to prompt necessary intervention.

[0355] Step 6:

[0356] The server further adjusts the user's schedule based on their emotions and health status, and offers suggestions to reduce stress. This adjustment is sent to the user's device for them to review.

[0357] Step 7:

[0358] The device notifies the user of appropriate reminders and changes in care plans, helping to optimize daily activities for health and emotional well-being.

[0359] (Example 2)

[0360] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0361] In managing the health of the elderly, traditional methods often focus solely on physical health, leaving insufficient assessment of health risks associated with changes in emotional state and inadequate support for daily living. Furthermore, there is a lack of measures to prevent health problems arising from the accumulation of long-term stress and anxiety.

[0362] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0363] This invention includes a server that uses an emotion engine to analyze emotional states from voice data and facial expression data and evaluates health risks based on specific emotions; means for integrating the analyzed emotional data and health data to predict long-term stress risks; and means for automatically adjusting care plans and schedules according to the emotional data. This enables rapid response to changes in emotional states and allows for comprehensive health risk assessment and lifestyle support.

[0364] An "emotion engine" is a part of a system that includes algorithms for identifying a user's emotional state by analyzing voice data and facial expression data.

[0365] "Health risk" is an indicator that shows the possibility of future health problems or abnormalities based on the user's emotional state and health data.

[0366] "Emotional data" refers to information about a user's emotional state obtained through the analysis of voice data and facial expression data.

[0367] "Health data" refers to information that indicates a user's physical health status, and generally includes physiological indicators such as blood pressure and heart rate.

[0368] A "care plan" is a plan of support and care that is formulated based on the user's emotional state and health condition.

[0369] "Automatic schedule adjustment" is a process that automatically modifies or optimizes existing activities and appointments based on the user's emotional state.

[0370] An "alert" is a notification sent to supporters to alert them when a user's health or emotional state exceeds predetermined criteria.

[0371] A "reminder" is a notification that reminds users or supporters of scheduled events or necessary activities.

[0372] This invention utilizes a system combining an emotion engine and health data analysis functions to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. A voice recognition device and a camera-equipped terminal are used to collect user voice and facial expression data. The terminal acquires data in real time and performs analysis using a locally installed emotion engine. The emotion engine implements machine learning algorithms to identify emotions such as joy, anxiety, anger, and sadness.

[0373] The device sends the analysis results to the server. The server combines emotional data with accumulated health data to predict health risks resulting from prolonged stress and anxiety. This typically utilizes physiological data such as the user's blood pressure and heart rate. Based on the analysis results, alarms and reminders are automatically generated and sent to the user and caregivers. Furthermore, if the user's emotional state meets certain conditions, care plans and schedules are automatically adjusted.

[0374] For example, if a user displays a sad expression while watching television one afternoon, the device instantly analyzes this using its emotion engine and sends the data to the server. The server integrates this data with the user's health data for the day, and if it determines that the user may be experiencing ongoing stress, it sends a notification to a support worker suggesting stress management activities. This allows support workers to take action quickly.

[0375] An example of a prompt for a generative AI model might be, "We want to design a system that analyzes the emotions of elderly people in real time and predicts their health risks." By using this system, users can respond flexibly to emotional challenges in their daily lives and improve their overall quality of life.

[0376] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0377] Step 1:

[0378] The device collects the user's voice and facial expression data in real time using a voice recognition device and a camera. The input data consists of audio of everyday conversations and video of facial expressions, which are used to prepare for the identification of specific emotional states. The output is raw audio and video data formatted in a way that is necessary for analysis.

[0379] Step 2:

[0380] The device analyzes collected voice and facial expression data using an emotion engine. The emotion engine uses machine learning algorithms to evaluate voice intonation and facial features, identifying emotions such as joy, anxiety, anger, and sadness from the input data. Specifically, it quantifies human emotions using voice waveform analysis and image processing techniques. The output obtained at this stage is emotion data representing the user's identified emotions.

[0381] Step 3:

[0382] The terminal transmits the analyzed sentiment data to the server via the network. The input is the sentiment data identified in the previous step, and the output is the transmitted sentiment data. Metadata such as timestamps and location information are also sent during this process and used for subsequent analysis.

[0383] Step 4:

[0384] The server integrates and analyzes received emotional data with existing health data. Inputs include emotional data and physiological data from health monitoring devices. Based on this, the server performs statistical modeling to predict long-term stress and health risks. The output is a risk assessment result, which is notified to support staff or users as needed.

[0385] Step 5:

[0386] The server generates and sends alarms and reminders to users and support staff based on the results of the risk assessment. The input is the risk assessment results, and the output is alarms and reminders for management. Specifically, it sends a warning notification to the support staff's smartphone and displays a message on the user's device.

[0387] Step 6:

[0388] The server automatically adjusts care plans and schedules based on the emotional data obtained. Inputs are the individual user's emotional state and health risk assessment results. Outputs are new activity proposals and adjustment plans incorporated into the user's schedule. This process determines the optimal care policy by comparing it with the user's past behavioral history.

[0389] (Application Example 2)

[0390] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0391] In managing the health of the elderly, it is necessary to understand not only their physical health but also their emotional state in real time and provide mental support. Conventional systems make it difficult to provide comprehensive health management that takes emotional states into account, and there is a problem in that it is difficult to respond quickly and appropriately to maintain the mental health of the elderly.

[0392] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0393] This invention includes a server that uses an AI algorithm to analyze health data and assess specific health risks, an emotion recognition engine to analyze the user's emotional state in real time and assess their mental health, and a means to automatically adjust the care support plan according to the user's emotional state. This enables the integration and analysis of emotional and health data to provide appropriate care support quickly.

[0394] An "AI algorithm" is a method of analyzing data using artificial intelligence technology and mimicking human intellectual work.

[0395] "Health data" refers to various measurements and observations that indicate an individual's physical and mental state.

[0396] "Health risk" refers to potential dangers that could affect an individual's health.

[0397] An "emotion recognition engine" is a technology that analyzes voice and facial expression data to identify the user's inner emotional state.

[0398] "Real-time" refers to the temporal characteristic of being able to process and use data immediately.

[0399] "Mental health" refers to a state in which an individual's emotional and psychological state is good and well-adjusted.

[0400] A "care support plan" is a specific action plan formulated to provide appropriate care based on an individual's health and emotional state.

[0401] "Automatic adjustment" refers to the characteristic of a system that can change its settings to an optimal state on its own without external instructions or intervention.

[0402] This system consists of a device worn by the user and a server in the cloud. The device is equipped with a microphone and camera to collect the user's voice and facial expression data in real time. Based on this data, an AI algorithm analyzes the user's emotional state in an emotion recognition engine. This analysis utilizes software specifically designed for emotion recognition, such as Microsoft Azure's Face API and Google Cloud's Vision API.

[0403] The collected data is sent to a server where it is integrated with health data and analyzed. The server uses AI algorithms to identify health risks. Furthermore, it automatically adjusts care plans in response to changes in emotional state, providing appropriate support to users and care staff. This function makes it possible to prevent mental health problems such as long-term stress and feelings of isolation.

[0404] For example, if a user shows a sad expression while watching television in the living room, the device immediately sends that data to a server. The server analyzes the data and notifies care staff of the possibility of an emotional change. Based on this information, the staff can suggest appropriate conversations and activities for the user, thereby strengthening their emotional support.

[0405] An example of a prompt in a generative AI model is, "If the user is showing a sad expression, what kind of caregiving response would you suggest?" By using this prompt, the AI ​​model can come up with caregiving responses that are appropriate to the user's emotional state.

[0406] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0407] Step 1:

[0408] The device collects the user's voice and facial expressions using a microphone and camera. It acquires real-time audio and video data as input, converts it to a digital format, and prepares it for transmission to the server.

[0409] Step 2:

[0410] The server receives audio and video data transmitted from the terminal. The input data is passed to an AI algorithm for speech recognition and image analysis. This algorithm extracts features from the voice and facial expressions and performs calculations to identify the user's emotional state. As output, it generates digital data representing the emotion.

[0411] Step 3:

[0412] The server analyzes the user's health status along with health data based on emotional data identified by the emotion recognition engine. It integrates the input emotional and health data to assess health risks such as stress and anxiety. As output, it generates a digital report regarding the presence or absence of health risks.

[0413] Step 4:

[0414] The server determines the necessary actions based on the analysis results. Based on the entered health risk information, it automatically adjusts the care support plan as needed. As output, it creates an adjusted care plan to present to the care staff.

[0415] Step 5:

[0416] The server sends the adjusted care plan and necessary notifications to the care staff. Using the adjusted care plan and notification data as input, it generates prompts through a generative AI model to suggest appropriate care responses. As output, it delivers notifications to the staff's smart devices.

[0417] Step 6:

[0418] Users receive support from care staff as needed. The input is the maintenance of mental and physical health through care and activities provided by staff. The output is the confirmation of users' stable emotional state and improved quality of life.

[0419] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0420] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0421] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0422] [Third Embodiment]

[0423] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0424] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0425] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0426] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0427] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0428] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0429] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0430] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0431] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0432] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0433] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0434] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0435] To implement this invention, it is first necessary to provide the elderly user with a smart device. This device has the function of continuously monitoring and collecting health data such as heart rate, blood pressure, and body temperature in real time. This data is also transmitted to a terminal via a communication means. The terminal organizes the received health data and transfers it to a server in an appropriate format.

[0436] The server runs a program that includes an AI algorithm to analyze the received data. When the server analyzes the data and evaluates and detects abnormal values ​​or health risks, it generates an alarm. For example, if a user's blood pressure rises to a dangerous level, the server immediately notifies the care staff and the user's terminal. This enables a rapid response.

[0437] Furthermore, users can safely move around the facility using a mobility assistance device with autonomous driving capabilities. Users specify their desired destination via voice input, and the terminal transmits this information to the robot, which then guides them along the optimal route to their destination.

[0438] The device also has a function to update and manage the user's schedule on a daily basis. When the user records an appointment using voice commands, the device sends it to the server and notifies the user at the set time as a reminder. In this way, the worry of forgetting important data or appointments is reduced, improving the quality of daily life.

[0439] As a concrete example, suppose a user registers the time to take their medication the following day on their device. The device sends this information to a server, which then uses that information to provide the user with a voice reminder at the designated time. This ensures that the medication is taken on time.

[0440] The system of this invention allows users to enjoy a safer and more controlled living environment, significantly improving their quality of life.

[0441] The following describes the processing flow.

[0442] Step 1:

[0443] The user wears a smart device to measure health data. The device continuously measures heart rate, blood pressure, and body temperature, generating data in real time.

[0444] Step 2:

[0445] The terminal receives health data from smart devices. It then checks for data inconsistencies or omissions and records the results.

[0446] Step 3:

[0447] The device converts the verified health data into a format suitable for the database and sends it to the server. A timestamp is attached to the transmitted data.

[0448] Step 4:

[0449] The server analyzes health data received from the terminal in real time. An AI algorithm on the server processes this data and evaluates any anomalies or health risks.

[0450] Step 5:

[0451] If the server detects an anomaly, it immediately generates an alarm. The alarm includes specific anomaly parameters and is sent to the terminals of both care staff and users.

[0452] Step 6:

[0453] The user instructs the autonomous mobility assistance device on its destination. The voice instructions are registered on a terminal and then transmitted from the terminal to the robot.

[0454] Step 7:

[0455] The terminal transmits destination information to the robot. The robot uses a map of the facility to calculate the optimal route and begins moving automatically.

[0456] Step 8:

[0457] The user enters their schedule into the device using voice or touch input. The device then sends this schedule data to the server.

[0458] Step 9:

[0459] The server receives the schedule data and stores it in the database. The server generates reminders according to the set time and prepares to send them to the user's device.

[0460] Step 10:

[0461] The device receives reminders from the server and notifies the user at the specified time. This ensures that the user can reliably complete important appointments and tasks.

[0462] (Example 1)

[0463] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0464] In modern society, with its aging population, managing the health of the elderly and ensuring safe mobility are crucial issues. Traditional methods fail to adequately provide real-time monitoring of individual health information, rapid response to emergencies, and support for daily activities. Therefore, a comprehensive support system is needed to enable the elderly to live safely and comfortably.

[0465] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0466] In this invention, the server includes means for acquiring biometric data from a measuring device for detecting health information, means for converting the acquired biometric data into a data structure and transmitting it to an information processing device, and means for analyzing the received biometric data using a generative AI model and evaluating the risk of specific health conditions. This enables real-time monitoring of the user's health status, improving safety and quality of life.

[0467] A "measuring device" is a device that measures biological data and monitors the user's health status in real time.

[0468] "Biometric data" refers to information that represents the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0469] "Data structure" refers to the format or style used to convert biometric data into a format that an information processing device can recognize.

[0470] An "information processing device" is a device that has the function of receiving biological data, analyzing it, or transmitting it to other devices.

[0471] A "generative AI model" is a machine learning-based algorithm used to analyze received data and recognize patterns.

[0472] "Health risk" refers to factors that may affect a specific physical condition, based on the user's health data.

[0473] A "mobility assistance device" is a device equipped with an autonomous driving function that supports users in safely traveling to their destination.

[0474] "Voice commands" refer to commands or requests that a user inputs into a device by speaking.

[0475] A "notification" is a message sent to inform a user or a third party of pre-configured information or alerts.

[0476] This invention is a system that supports the health management and safe mobility of the elderly. The following hardware and software are required to implement this invention.

[0477] The user wears a measuring device that measures heart rate, blood pressure, body temperature, etc. This device incorporates heart rate and blood pressure sensors, allowing for real-time collection of biometric data. This data is transmitted from the measuring device to the terminal using communication methods such as Bluetooth or Wi-Fi.

[0478] The device converts the received biometric data into a standardized data structure and sends it to the server. The server analyzes the received data using an analysis algorithm based on a generative AI model. Specifically, it detects anomalies and assesses the risk of specific health conditions. If an anomaly is detected, the server immediately generates an alarm and notifies remote care staff or family members. This notification is sent via email or a dedicated app.

[0479] Furthermore, users can utilize mobility assistance devices equipped with autonomous driving capabilities through voice commands. The terminal recognizes the user's voice commands and transmits destination information to the mobility assistance device. The mobility assistance device calculates the optimal route within the facility and safely guides the user to their destination. In addition, users can register their daily schedules on the terminal using voice commands. Registered schedules are stored on the server and notified to the user as reminders at the specified time.

[0480] As a concrete example, consider a scenario where a user registers the time to take their medication the following day by voice on their device. The device sends this information to a server, which then performs a process of providing the user with a voice and visual reminder at the designated time. This helps users manage their health and contributes to a safer and more comfortable living environment.

[0481] Examples of prompt statements include the following:

[0482] "Please explain how this system monitors the user's blood pressure and detects abnormalities."

[0483] "Please explain the process of how the vehicle autonomously guides users to their desired location."

[0484] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0485] Step 1:

[0486] The user wears a measuring device that detects biometric data such as heart rate, blood pressure, and body temperature. The input is the user's biological state, and the output is various biometric data. The measuring device collects this data in real time using built-in sensors and transmits it to the terminal in digital format via Bluetooth.

[0487] Step 2:

[0488] The terminal organizes the biometric data received from the measuring device and converts it into a data structure. The input is the biometric data from the measuring device, and the output is the formatted data structure. For example, the terminal converts the biometric data into JSON format, formatting it so that the server can easily analyze it. This process ensures data consistency.

[0489] Step 3:

[0490] The terminal sends the formatted data structure to the server. The input is the converted data structure, and the output is the biometric data received by the server. Using TCP / IP as the communication protocol ensures reliable data transfer.

[0491] Step 4:

[0492] The server applies a generative AI model to analyze the received biometric data. The input is biometric data sent from the terminal, and the output is the analysis result. The generative AI model helps to detect anomalies by comparing them with past data and assessing the risk of specific health conditions. For example, if the server determines that a blood pressure value exceeds the standard value, it flags it as an anomaly.

[0493] Step 5:

[0494] Based on the analysis results, the server generates an alarm if an anomaly is detected and notifies remote care staff or family members. The input is the analysis results, and the output is a notification message. The notification is sent via SMS or a dedicated app to quickly inform recipients of the situation.

[0495] Step 6:

[0496] The user specifies the destination to the mobility assistance device using voice commands. The input is the user's voice command, and the output is data as an instruction to the device. The terminal recognizes and analyzes these voice commands and transmits the destination data to the mobility assistance device.

[0497] Step 7:

[0498] Mobility assistance devices calculate the optimal route based on destination information transmitted from a terminal and guide the user safely. The input is destination information, and the output is physical movement and guidance. The device uses a recognition system installed within the facility to navigate to the destination while avoiding obstacles.

[0499] Step 8:

[0500] The user registers schedules and sets reminders on the device via voice commands. The input is voice data from the user, and the output is schedule data. The device analyzes the user's commands, sends the schedule to the server, and prepares to notify the user at the necessary time.

[0501] These processing steps enable the system to function effectively, supporting the health management and safe mobility of the elderly.

[0502] (Application Example 1)

[0503] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0504] In recent years, the importance of health management and lifestyle support for the elderly has increased. In particular, there is a need for technologies that efficiently collect and analyze health data, provide mobility support, and manage schedules. However, current systems lack the visual support necessary for the elderly to access information in real time, limiting their ability to ensure a safe and smooth life for them.

[0505] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0506] In this invention, the server includes means for analyzing health-related data using an AI algorithm and evaluating specific health risks, means for providing route information using a visual information display device, and means for detecting abnormalities according to the health status and issuing alarms. This enables elderly people to live independently and engage in safe and smooth daily activities.

[0507] An "AI algorithm" is a set of computational procedures that use artificial intelligence to analyze data and detect patterns and anomalies.

[0508] "Health data" refers to a collection of information that indicates an individual's health status, such as heart rate, blood pressure, and body temperature.

[0509] A "means for assessing health risks" is a function that determines the possibility of potential health problems based on collected health data.

[0510] A "mobility support device with autonomous driving capabilities" is a device that has autonomous driving functions to safely transport users to their destination.

[0511] A "visual information display device" is a device that provides users with visual information in real time.

[0512] "Means for detecting abnormalities and issuing alarms" refers to a function that automatically generates and notifies an alarm when a preset threshold value is exceeded.

[0513] "A means of managing schedules and providing reminders" refers to a function that records the user's schedule and sends notifications at specified times.

[0514] To implement this invention, first, a smart device for monitoring health data is provided to the user. The smart device measures data such as heart rate, blood pressure, and body temperature and transmits it to the terminal in real time. The terminal organizes this data and sends it to a server. The server analyzes the received health data using an AI algorithm and issues an alarm if it detects any abnormal values. The alarm is notified to the user and their family or care staff who are located remotely.

[0515] Furthermore, users can safely travel to their destination using a mobility assistance device equipped with autonomous driving capabilities. This device provides route information to a visual information display device based on the destination specified by the user via voice input, and provides real-time navigation. When a user enters their schedule by voice through an application installed on their smartphone, the device manages it as a schedule and provides reminders at the specified time.

[0516] Specifically, the server analyzes the collected data using AI algorithms such as Firebase ML Kit and TensorFlow to detect anomalies. The data is stored in databases such as Firestore and SQLite. Google Glass and other similar devices are used as visual information display devices. The notification function uses Firebase Cloud Messaging to send information to users and care staff in real time.

[0517] For example, when a user leaves home to go to a cafe, route information is visually displayed on their glasses, allowing them to travel safely and with peace of mind. If a user records the time for their medication the next day in the app, they will receive a reminder notification at the specified time, so they don't have to worry about forgetting to take their medication.

[0518] As an example of using a generative AI model, consider the following prompt message: "Analyze health monitoring data and generate a prompt that warns if the heart rate is higher than normal."

[0519] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0520] Step 1:

[0521] The user wears a smart device on their body to collect data such as heart rate, blood pressure, and body temperature. The input is health data from the sensors, and the output is the transmission of this data to the device.

[0522] Step 2:

[0523] The terminal stores the received health data in a database, standardizes the format, and then transfers it to the server. The input is raw data received from the smart device, and the output is health data in a standardized format.

[0524] Step 3:

[0525] The server analyzes received health data using an AI algorithm. Data processing includes anomaly detection, distinguishing between normal and abnormal values. The input is organized health data, and the output consists of data analysis results and anomaly detection information.

[0526] Step 4:

[0527] The server generates an alarm if it detects an anomaly based on the analysis results. The generated alarm is sent to care staff or family members located remotely. The input is anomaly detection information, and the output is the alarm notification.

[0528] Step 5:

[0529] The user specifies their desired destination by voice via their smartphone. The input is the user's voice command, and the output is destination information.

[0530] Step 6:

[0531] The terminal transmits destination information to a mobility assistance device equipped with an autonomous driving function, which calculates the optimal route and provides real-time navigation information to a visual information display device. The input is destination information, and the output is route information and navigation instructions.

[0532] Step 7:

[0533] The user registers their schedule using voice input on the device. The input is the user's voice-generated schedule information, and the output is the recorded schedule information.

[0534] Step 8:

[0535] The device stores schedule information, generates a reminder at the specified time, and notifies the user. The input is schedule information, and the output is the reminder notification.

[0536] Step 9:

[0537] The server utilizes a generative AI model to generate prompt messages related to anomaly detection in health data and schedule management. Inputs are analytical data and schedule information, while output is the prompt messages.

[0538] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0539] This invention provides a system incorporating an emotion engine to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. First, the user is provided with a terminal equipped with a voice recognition device and a camera, through which voice and facial expression data are collected. This data is analyzed by the terminal to recognize the user's emotional state in real time. The emotion engine uses algorithms to identify emotions such as joy, anxiety, anger, and sadness, and based on this, health risks and quality of life are assessed.

[0540] The terminal is responsible for transmitting emotional data collected from users to a server. The server integrates and analyzes the emotional data and health data to predict health risks if stress and anxiety persist for a long period. Based on this analysis, it generates alarms and reminders and sends them to users and care staff. It also provides a function to automatically adjust care plans and schedules accordingly if the user is in a specific emotional state.

[0541] As a concrete example, consider a scenario where a user utilizes the system: If, one afternoon, the user displays a sad expression while watching television, the terminal instantly analyzes this using an emotion engine and sends the data to the server. The server then combines this data with the user's health data for the day, and if it determines that the user may already be experiencing prolonged stress, it sends a notification to the care staff suggesting stress management activities. This process allows care staff to take swift action and support the user's mental well-being.

[0542] The system incorporating the emotion engine of this invention is a means of improving the overall quality of life of users by focusing not only on health management but also on emotional care. This system provides a form that can address the emotional challenges that users face in their daily lives.

[0543] The following describes the processing flow.

[0544] Step 1:

[0545] Users go about their daily lives using a device equipped with a voice recognition device and a camera. The device collects not only the user's voice commands but also image data of their facial expressions in real time.

[0546] Step 2:

[0547] The device analyzes the user's emotional state using an emotion engine based on collected voice and facial expression data. It identifies emotions such as joy, anxiety, anger, and sadness, and records their changes.

[0548] Step 3:

[0549] The device sends the results of its emotion analysis to the server. The transmitted information includes the type and intensity of the emotion, along with other relevant time information.

[0550] Step 4:

[0551] The server integrates the received emotional data with existing health data to assess the user's overall health status. This assessment pays particular attention to the effects of persistent stress and anxiety on the body.

[0552] Step 5:

[0553] If the server detects a severe stress level, it will generate an alert for care staff and family members and send a notification to prompt necessary intervention.

[0554] Step 6:

[0555] The server further adjusts the user's schedule based on their emotions and health status, and offers suggestions to reduce stress. This adjustment is sent to the user's device for them to review.

[0556] Step 7:

[0557] The device notifies the user of appropriate reminders and changes in care plans, helping to optimize daily activities for health and emotional well-being.

[0558] (Example 2)

[0559] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] In managing the health of the elderly, traditional methods often focus solely on physical health, leaving insufficient assessment of health risks associated with changes in emotional state and inadequate support for daily living. Furthermore, there is a lack of measures to prevent health problems arising from the accumulation of long-term stress and anxiety.

[0561] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0562] This invention includes a server that uses an emotion engine to analyze emotional states from voice data and facial expression data and evaluates health risks based on specific emotions; means for integrating the analyzed emotional data and health data to predict long-term stress risks; and means for automatically adjusting care plans and schedules according to the emotional data. This enables rapid response to changes in emotional states and allows for comprehensive health risk assessment and lifestyle support.

[0563] An "emotion engine" is a part of a system that includes algorithms for identifying a user's emotional state by analyzing voice data and facial expression data.

[0564] "Health risk" is an indicator that shows the possibility of future health problems or abnormalities based on the user's emotional state and health data.

[0565] "Emotional data" refers to information about a user's emotional state obtained through the analysis of voice data and facial expression data.

[0566] "Health data" refers to information that indicates a user's physical health status, and generally includes physiological indicators such as blood pressure and heart rate.

[0567] A "care plan" is a plan of support and care that is formulated based on the user's emotional state and health condition.

[0568] "Automatic schedule adjustment" is a process that automatically modifies or optimizes existing activities and appointments based on the user's emotional state.

[0569] An "alert" is a notification sent to supporters to alert them when a user's health or emotional state exceeds predetermined criteria.

[0570] A "reminder" is a notification that reminds users or supporters of scheduled events or necessary activities.

[0571] This invention utilizes a system combining an emotion engine and health data analysis functions to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. A voice recognition device and a camera-equipped terminal are used to collect user voice and facial expression data. The terminal acquires data in real time and performs analysis using a locally installed emotion engine. The emotion engine implements machine learning algorithms to identify emotions such as joy, anxiety, anger, and sadness.

[0572] The device sends the analysis results to the server. The server combines emotional data with accumulated health data to predict health risks resulting from prolonged stress and anxiety. This typically utilizes physiological data such as the user's blood pressure and heart rate. Based on the analysis results, alarms and reminders are automatically generated and sent to the user and caregivers. Furthermore, if the user's emotional state meets certain conditions, care plans and schedules are automatically adjusted.

[0573] For example, if a user displays a sad expression while watching television one afternoon, the device instantly analyzes this using its emotion engine and sends the data to the server. The server integrates this data with the user's health data for the day, and if it determines that the user may be experiencing ongoing stress, it sends a notification to a support worker suggesting stress management activities. This allows support workers to take action quickly.

[0574] An example of a prompt for a generative AI model might be, "We want to design a system that analyzes the emotions of elderly people in real time and predicts their health risks." By using this system, users can respond flexibly to emotional challenges in their daily lives and improve their overall quality of life.

[0575] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0576] Step 1:

[0577] The device collects the user's voice and facial expression data in real time using a voice recognition device and a camera. The input data consists of audio of everyday conversations and video of facial expressions, which are used to prepare for the identification of specific emotional states. The output is raw audio and video data formatted in a way that is necessary for analysis.

[0578] Step 2:

[0579] The device analyzes collected voice and facial expression data using an emotion engine. The emotion engine uses machine learning algorithms to evaluate voice intonation and facial features, identifying emotions such as joy, anxiety, anger, and sadness from the input data. Specifically, it quantifies human emotions using voice waveform analysis and image processing techniques. The output obtained at this stage is emotion data representing the user's identified emotions.

[0580] Step 3:

[0581] The terminal transmits the analyzed sentiment data to the server via the network. The input is the sentiment data identified in the previous step, and the output is the transmitted sentiment data. Metadata such as timestamps and location information are also sent during this process and used for subsequent analysis.

[0582] Step 4:

[0583] The server integrates and analyzes received emotional data with existing health data. Inputs include emotional data and physiological data from health monitoring devices. Based on this, the server performs statistical modeling to predict long-term stress and health risks. The output is a risk assessment result, which is notified to support staff or users as needed.

[0584] Step 5:

[0585] The server generates and sends alarms and reminders to users and support staff based on the results of the risk assessment. The input is the risk assessment results, and the output is alarms and reminders for management. Specifically, it sends a warning notification to the support staff's smartphone and displays a message on the user's device.

[0586] Step 6:

[0587] The server automatically adjusts care plans and schedules based on the emotional data obtained. Inputs are the individual user's emotional state and health risk assessment results. Outputs are new activity proposals and adjustment plans incorporated into the user's schedule. This process determines the optimal care policy by comparing it with the user's past behavioral history.

[0588] (Application Example 2)

[0589] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0590] In managing the health of the elderly, it is necessary to understand not only their physical health but also their emotional state in real time and provide mental support. Conventional systems make it difficult to provide comprehensive health management that takes emotional states into account, and there is a problem in that it is difficult to respond quickly and appropriately to maintain the mental health of the elderly.

[0591] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0592] This invention includes a server that uses an AI algorithm to analyze health data and assess specific health risks, an emotion recognition engine to analyze the user's emotional state in real time and assess their mental health, and a means to automatically adjust the care support plan according to the user's emotional state. This enables the integration and analysis of emotional and health data to provide appropriate care support quickly.

[0593] An "AI algorithm" is a method of analyzing data using artificial intelligence technology and mimicking human intellectual work.

[0594] "Health data" refers to various measurements and observations that indicate an individual's physical and mental state.

[0595] "Health risk" refers to potential dangers that could affect an individual's health.

[0596] An "emotion recognition engine" is a technology that analyzes voice and facial expression data to identify the user's inner emotional state.

[0597] "Real-time" refers to the temporal characteristic of being able to process and use data immediately.

[0598] "Mental health" refers to a state in which an individual's emotional and psychological state is good and well-adjusted.

[0599] A "care support plan" is a specific action plan formulated to provide appropriate care based on an individual's health and emotional state.

[0600] "Automatic adjustment" refers to the characteristic of a system that can change its settings to an optimal state on its own without external instructions or intervention.

[0601] This system consists of a device worn by the user and a server in the cloud. The device is equipped with a microphone and camera to collect the user's voice and facial expression data in real time. Based on this data, an AI algorithm analyzes the user's emotional state in an emotion recognition engine. This analysis utilizes software specifically designed for emotion recognition, such as Microsoft Azure's Face API and Google Cloud's Vision API.

[0602] The collected data is sent to a server where it is integrated with health data and analyzed. The server uses AI algorithms to identify health risks. Furthermore, it automatically adjusts care plans in response to changes in emotional state, providing appropriate support to users and care staff. This function makes it possible to prevent mental health problems such as long-term stress and feelings of isolation.

[0603] For example, if a user shows a sad expression while watching television in the living room, the device immediately sends that data to a server. The server analyzes the data and notifies care staff of the possibility of an emotional change. Based on this information, the staff can suggest appropriate conversations and activities for the user, thereby strengthening their emotional support.

[0604] An example of a prompt in a generative AI model is, "If the user is showing a sad expression, what kind of caregiving response would you suggest?" By using this prompt, the AI ​​model can come up with caregiving responses that are appropriate to the user's emotional state.

[0605] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0606] Step 1:

[0607] The device collects the user's voice and facial expressions using a microphone and camera. It acquires real-time audio and video data as input, converts it to a digital format, and prepares it for transmission to the server.

[0608] Step 2:

[0609] The server receives audio and video data transmitted from the terminal. The input data is passed to an AI algorithm for speech recognition and image analysis. This algorithm extracts features from the voice and facial expressions and performs calculations to identify the user's emotional state. As output, it generates digital data representing the emotion.

[0610] Step 3:

[0611] The server analyzes the user's health status along with health data based on emotional data identified by the emotion recognition engine. It integrates the input emotional and health data to assess health risks such as stress and anxiety. As output, it generates a digital report regarding the presence or absence of health risks.

[0612] Step 4:

[0613] The server determines the necessary actions based on the analysis results. Based on the entered health risk information, it automatically adjusts the care support plan as needed. As output, it creates an adjusted care plan to present to the care staff.

[0614] Step 5:

[0615] The server sends the adjusted care plan and necessary notifications to the care staff. Using the adjusted care plan and notification data as input, it generates prompts through a generative AI model to suggest appropriate care responses. As output, it delivers notifications to the staff's smart devices.

[0616] Step 6:

[0617] Users receive support from care staff as needed. The input is the maintenance of mental and physical health through care and activities provided by staff. The output is the confirmation of users' stable emotional state and improved quality of life.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] To implement this invention, it is first necessary to provide the elderly user with a smart device. This device has the function of continuously monitoring and collecting health data such as heart rate, blood pressure, and body temperature in real time. This data is also transmitted to a terminal via a communication means. The terminal organizes the received health data and transfers it to a server in an appropriate format.

[0636] The server runs a program that includes an AI algorithm to analyze the received data. When the server analyzes the data and evaluates and detects abnormal values ​​or health risks, it generates an alarm. For example, if a user's blood pressure rises to a dangerous level, the server immediately notifies the care staff and the user's terminal. This enables a rapid response.

[0637] Furthermore, users can safely move around the facility using a mobility assistance device with autonomous driving capabilities. Users specify their desired destination via voice input, and the terminal transmits this information to the robot, which then guides them along the optimal route to their destination.

[0638] The device also has a function to update and manage the user's schedule on a daily basis. When the user records an appointment using voice commands, the device sends it to the server and notifies the user at the set time as a reminder. In this way, the worry of forgetting important data or appointments is reduced, improving the quality of daily life.

[0639] As a concrete example, suppose a user registers the time to take their medication the following day on their device. The device sends this information to a server, which then uses that information to provide the user with a voice reminder at the designated time. This ensures that the medication is taken on time.

[0640] The system of this invention allows users to enjoy a safer and more controlled living environment, significantly improving their quality of life.

[0641] The following describes the processing flow.

[0642] Step 1:

[0643] The user wears a smart device to measure health data. The device continuously measures heart rate, blood pressure, and body temperature, generating data in real time.

[0644] Step 2:

[0645] The terminal receives health data from smart devices. It then checks for data inconsistencies or omissions and records the results.

[0646] Step 3:

[0647] The device converts the verified health data into a format suitable for the database and sends it to the server. A timestamp is attached to the transmitted data.

[0648] Step 4:

[0649] The server analyzes health data received from the terminal in real time. An AI algorithm on the server processes this data and evaluates any anomalies or health risks.

[0650] Step 5:

[0651] If the server detects an anomaly, it immediately generates an alarm. The alarm includes specific anomaly parameters and is sent to the terminals of both care staff and users.

[0652] Step 6:

[0653] The user instructs the autonomous mobility assistance device on its destination. The voice instructions are registered on a terminal and then transmitted from the terminal to the robot.

[0654] Step 7:

[0655] The terminal transmits destination information to the robot. The robot uses a map of the facility to calculate the optimal route and begins moving automatically.

[0656] Step 8:

[0657] The user enters their schedule into the device using voice or touch input. The device then sends this schedule data to the server.

[0658] Step 9:

[0659] The server receives the schedule data and stores it in the database. The server generates reminders according to the set time and prepares to send them to the user's device.

[0660] Step 10:

[0661] The device receives reminders from the server and notifies the user at the specified time. This ensures that the user can reliably complete important appointments and tasks.

[0662] (Example 1)

[0663] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0664] In modern society, with its aging population, managing the health of the elderly and ensuring safe mobility are crucial issues. Traditional methods fail to adequately provide real-time monitoring of individual health information, rapid response to emergencies, and support for daily activities. Therefore, a comprehensive support system is needed to enable the elderly to live safely and comfortably.

[0665] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0666] In this invention, the server includes means for acquiring biometric data from a measuring device for detecting health information, means for converting the acquired biometric data into a data structure and transmitting it to an information processing device, and means for analyzing the received biometric data using a generative AI model and evaluating the risk of specific health conditions. This enables real-time monitoring of the user's health status, improving safety and quality of life.

[0667] A "measuring device" is a device that measures biological data and monitors the user's health status in real time.

[0668] "Biometric data" refers to information that represents the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0669] "Data structure" refers to the format or style used to convert biometric data into a format that an information processing device can recognize.

[0670] An "information processing device" is a device that has the function of receiving biological data, analyzing it, or transmitting it to other devices.

[0671] A "generative AI model" is a machine learning-based algorithm used to analyze received data and recognize patterns.

[0672] "Health risk" refers to factors that may affect a specific physical condition, based on the user's health data.

[0673] A "mobility assistance device" is a device equipped with an autonomous driving function that supports users in safely traveling to their destination.

[0674] "Voice commands" refer to commands or requests that a user inputs into a device by speaking.

[0675] A "notification" is a message sent to inform a user or a third party of pre-configured information or alerts.

[0676] This invention is a system that supports the health management and safe mobility of the elderly. The following hardware and software are required to implement this invention.

[0677] The user wears a measuring device that measures heart rate, blood pressure, body temperature, etc. This device incorporates heart rate and blood pressure sensors, allowing for real-time collection of biometric data. This data is transmitted from the measuring device to the terminal using communication methods such as Bluetooth or Wi-Fi.

[0678] The device converts the received biometric data into a standardized data structure and sends it to the server. The server analyzes the received data using an analysis algorithm based on a generative AI model. Specifically, it detects anomalies and assesses the risk of specific health conditions. If an anomaly is detected, the server immediately generates an alarm and notifies remote care staff or family members. This notification is sent via email or a dedicated app.

[0679] Furthermore, users can utilize mobility assistance devices equipped with autonomous driving capabilities through voice commands. The terminal recognizes the user's voice commands and transmits destination information to the mobility assistance device. The mobility assistance device calculates the optimal route within the facility and safely guides the user to their destination. In addition, users can register their daily schedules on the terminal using voice commands. Registered schedules are stored on the server and notified to the user as reminders at the specified time.

[0680] As a concrete example, consider a scenario where a user registers the time to take their medication the following day by voice on their device. The device sends this information to a server, which then performs a process of providing the user with a voice and visual reminder at the designated time. This helps users manage their health and contributes to a safer and more comfortable living environment.

[0681] Examples of prompt statements include the following:

[0682] "Please explain how this system monitors the user's blood pressure and detects abnormalities."

[0683] "Please explain the process of how the vehicle autonomously guides users to their desired location."

[0684] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0685] Step 1:

[0686] The user wears a measuring device that detects biometric data such as heart rate, blood pressure, and body temperature. The input is the user's biological state, and the output is various biometric data. The measuring device collects this data in real time using built-in sensors and transmits it to the terminal in digital format via Bluetooth.

[0687] Step 2:

[0688] The terminal organizes the biometric data received from the measuring device and converts it into a data structure. The input is the biometric data from the measuring device, and the output is the formatted data structure. For example, the terminal converts the biometric data into JSON format, formatting it so that the server can easily analyze it. This process ensures data consistency.

[0689] Step 3:

[0690] The terminal sends the formatted data structure to the server. The input is the converted data structure, and the output is the biometric data received by the server. Using TCP / IP as the communication protocol ensures reliable data transfer.

[0691] Step 4:

[0692] The server applies a generative AI model to analyze the received biometric data. The input is biometric data sent from the terminal, and the output is the analysis result. The generative AI model helps to detect anomalies by comparing them with past data and assessing the risk of specific health conditions. For example, if the server determines that a blood pressure value exceeds the standard value, it flags it as an anomaly.

[0693] Step 5:

[0694] Based on the analysis results, the server generates an alarm if an anomaly is detected and notifies remote care staff or family members. The input is the analysis results, and the output is a notification message. The notification is sent via SMS or a dedicated app to quickly inform recipients of the situation.

[0695] Step 6:

[0696] The user specifies the destination to the mobility assistance device using voice commands. The input is the user's voice command, and the output is data as an instruction to the device. The terminal recognizes and analyzes these voice commands and transmits the destination data to the mobility assistance device.

[0697] Step 7:

[0698] Mobility assistance devices calculate the optimal route based on destination information transmitted from a terminal and guide the user safely. The input is destination information, and the output is physical movement and guidance. The device uses a recognition system installed within the facility to navigate to the destination while avoiding obstacles.

[0699] Step 8:

[0700] The user registers schedules and sets reminders on the device via voice commands. The input is voice data from the user, and the output is schedule data. The device analyzes the user's commands, sends the schedule to the server, and prepares to notify the user at the necessary time.

[0701] These processing steps enable the system to function effectively, supporting the health management and safe mobility of the elderly.

[0702] (Application Example 1)

[0703] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0704] In recent years, the importance of health management and lifestyle support for the elderly has increased. In particular, there is a need for technologies that efficiently collect and analyze health data, provide mobility support, and manage schedules. However, current systems lack the visual support necessary for the elderly to access information in real time, limiting their ability to ensure a safe and smooth life for them.

[0705] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0706] In this invention, the server includes means for analyzing health-related data using an AI algorithm and evaluating specific health risks, means for providing route information using a visual information display device, and means for detecting abnormalities according to the health status and issuing alarms. This enables elderly people to live independently and engage in safe and smooth daily activities.

[0707] An "AI algorithm" is a set of computational procedures that use artificial intelligence to analyze data and detect patterns and anomalies.

[0708] "Health data" refers to a collection of information that indicates an individual's health status, such as heart rate, blood pressure, and body temperature.

[0709] A "means for assessing health risks" is a function that determines the possibility of potential health problems based on collected health data.

[0710] A "mobility support device with autonomous driving capabilities" is a device that has autonomous driving functions to safely transport users to their destination.

[0711] A "visual information display device" is a device that provides users with visual information in real time.

[0712] "Means for detecting abnormalities and issuing alarms" refers to a function that automatically generates and notifies an alarm when a preset threshold value is exceeded.

[0713] "A means of managing schedules and providing reminders" refers to a function that records the user's schedule and sends notifications at specified times.

[0714] To implement this invention, first, a smart device for monitoring health data is provided to the user. The smart device measures data such as heart rate, blood pressure, and body temperature and transmits it to the terminal in real time. The terminal organizes this data and sends it to a server. The server analyzes the received health data using an AI algorithm and issues an alarm if it detects any abnormal values. The alarm is notified to the user and their family or care staff who are located remotely.

[0715] Furthermore, users can safely travel to their destination using a mobility assistance device equipped with autonomous driving capabilities. This device provides route information to a visual information display device based on the destination specified by the user via voice input, and provides real-time navigation. When a user enters their schedule by voice through an application installed on their smartphone, the device manages it as a schedule and provides reminders at the specified time.

[0716] Specifically, the server analyzes the collected data using AI algorithms such as Firebase ML Kit and TensorFlow to detect anomalies. The data is stored in databases such as Firestore and SQLite. Google Glass and other similar devices are used as visual information display devices. The notification function uses Firebase Cloud Messaging to send information to users and care staff in real time.

[0717] For example, when a user leaves home to go to a cafe, route information is visually displayed on their glasses, allowing them to travel safely and with peace of mind. If a user records the time for their medication the next day in the app, they will receive a reminder notification at the specified time, so they don't have to worry about forgetting to take their medication.

[0718] As an example of using a generative AI model, consider the following prompt message: "Analyze health monitoring data and generate a prompt that warns if the heart rate is higher than normal."

[0719] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0720] Step 1:

[0721] The user wears a smart device on their body to collect data such as heart rate, blood pressure, and body temperature. The input is health data from the sensors, and the output is the transmission of this data to the device.

[0722] Step 2:

[0723] The terminal stores the received health data in a database, standardizes the format, and then transfers it to the server. The input is raw data received from the smart device, and the output is health data in a standardized format.

[0724] Step 3:

[0725] The server analyzes received health data using an AI algorithm. Data processing includes anomaly detection, distinguishing between normal and abnormal values. The input is organized health data, and the output consists of data analysis results and anomaly detection information.

[0726] Step 4:

[0727] The server generates an alarm if it detects an anomaly based on the analysis results. The generated alarm is sent to care staff or family members located remotely. The input is anomaly detection information, and the output is the alarm notification.

[0728] Step 5:

[0729] The user specifies their desired destination by voice via their smartphone. The input is the user's voice command, and the output is destination information.

[0730] Step 6:

[0731] The terminal transmits destination information to a mobility assistance device equipped with an autonomous driving function, which calculates the optimal route and provides real-time navigation information to a visual information display device. The input is destination information, and the output is route information and navigation instructions.

[0732] Step 7:

[0733] The user registers their schedule using voice input on the device. The input is the user's voice-generated schedule information, and the output is the recorded schedule information.

[0734] Step 8:

[0735] The device stores schedule information, generates a reminder at the specified time, and notifies the user. The input is schedule information, and the output is the reminder notification.

[0736] Step 9:

[0737] The server utilizes a generative AI model to generate prompt messages related to anomaly detection in health data and schedule management. Inputs are analytical data and schedule information, while output is the prompt messages.

[0738] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0739] This invention provides a system incorporating an emotion engine to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. First, the user is provided with a terminal equipped with a voice recognition device and a camera, through which voice and facial expression data are collected. This data is analyzed by the terminal to recognize the user's emotional state in real time. The emotion engine uses algorithms to identify emotions such as joy, anxiety, anger, and sadness, and based on this, health risks and quality of life are assessed.

[0740] The terminal is responsible for transmitting emotional data collected from users to a server. The server integrates and analyzes the emotional data and health data to predict health risks if stress and anxiety persist for a long period. Based on this analysis, it generates alarms and reminders and sends them to users and care staff. It also provides a function to automatically adjust care plans and schedules accordingly if the user is in a specific emotional state.

[0741] As a concrete example, consider a scenario where a user utilizes the system: If, one afternoon, the user displays a sad expression while watching television, the terminal instantly analyzes this using an emotion engine and sends the data to the server. The server then combines this data with the user's health data for the day, and if it determines that the user may already be experiencing prolonged stress, it sends a notification to the care staff suggesting stress management activities. This process allows care staff to take swift action and support the user's mental well-being.

[0742] The system incorporating the emotion engine of this invention is a means of improving the overall quality of life of users by focusing not only on health management but also on emotional care. This system provides a form that can address the emotional challenges that users face in their daily lives.

[0743] The following describes the processing flow.

[0744] Step 1:

[0745] Users go about their daily lives using a device equipped with a voice recognition device and a camera. The device collects not only the user's voice commands but also image data of their facial expressions in real time.

[0746] Step 2:

[0747] The device analyzes the user's emotional state using an emotion engine based on collected voice and facial expression data. It identifies emotions such as joy, anxiety, anger, and sadness, and records their changes.

[0748] Step 3:

[0749] The device sends the results of its emotion analysis to the server. The transmitted information includes the type and intensity of the emotion, along with other relevant time information.

[0750] Step 4:

[0751] The server integrates the received emotional data with existing health data to assess the user's overall health status. This assessment pays particular attention to the effects of persistent stress and anxiety on the body.

[0752] Step 5:

[0753] If the server detects a severe stress level, it will generate an alert for care staff and family members and send a notification to prompt necessary intervention.

[0754] Step 6:

[0755] The server further adjusts the user's schedule based on their emotions and health status, and offers suggestions to reduce stress. This adjustment is sent to the user's device for them to review.

[0756] Step 7:

[0757] The device notifies the user of appropriate reminders and changes in care plans, helping to optimize daily activities for health and emotional well-being.

[0758] (Example 2)

[0759] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0760] In managing the health of the elderly, traditional methods often focus solely on physical health, leaving insufficient assessment of health risks associated with changes in emotional state and inadequate support for daily living. Furthermore, there is a lack of measures to prevent health problems arising from the accumulation of long-term stress and anxiety.

[0761] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0762] This invention includes a server that uses an emotion engine to analyze emotional states from voice data and facial expression data and evaluates health risks based on specific emotions; means for integrating the analyzed emotional data and health data to predict long-term stress risks; and means for automatically adjusting care plans and schedules according to the emotional data. This enables rapid response to changes in emotional states and allows for comprehensive health risk assessment and lifestyle support.

[0763] An "emotion engine" is a part of a system that includes algorithms for identifying a user's emotional state by analyzing voice data and facial expression data.

[0764] "Health risk" is an indicator that shows the possibility of future health problems or abnormalities based on the user's emotional state and health data.

[0765] "Emotional data" refers to information about a user's emotional state obtained through the analysis of voice data and facial expression data.

[0766] "Health data" refers to information that indicates a user's physical health status, and generally includes physiological indicators such as blood pressure and heart rate.

[0767] A "care plan" is a plan of support and care that is formulated based on the user's emotional state and health condition.

[0768] "Automatic schedule adjustment" is a process that automatically modifies or optimizes existing activities and appointments based on the user's emotional state.

[0769] An "alert" is a notification sent to supporters to alert them when a user's health or emotional state exceeds predetermined criteria.

[0770] A "reminder" is a notification that reminds users or supporters of scheduled events or necessary activities.

[0771] This invention utilizes a system combining an emotion engine and health data analysis functions to provide comprehensive health management and lifestyle support that takes into account the emotional state of elderly individuals. A voice recognition device and a camera-equipped terminal are used to collect user voice and facial expression data. The terminal acquires data in real time and performs analysis using a locally installed emotion engine. The emotion engine implements machine learning algorithms to identify emotions such as joy, anxiety, anger, and sadness.

[0772] The device sends the analysis results to the server. The server combines emotional data with accumulated health data to predict health risks resulting from prolonged stress and anxiety. This typically utilizes physiological data such as the user's blood pressure and heart rate. Based on the analysis results, alarms and reminders are automatically generated and sent to the user and caregivers. Furthermore, if the user's emotional state meets certain conditions, care plans and schedules are automatically adjusted.

[0773] For example, if a user displays a sad expression while watching television one afternoon, the device instantly analyzes this using its emotion engine and sends the data to the server. The server integrates this data with the user's health data for the day, and if it determines that the user may be experiencing ongoing stress, it sends a notification to a support worker suggesting stress management activities. This allows support workers to take action quickly.

[0774] An example of a prompt for a generative AI model might be, "We want to design a system that analyzes the emotions of elderly people in real time and predicts their health risks." By using this system, users can respond flexibly to emotional challenges in their daily lives and improve their overall quality of life.

[0775] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0776] Step 1:

[0777] The device collects the user's voice and facial expression data in real time using a voice recognition device and a camera. The input data consists of audio of everyday conversations and video of facial expressions, which are used to prepare for the identification of specific emotional states. The output is raw audio and video data formatted in a way that is necessary for analysis.

[0778] Step 2:

[0779] The device analyzes collected voice and facial expression data using an emotion engine. The emotion engine uses machine learning algorithms to evaluate voice intonation and facial features, identifying emotions such as joy, anxiety, anger, and sadness from the input data. Specifically, it quantifies human emotions using voice waveform analysis and image processing techniques. The output obtained at this stage is emotion data representing the user's identified emotions.

[0780] Step 3:

[0781] The terminal transmits the analyzed sentiment data to the server via the network. The input is the sentiment data identified in the previous step, and the output is the transmitted sentiment data. Metadata such as timestamps and location information are also sent during this process and used for subsequent analysis.

[0782] Step 4:

[0783] The server integrates and analyzes received emotional data with existing health data. Inputs include emotional data and physiological data from health monitoring devices. Based on this, the server performs statistical modeling to predict long-term stress and health risks. The output is a risk assessment result, which is notified to support staff or users as needed.

[0784] Step 5:

[0785] The server generates and sends alarms and reminders to users and support staff based on the results of the risk assessment. The input is the risk assessment results, and the output is alarms and reminders for management. Specifically, it sends a warning notification to the support staff's smartphone and displays a message on the user's device.

[0786] Step 6:

[0787] The server automatically adjusts care plans and schedules based on the emotional data obtained. Inputs are the individual user's emotional state and health risk assessment results. Outputs are new activity proposals and adjustment plans incorporated into the user's schedule. This process determines the optimal care policy by comparing it with the user's past behavioral history.

[0788] (Application Example 2)

[0789] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0790] In managing the health of the elderly, it is necessary to understand not only their physical health but also their emotional state in real time and provide mental support. Conventional systems make it difficult to provide comprehensive health management that takes emotional states into account, and there is a problem in that it is difficult to respond quickly and appropriately to maintain the mental health of the elderly.

[0791] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0792] This invention includes a server that uses an AI algorithm to analyze health data and assess specific health risks, an emotion recognition engine to analyze the user's emotional state in real time and assess their mental health, and a means to automatically adjust the care support plan according to the user's emotional state. This enables the integration and analysis of emotional and health data to provide appropriate care support quickly.

[0793] An "AI algorithm" is a method of analyzing data using artificial intelligence technology and mimicking human intellectual work.

[0794] "Health data" refers to various measurements and observations that indicate an individual's physical and mental state.

[0795] "Health risk" refers to potential dangers that could affect an individual's health.

[0796] An "emotion recognition engine" is a technology that analyzes voice and facial expression data to identify the user's inner emotional state.

[0797] "Real-time" refers to the temporal characteristic of being able to process and use data immediately.

[0798] "Mental health" refers to a state in which an individual's emotional and psychological state is good and well-adjusted.

[0799] A "care support plan" is a specific action plan formulated to provide appropriate care based on an individual's health and emotional state.

[0800] "Automatic adjustment" refers to the characteristic of a system that can change its settings to an optimal state on its own without external instructions or intervention.

[0801] This system consists of a device worn by the user and a server in the cloud. The device is equipped with a microphone and camera to collect the user's voice and facial expression data in real time. Based on this data, an AI algorithm analyzes the user's emotional state in an emotion recognition engine. This analysis utilizes software specifically designed for emotion recognition, such as Microsoft Azure's Face API and Google Cloud's Vision API.

[0802] The collected data is sent to a server where it is integrated with health data and analyzed. The server uses AI algorithms to identify health risks. Furthermore, it automatically adjusts care plans in response to changes in emotional state, providing appropriate support to users and care staff. This function makes it possible to prevent mental health problems such as long-term stress and feelings of isolation.

[0803] For example, if a user shows a sad expression while watching television in the living room, the device immediately sends that data to a server. The server analyzes the data and notifies care staff of the possibility of an emotional change. Based on this information, the staff can suggest appropriate conversations and activities for the user, thereby strengthening their emotional support.

[0804] An example of a prompt in a generative AI model is, "If the user is showing a sad expression, what kind of caregiving response would you suggest?" By using this prompt, the AI ​​model can come up with caregiving responses that are appropriate to the user's emotional state.

[0805] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0806] Step 1:

[0807] The device collects the user's voice and facial expressions using a microphone and camera. It acquires real-time audio and video data as input, converts it to a digital format, and prepares it for transmission to the server.

[0808] Step 2:

[0809] The server receives audio and video data transmitted from the terminal. The input data is passed to an AI algorithm for speech recognition and image analysis. This algorithm extracts features from the voice and facial expressions and performs calculations to identify the user's emotional state. As output, it generates digital data representing the emotion.

[0810] Step 3:

[0811] The server analyzes the user's health status along with health data based on emotional data identified by the emotion recognition engine. It integrates the input emotional and health data to assess health risks such as stress and anxiety. As output, it generates a digital report regarding the presence or absence of health risks.

[0812] Step 4:

[0813] The server determines the necessary actions based on the analysis results. Based on the entered health risk information, it automatically adjusts the care support plan as needed. As output, it creates an adjusted care plan to present to the care staff.

[0814] Step 5:

[0815] The server sends the adjusted care plan and necessary notifications to the care staff. Using the adjusted care plan and notification data as input, it generates prompts through a generative AI model to suggest appropriate care responses. As output, it delivers notifications to the staff's smart devices.

[0816] Step 6:

[0817] Users receive support from care staff as needed. The input is the maintenance of mental and physical health through care and activities provided by staff. The output is the confirmation of users' stable emotional state and improved quality of life.

[0818] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0819] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0820] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0821] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0822] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0823] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0824] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0825] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0826] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0827] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0828] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0829] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0830] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0831] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0832] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0833] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0834] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0835] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0836] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0837] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0838] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0839] The following is further disclosed regarding the embodiments described above.

[0840] (Claim 1)

[0841] A means of analyzing health data using AI algorithms and evaluating specific health risks,

[0842] A means of safely transporting users to their destination using a mobility assistance device equipped with an autonomous driving function,

[0843] A means of detecting abnormalities and issuing alarms according to the health status,

[0844] A means of managing schedules and providing reminders based on user instructions,

[0845] A system that includes this.

[0846] (Claim 2)

[0847] The system according to claim 1, which collects health data 24 hours a day and provides optimized feedback to the user.

[0848] (Claim 3)

[0849] The system according to claim 1, which automatically notifies family members or care staff in a remote location based on the detection of an anomaly.

[0850] "Example 1"

[0851] (Claim 1)

[0852] A means of acquiring biometric data from a measuring device for detecting health information,

[0853] A means for converting acquired biometric data into a data structure and transmitting it to an information processing device,

[0854] A means of analyzing received biometric data using a generative AI model to assess the risk of a specific health condition,

[0855] A means of detecting an anomaly, generating an alarm indicating an emergency situation, and notifying a third party remotely,

[0856] A means of safely transporting users using mobility assistance devices equipped with autonomous driving functions,

[0857] A means of managing schedules and generating notifications based on voice instructions from the user,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, which continuously collects health data and provides individually tailored feedback to the user.

[0861] (Claim 3)

[0862] The system according to claim 1, which automatically notifies a remotely located supporter based on the detection of an anomaly.

[0863] "Application Example 1"

[0864] (Claim 1)

[0865] A means of analyzing health data using AI algorithms and evaluating specific health risks,

[0866] A means of safely transporting users to their destination using a mobility assistance device equipped with an autonomous driving function,

[0867] A means of detecting abnormalities and issuing alarms according to the health status,

[0868] A means of managing schedules and providing reminders based on user instructions,

[0869] Means for providing route information using a visual information display device,

[0870] A system that includes this.

[0871] (Claim 2)

[0872] The system according to claim 1, which collects health data 24 hours a day, provides optimized feedback to the user, and notifies the user based on abnormal values.

[0873] (Claim 3)

[0874] The system according to claim 1, which automatically notifies family members or care staff in a remote location based on anomaly detection and provides visual guidance information while the person is in transit.

[0875] "Example 2 of combining an emotion engine"

[0876] (Claim 1)

[0877] A means of analyzing emotional states from voice data and facial expression data using an emotion engine, and evaluating health risks based on specific emotions,

[0878] A means of integrating analyzed emotional data and health data to predict long-term stress risk,

[0879] A means to automatically adjust care plans and schedules based on emotional data,

[0880] A means of providing alerts and reminders to users and supporters,

[0881] A system that includes this.

[0882] (Claim 2)

[0883] The system according to claim 1, which collects emotional and health data 24 hours a day and provides optimized feedback to the user.

[0884] (Claim 3)

[0885] The system according to claim 1, which automatically notifies a support worker in a remote location based on anomaly detection using emotional data.

[0886] "Application example 2 when combining with an emotional engine"

[0887] (Claim 1)

[0888] A means of analyzing health data using AI algorithms and evaluating specific health risks,

[0889] A means of evaluating mental health by analyzing a user's emotional state in real time using an emotion recognition engine,

[0890] A means of automatically adjusting the care support plan according to the emotional state of the user,

[0891] A means of detecting abnormalities and issuing alarms according to the health status,

[0892] A means of managing schedules and providing reminders based on user instructions,

[0893] A system that includes this.

[0894] (Claim 2)

[0895] The system according to claim 1, which collects emotional data 24 hours a day and provides optimized feedback to the user.

[0896] (Claim 3)

[0897] The system according to claim 1, which automatically notifies family members or care staff in a remote location based on emotion analysis. [Explanation of symbols]

[0898] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of analyzing health-related data using AI algorithms and evaluating specific health risks, A means of safely transporting users to their destination using a mobility assistance device equipped with an autonomous driving function, A means of detecting abnormalities and issuing alarms according to the health status, A means of managing schedules and providing reminders based on user instructions, A system that includes this.

2. The system according to claim 1, which collects health data 24 hours a day and provides optimized feedback to the user.

3. The system according to claim 1, which automatically notifies family members or care staff in a remote location based on the detection of an anomaly.

Images