system

Abstract

Provide a system. [Solution] a sensor means for collecting biometric information of a user; a terminal means for transmitting the biological information collected by the sensor means to a server; an analysis means for analyzing the biological information and determining whether or not there is an abnormality; a notification means for sending a notification to the user and designated parties when the analysis means determines an abnormality; A support tool providing means for providing physical therapy and health guidelines to users; a reminder means for sending reminders based on the user's schedule; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For the elderly and people with physical disabilities, living at home is often unsafe. This problem stems from the difficulty of taking appropriate action on their own, especially in emergencies. Furthermore, receiving regular health care and medical support requires a great deal of effort. Existing systems lack mechanisms for monitoring users' health status in real time and prompting appropriate action when abnormalities are detected. Furthermore, they lack the provision of reminders and support tools to improve quality of life. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that provides a user with an environment in which they can live safely at home. This system includes a sensor means for collecting a user's biometric information, a terminal means for transmitting the collected biometric information to a server, an analysis means for performing analytics, a notification means for notifying users when an abnormality is detected, a support tool provision means for providing physical therapy and health guidelines, and a reminder means based on the user's schedule. This allows the system to monitor the user's health status in real time and promptly notify users when an abnormality occurs, encouraging them to take appropriate action. Furthermore, by providing support tools and reminders for daily health management and improving quality of life, the system allows users to continue living at home with peace of mind.

[0006] "Sensor means" refers to a device or group of devices that collect biometric information from a user.

[0007] The term "terminal means" refers to a device or group of devices that have a communication function for transmitting the biometric information collected by the sensor means to a server.

[0008] "Analysis means" refers to a device or software module that analyzes the biometric information sent to the server and determines whether there are any abnormalities.

[0009] The "notification means" refers to a device or software module that sends a notification to the user and designated parties when the analysis means determines that an abnormality has occurred.

[0010] "Support tool providing means" refers to a device or software module that provides support tools such as physical therapy and health guidelines to users.

[0011] "Reminder means" refers to a device or software module that sends notifications based on a user's schedule. [Brief explanation of the drawings]

[0012] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] This invention is a system for providing a safe and secure living environment for users at home. This system monitors the user's health status in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[0034] System programs and their processing

[0035] 1. Health monitoring

[0036] User: The user wears a smartwatch and health monitoring sensors on a daily basis, which collect biometric information such as heart rate, blood pressure, and body temperature in real time.

[0037] Device: The device periodically transmits the collected biometric information to the server. This device includes any communication-enabled device that the user normally uses, such as a smartphone or tablet.

[0038] Server: The server stores the received biometric information in a database and analyzes it using an analysis program, which includes algorithms that determine whether the heart rate and blood pressure are within normal ranges.

[0039] 2. Notification System

[0040] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties in the event of an emergency or high risk.

[0041] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[0042] User: The user checks the alert displayed on the device and contacts a doctor if necessary.

[0043] 3. Providing support tools

[0044] User: When a user needs to manage their daily health, they operate the device and request a support tool, such as watching a physical therapy video.

[0045] Terminal: The terminal starts the support tool providing program based on the user's request and displays the required content.

[0046] Server: The server retrieves the corresponding support tools (e.g., breathing exercise videos and exercise guidelines) from the database and sends them to the terminal.

[0047] User: Users can access support content at their own pace and practice physical therapy and health guidelines.

[0048] 4. Reminders

[0049] User: The user sets up a schedule for taking medicines and exercising on the device, enabling regular health management.

[0050] Server: The server runs a program that manages the user's schedule and generates reminders.

[0051] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[0052] User: The user follows the reminders to take their medicine and manage their health.

[0053] Specific examples

[0054] For example, assume that User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high, the procedure is as follows:

[0055] 1. User: The smartwatch detects an abnormal heart rate.

[0056] 2. Device: Sends collected heart rate data to the server.

[0057] 3. Server: Analyzes the data and detects anomalies.

[0058] 4. Server: Sends notifications about abnormal heart rate to User B and his family.

[0059] 5. Device: An alert is displayed on User B's smartphone and an email is sent to family members.

[0060] 6. User: Checks the alert and contacts the doctor.

[0061] 7. Support tool provided: The device will play a breathing exercise video to stabilize your heart rate.

[0062] 8. Reminder: User B sets the time to take the medicine, and the device notifies him / her at the specified time.

[0063] This system provides continuous monitoring, prompt notifications, and appropriate support tools to help users create an environment where they can live safely at home.

[0064] The processing flow will be explained below.

[0065] 1. Health monitoring

[0066] Step 1:

[0067] The user wears a smartwatch or health management sensor, which automatically collects biometric information such as heart rate, blood pressure, and body temperature in real time.

[0068] Step 2:

[0069] The device receives the collected biometric information on the user's smartphone or tablet and temporarily stores it.

[0070] Step 3:

[0071] The terminal transmits biometric information to the server at regular intervals.

[0072] Step 4:

[0073] The server stores the received biometric information in a database.

[0074] Step 5:

[0075] The server analyzes the biometric information and runs an analysis program to determine whether there are any abnormal values ​​for heart rate or blood pressure.

[0076] 2. Notification System

[0077] Step 6:

[0078] If the server detects an abnormality based on the analysis results, it generates a message to notify the user and designated parties.

[0079] Step 7:

[0080] The server sends the generated notification message to the user's terminal and the designated parties.

[0081] Step 8:

[0082] Based on the notification message received by the device, an alert is displayed on the user's smartphone or tablet.

[0083] Step 9:

[0084] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[0085] Step 10:

[0086] The user checks the alert on the device and contacts a doctor if necessary.

[0087] 3. Providing support tools

[0088] Step 11:

[0089] When a user needs to manage their daily health, they can operate the device to request a support tool, such as watching a video on breathing exercises.

[0090] Step 12:

[0091] The terminal sends the user's request to the server.

[0092] Step 13:

[0093] The server retrieves the corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database.

[0094] Step 14:

[0095] The server transmits the acquired support tool to the terminal.

[0096] Step 15:

[0097] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[0098] Step 16:

[0099] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[0100] 4. Reminders

[0101] Step 17:

[0102] The user operates the device to set medication times and exercise schedules.

[0103] Step 18:

[0104] The server saves the configured schedule in the database.

[0105] Step 19:

[0106] The server launches a program to generate reminders based on the saved schedule.

[0107] Step 20:

[0108] The server sends a reminder notification to the device at the scheduled time.

[0109] Step 21:

[0110] The device receives the reminder notification, displays a notification on the user's smartphone or tablet, and sounds an alarm.

[0111] Step 22:

[0112] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[0113] In this way, the system monitors the user's health at each step and provides appropriate notifications and support tools, creating an environment where the user can live safely at home.

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] In modern society, home health management is crucial, especially for the elderly and those with chronic illnesses. However, achieving this requires complex health management systems that monitor vital signs in real time and respond quickly to any abnormalities. It also requires a means to support daily health management and provide appropriate advice and reminders. Conventional systems struggle to meet these requirements and often only provide partial solutions. Therefore, there is an urgent need to develop more sophisticated and comprehensive health management systems.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes an input means for collecting biometric information of a user, a communication means for transmitting the biometric information collected by the input means to a processing system, an analysis means for analyzing the biometric information to determine whether an abnormality is present, a notification means for sending a notification to the user and designated related parties if the analysis means determines an abnormality, a support means for providing the user with health management and health guidelines, a schedule support means for sending reminders based on the user's schedule, a message generation means for the notification means to send notifications to the user and related parties including specific abnormalities and recommended actions, and a reminder generation means for the schedule support means to generate periodic reminders based on a time set by the user. This enables comprehensive real-time health management, prompt notification of abnormalities, and support for improving quality of life.

[0119] A "user" is an individual who uses the system to manage their health.

[0120] "Biometric information" is data relating to the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0121] "Input means" refers to a device such as a sensor that collects biometric information.

[0122] A "communication means" is a device or method that transmits biometric information collected by an input means to a processing system.

[0123] "Analysis means" refers to algorithms or software that analyze biometric information and determine whether or not there are any abnormalities.

[0124] The "notification means" is a function that sends a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[0125] "Support means" refers to services and tools that provide users with health management and health guidelines.

[0126] The "schedule support means" is a function that sends reminders based on the user's schedule.

[0127] The "message generating means" is an algorithm or software that generates a message including the abnormality details and recommended actions when the notification means sends a notification.

[0128] The "reminder generation means" is a function that generates periodic reminders based on a time set by the user.

[0129] "Processing system" refers to the entire system for receiving, managing, and analyzing biometric information and providing notifications and support based on the results.

[0130] This invention is a health management system that provides a safe and secure environment for users at home. This system monitors the user's biological information in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[0131] Health Status Monitoring

[0132] User: The user wears a smartwatch or health monitoring sensor to measure heart rate, blood pressure, and temperature. The sensor collects real-time biometric information on a daily basis.

[0133] Device: The device periodically transmits the biometric information acquired from the smartwatch to the server. This device is the user's smartphone or tablet. The smartphone or tablet transmits data to the server using Bluetooth, Wi-Fi, or mobile data communication.

[0134] Server: The server stores the received biometric data in a database and analyzes the data using analytical programs. For example, algorithms are used to determine whether heart rate or blood pressure is within normal ranges. The algorithms generate warnings if abnormal values ​​are detected.

[0135] Notification System

[0136] Server: If the server detects an abnormality based on the analysis results, it immediately generates a notification. The notification will include the specific abnormality and a recommended action, such as "Your heart rate is out of the normal range. Contact your doctor immediately."

[0137] Device: The device receives notifications from the server and displays an alert on the user's smartphone or tablet. At the same time, the server also sends notifications to relevant parties (e.g., family members or doctors) via email or SMS.

[0138] User: The user checks the alert displayed on the device and contacts a doctor if necessary. Notifications that can be viewed by the user allow for prompt and appropriate action to be taken.

[0139] Support tools provided

[0140] User: When a user needs health management, they operate their device to request support tools. For example, they may want to view physical therapy videos or breathing exercise guides.

[0141] Terminal: The terminal receives a user request, launches the support tool providing program, retrieves the necessary content from the server, and displays it.

[0142] Server: The server retrieves appropriate support tools (such as breathing exercise videos or exercise guidelines) from the database and sends them to the device.

[0143] Users: Users can use support content to practice physical therapy and health guidelines.

[0144] Reminders

[0145] User: The user sets the time to take medicine and exercise schedule on the device. For example, the user sets the time to take medicine at 9:00 every day using a dedicated app.

[0146] Server: The server manages the user's schedule and generates reminders at the specified time.

[0147] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[0148] User: Users can take their medicines according to the reminders and maintain a regular health regime.

[0149] Examples of prompt statements

[0150] Describe how the following system works: A user wears a smartwatch to monitor their heart rate. If the heart rate reaches an abnormal level, the server generates a notification, displays an alert on the user's smartphone, and provides a video with appropriate breathing exercises.

[0151] Through these functions, the system provides a safe and secure environment for users to live at home. It enables comprehensive and real-time health management, enables rapid response in the event of an abnormality, and provides tools to support daily health management.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Specific explanation of processing steps

[0154] Step 1: Collecting biometric information

[0155] User: The user wears a smartwatch or health monitoring sensors that measure heart rate, blood pressure, and body temperature. The input is real-time biometric information from these sensors. The output is the acquired biometric data: heart rate, blood pressure, and body temperature.

[0156] How it works: When a user wakes up in the morning and puts on the smartwatch, the sensors start collecting data on their heart rate and body temperature from that moment on.

[0157] Step 2: Sending biometric information

[0158] Terminal: The terminal periodically transmits biometric information acquired from the smartwatch to the server. The input is biometric information acquired from the smartwatch via Bluetooth connection. The output is biometric information transmitted to the server via Wi-Fi or mobile data communication.

[0159] How it works: The smartwatch connects to the smartphone via Bluetooth, and the smartphone sends data to the server using Wi-Fi.

[0160] Step 3: Analysis of biological information

[0161] Server: The server stores the received biometric information in a database and analyzes the data using an analysis program. The input is the biometric information received from the device. The output is the analysis result, judging whether it is within the normal range or abnormal.

[0162] Specific operation: The server analyzes the received heart rate data, calculates the average and standard deviation, and generates an alert if an abnormal value is detected.

[0163] Step 4: Generate a notification message

[0164] Server: The server immediately generates a notification message if an anomaly is detected based on the analysis results. The input is the anomaly judgment of the analysis results. The output is a notification message containing the specific anomaly content and recommended actions.

[0165] What happens: The server generates the message "Your heart rate is outside the normal range. Contact your doctor immediately."

[0166] Step 5: Sending notifications

[0167] Server: The server generates and sends notification messages to the user and the designated parties. The input is the generated notification message. The output is the notification message sent to the user's smartphone and the designated parties.

[0168] Specific operation: The server sends a notification to the user's smartphone and simultaneously notifies the user's family and doctor by email.

[0169] Step 6: Receive and confirm notifications

[0170] Terminal: The terminal receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification message sent from the server. The output is the alert displayed on the user's smartphone.

[0171] User: The user checks the alert displayed on the device and contacts a doctor if necessary. The input is the alert message displayed on the smartphone. The output is the user's response (e.g., contacting a doctor).

[0172] Specific operation: The user sees the "abnormal heart rate" notification on their smartphone screen and then uses the app to contact their doctor.

[0173] Step 7: Request Support Tools

[0174] User: The user requests the necessary support tools through the terminal. The input is an operation to request a physical therapy video or a breathing exercise guide. The output is the information sent in response to the request.

[0175] Specific operation: The user selects "Breathing exercise video" from the menu of the dedicated app.

[0176] Step 8: Providing support tools

[0177] Terminal: The terminal sends the user's request to the server and starts the support tool providing program. The input is the user's request information. The output is the start of the support tool providing program and the transmission of the request to the server.

[0178] Server: The server retrieves the corresponding support tool (such as a breathing exercise video or exercise guidelines) from the database and sends it to the device. The input is the user's request information. The output is the data of the support tool that was sent.

[0179] Specific operation: The server sends a link to a breathing exercise video to the device, and the device plays the video.

[0180] Step 9: Set reminders

[0181] User: The user sets up schedules such as medication times and exercise schedules on the device. The input is the schedule setting information in the dedicated app. The output is the set schedule information.

[0182] Specific operation: The user uses the calendar function of the dedicated app to schedule taking the medicine at 9:00 every day.

[0183] Step 10: Generate reminders

[0184] Server: The server manages the user's schedule and generates reminders at the specified time. The input is the user's schedule information. The output is the generated reminder notification.

[0185] What happens: The server generates a reminder notification every day at 9:00 AM based on the user's settings.

[0186] Step 11: Reminder Notifications

[0187] Terminal: The terminal receives reminders from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server. The output is the reminder displayed on the terminal.

[0188] User: The user takes medicine according to the reminder and manages their health. The input is the reminder notification displayed on the device. The output is the user's coping behavior (e.g., taking medicine).

[0189] Specific operation: At 9 o'clock, the user's smartphone will sound an alarm and notify them that it's time for their medicine, and they will then take their medicine.

[0190] (Application example 1)

[0191] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] For elderly people and those with chronic illnesses, constant monitoring of their health status and prompt response when an abnormality occurs are extremely important for them to continue living safely at home. However, conventional systems lack the functionality to monitor users' health status in real time or an immediate notification system in case of an emergency, making it difficult to fully ensure user safety. Furthermore, the provision of support tools for daily health management and improving quality of life is also insufficient.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0194] In this invention, the server includes a signal generating means for generating an output signal when an abnormality is detected based on the user's biological information, a support tool providing means, and a schedule management means. This makes it possible to monitor the user's health condition in real time and quickly detect and notify abnormalities. It also provides various functions to support daily health management and can support the user in improving their lifestyle habits through reminders.

[0195] The "sensor means" is a device for collecting biometric information of a user.

[0196] The "terminal means" is a communication device for transmitting the biometric information collected by the sensor means to the server.

[0197] The "analysis means" is a system that analyzes the biometric information sent to the server and determines whether or not there is an abnormality.

[0198] The "notification means" is a function for sending a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[0199] The "support tool providing means" is a function for providing physical therapy and physical guidelines to the user.

[0200] The "schedule management means" is a function for sending reminders based on the user's schedule.

[0201] The "signal generating means" is a device or system that generates an output signal when an abnormality is detected based on the user's biological information.

[0202] "Electronic communication" means a means of communication to send notices to designated parties via email or SMS.

[0203] An "alert" is a warning message that is displayed on the user's terminal when an abnormality occurs.

[0204] 1. System Configuration

[0205] The present invention provides a system that monitors the user's health condition in real time and notifies the user promptly if an abnormality occurs, thereby providing an environment in which the user can live safely at home. Specific embodiments will be described below.

[0206] 2. Hardware and Software Used

[0207] Hardware:

[0208] Smartphone (iOS / ANDROID (registered trademark))

[0209] Smartwatches (Apple Watch, Fitbit, etc.)

[0210] Health management sensors (heart rate sensors, blood pressure monitors, etc.)

[0211] software:

[0212] Mobile application (cross-platform compatible using Flutter (registered trademark))

[0213] Server (using Amazon Web Services)

[0214] Database (using Amazon RDS)

[0215] Notification system (Firebase Cloud Messaging)

[0216] 3. Operation of various means

[0217] Sensor means

[0218] Users wear smartwatches and health monitoring sensors that collect real-time biometric information such as heart rate, blood pressure, and body temperature.

[0219] Terminal means

[0220] Smartphones and tablets function as terminal devices and send collected biometric information to Amazon Web Services (AWS (registered trademark)).

[0221] Analysis means

[0222] The server receives the vital signs and stores them in Amazon RDS, then uses an analysis program (using a generative AI model) to determine whether the heart rate and blood pressure are within normal ranges.

[0223] Notification means

[0224] If an abnormality is detected as a result of the analysis, an alert is sent to the user's smartphone using Firebase Cloud Messaging. At the same time, pre-registered contacts (family members and doctors) are also notified via SMS and email.

[0225] Support tool provision method

[0226] When a user requests a physical therapy video or health guidelines, the server retrieves the corresponding content from Amazon RDS and provides it to the user through a mobile application.

[0227] Schedule management tools

[0228] Users set their medication and exercise schedules in the mobile app. The server manages this schedule and generates reminders at the specified times. The mobile app then notifies the user of these reminders on their smartphone.

[0229] Signal generation means

[0230] When an abnormality is detected based on the user's biometric information, the server generates an output signal that displays an alert on the user's device and sends an electronic communication to designated parties.

[0231] 4. Specific Examples

[0232] For example, if User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring, the behavior is as follows:

[0233] 1. The smartwatch detects an abnormal heart rate.

[0234] 2. The collected heart rate data is sent to the AWS server via a smartphone app.

[0235] 3. The server analyzes the heart rate data and detects abnormalities.

[0236] 4. Use Firebase Cloud Messaging to send an alert to User B's smartphone.

[0237] 5. An abnormality notification is sent to User B's family and doctor via email or SMS.

[0238] 6. Play a breathing exercise video on your smartphone to help stabilize your heart rate.

[0239] 7. Remind me when it's time to take my medicine.

[0240] 5. Examples of prompts

[0241] "A 70-year-old man is wearing a smartwatch for monitoring his heart rate. If his heart rate becomes abnormally high, the AWS server will analyze it and send a notification to his smartphone using Firebase Cloud Messaging. The system will also notify his family of the abnormality via email and SMS, play a video on breathing exercises to stabilize his heart rate on the smartphone, and remind him to take his medication. Please generate appropriate prompts taking these situations into consideration."

[0242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0243] Step 1:

[0244] The user wears a smartwatch or health management sensor.

[0245] Input: User's biometric information (heart rate, blood pressure, temperature, etc.)

[0246] Data processing: Sensors collect data in real time

[0247] Output: Biometric data

[0248] Step 2:

[0249] The terminal transmits the collected biometric information to the server.

[0250] Input: Biometric data obtained from sensors

[0251] Data processing: Converting data into packets and sending them to a server via the Internet

[0252] Output: Biometric data transmitted to the server

[0253] Step 3:

[0254] The server receives the biometric information and stores it in a database.

[0255] Input: Transmitted biometric data

[0256] Data processing: storing data in a suitable format for the database

[0257] Output: Stored biometric data

[0258] Step 4:

[0259] The server analyzes the stored biometric information and detects any abnormalities.

[0260] Input: Stored biometric data

[0261] Data calculation: Analyze using generative AI models and compare with normal ranges

[0262] Output: Analysis result (normal / abnormal)

[0263] Step 5:

[0264] Generates a notification message if the server detects an abnormality.

[0265] Input: Analysis results (when an abnormality is detected)

[0266] Data processing: Generate notification messages about anomalies

[0267] Output: Information message

[0268] Step 6:

[0269] The server uses Firebase Cloud Messaging to send notifications to the user device.

[0270] Input: Notification message

[0271] Data processing: Encode the message in FMC protocol and send it

[0272] Output: An alert displayed on the user's terminal

[0273] Step 7:

[0274] At the same time, the server sends a notification to pre-registered contacts.

[0275] Input: Notification message, registered contact information

[0276] Data processing: Convert to email or SMS format and send to contacts

[0277] Output: Email or SMS notification sent to your contacts

[0278] Step 8:

[0279] The user terminal provides the support tool in response to a request.

[0280] Input: User request, analysis results

[0281] Data processing: Retrieve and play corresponding video and guideline content

[0282] Output: Supporting content (e.g., video) displayed on the user's device

[0283] Step 9:

[0284] The server generates and sends reminders based on a schedule set by the user.

[0285] Input: User's schedule information

[0286] Data processing: Generate and send reminders at specified times

[0287] Output: Reminder notification displayed on the user's device

[0288] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0289] This invention is a system that provides a safe and secure environment for users at home. By combining it with an emotion engine that recognizes the user's emotions, it provides advanced health management and support. This system monitors the user's health status in real time, promptly notifies the user when an abnormality occurs, and encourages appropriate action. It also provides physical therapy and health guidelines based on the user's emotional state, and reminders to improve quality of life.

[0290] System programs and their processing

[0291] Health Monitoring and Emotion Engine

[0292] User: Users wear smartwatches and health monitoring sensors on a daily basis, which collect real-time data such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[0293] Device: The device temporarily stores biometric and emotional state data collected from sensors and periodically transmits it to the server.

[0294] Server: The server stores the received biometric and emotional state data in a database and analyzes it using an analysis program that identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0295] Notification System

[0296] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties when an emergency call is required or there is a high risk.

[0297] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[0298] User: The user checks the alert on the device and contacts a doctor if necessary.

[0299] Support tools provided

[0300] User: When users need daily health management, they operate their device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[0301] Terminal: The terminal sends the user's request and emotional state information to the server.

[0302] Server: The server retrieves corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database and provides them tailored to the user's emotional state.

[0303] Terminal: The terminal displays the received support tool on the user's smartphone or tablet.

[0304] User: The user utilizes the displayed support tools and acts in accordance with the physical therapy and health guidelines.

[0305] Reminders

[0306] User: The user sets medication times and exercise schedules on the device.

[0307] Server: The server runs a program that stores the configured schedule in a database and generates reminders based on the schedule.

[0308] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time.

[0309] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[0310] Specific examples

[0311] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[0312] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[0313] 2. Device: Sends collected heart rate data and emotional state data to the server.

[0314] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[0315] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[0316] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[0317] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[0318] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[0319] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[0320] This system provides comprehensive monitoring, including emotional state, prompt notification, and emotional support tools to create an environment where users can live safely at home.

[0321] The processing flow will be explained below.

[0322] Health Monitoring and Emotion Engine

[0323] Step 1:

[0324] Users wear smartwatches and health management sensors that automatically collect data in real time, such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[0325] Step 2:

[0326] The device receives the collected biometric and emotional state data on the user's smartphone or tablet and temporarily stores it.

[0327] Step 3:

[0328] The terminal periodically transmits the collected data to the server.

[0329] Step 4:

[0330] The server stores the received biometric information and emotional state data in a database.

[0331] Step 5:

[0332] The server analyzes biometric information to identify abnormal values ​​such as heart rate and blood pressure, and also uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0333] Notification System

[0334] Step 6:

[0335] If the server detects an anomaly based on the analysis, it generates a notification message containing details about the user's health and emotional state.

[0336] Step 7:

[0337] The server generates and sends the notification message to the user's device and the designated parties, for example via email or SMS.

[0338] Step 8:

[0339] The notification message received by the device is displayed as an alert on the user's smartphone or tablet.

[0340] Step 9:

[0341] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[0342] Step 10:

[0343] The user can check the alerts on the device, contact a doctor if necessary, and receive relaxation techniques and support based on their emotional state.

[0344] Support tools provided

[0345] Step 11:

[0346] The user operates the device to request a support tool, for example, to watch a video on relaxation techniques.

[0347] Step 12:

[0348] The terminal transmits the user's request and emotional state information to the server.

[0349] Step 13:

[0350] The server retrieves the corresponding support tools (e.g., physical therapy videos or exercise guidelines) from the database and selects the appropriate content based on the emotion engine.

[0351] Step 14:

[0352] The server transmits the acquired support tool to the terminal.

[0353] Step 15:

[0354] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[0355] Step 16:

[0356] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[0357] Reminders

[0358] Step 17:

[0359] The user operates the device to set medication times and exercise schedules.

[0360] Step 18:

[0361] The server saves the configured schedule in the database.

[0362] Step 19:

[0363] The server launches a program to generate reminders based on the saved schedule.

[0364] Step 20:

[0365] The server sends a reminder notification to the device at the scheduled time.

[0366] Step 21:

[0367] Your device will receive reminder notifications and display alarms and notifications at the specified times.

[0368] Step 22:

[0369] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[0370] Through these steps, the system provides a safe and secure environment for users to live at home. Furthermore, by integrating an emotion engine, the system improves the user's quality of life by providing physical therapy and support tools based on the user's emotional state.

[0371] Example 2

[0372] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0373] The main problem that this invention aims to solve is to effectively manage a user's health and monitor their emotional state in their daily lives, and provide prompt and appropriate responses and support. Conventional health management systems lack the ability to detect abnormal conditions and provide emotional support, which can lead to a decline in the user's health and quality of life. In response, this invention aims to realize an environment in which users can live with peace of mind by comprehensively monitoring a user's biometric information and emotional state and providing appropriate notifications and support.

[0374] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a measurement means for collecting biometric information of the user, a communication means for transmitting the biometric information collected by the measurement means to the server, a diagnosis means for analyzing the biometric information and determining whether or not there is an abnormality, a warning means for sending a notification to the user and designated relevant parties when the diagnosis means determines an abnormality, a support means for providing the user with physical therapy and health guidelines, and a suggestion means for sending reminders based on the user's schedule. This makes it possible to comprehensively manage the user's health and emotional state in real time and provide prompt notification and appropriate support when an abnormality is detected.

[0375] "Measurement means" refers to a device or function that collects biometric information from a user.

[0376] "Communication means" refers to a device or function for transmitting the biometric information collected by the measurement means to a server.

[0377] "Diagnostic means" refers to a device or function that analyzes collected biological information to determine whether or not there is an abnormality.

[0378] "Alert means" refers to a device or function that sends a notification to the user and designated parties when the diagnostic means determines an abnormality.

[0379] "Assistive device" refers to a device or function that provides physical therapy or health guidelines to a user.

[0380] "Suggestion means" refers to a device or function that sends reminders based on the user's schedule.

[0381] "Reminder" refers to an alert or message that notifies a user of a specific action or schedule.

[0382] "Biometric information" refers to data such as a user's heart rate, blood pressure, body temperature, facial expression, and tone of voice.

[0383] "Abnormal" refers to a state in which the user's biological information deviates from a preset normal range.

[0384] "Emotional state" refers to the psychological and emotional state of a user as determined by their facial expression and tone of voice.

[0385] "Support tools" refers to resources such as physical therapy and health guidelines to help users manage their health.

[0386] This invention is a system for providing a user with a safe and secure living environment at home, monitoring the user's health and emotional state in real time, notifying them promptly when an abnormality occurs, and encouraging them to take appropriate action. This system consists of the following components:

[0387] Health Monitoring and Emotion Engine

[0388] User: The user wears a smartwatch or health management sensor on a daily basis. The smartwatch collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. Examples of such devices include the Apple Watch and Fitbit.

[0389] Device: The device temporarily stores the collected biometric and emotional data, which is then periodically sent to a server via Bluetooth or Wi-Fi.

[0390] Server: The server stores the received data in a database and analyzes the biometric information using an analysis program. Programming languages ​​such as Python and R can be used for the analysis. Specific libraries used include Pandas and NumPy. The analysis program identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0391] Notification System

[0392] Server: Based on the analysis results, the server generates a notification message if an anomaly is detected. This message is sent to the user and designated parties. The message can be sent via email or short message service (SMS).

[0393] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. Notifications can be pop-ups or audio alerts, and can also send emails or SMS to relevant parties such as family members or doctors.

[0394] User: The user checks the alerts on the device and contacts a doctor if necessary. For example, this includes the ability to press an emergency contact button to talk directly to a doctor.

[0395] Support tools provided

[0396] User: When users need daily health management, they operate the device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[0397] Terminal: The terminal sends the user's request and emotional state information to the server.

[0398] Server: The server retrieves the corresponding support tools from the database and provides health guidelines and physical therapy videos tailored to the user's emotional state. Specifically, YouTube (registered trademark) and Vimeo can be used as healthcare video resources.

[0399] Device: The device displays the received support tool on the user's smartphone or tablet, for example, by playing a physical therapy video using a video playback app.

[0400] User: The user utilizes the displayed support tools and follows the physical therapy and health guidelines.

[0401] Reminders

[0402] User: The user sets medication times and exercise schedules on the device using a dedicated app.

[0403] Server: The server stores the configured schedule in a database and runs a program that generates reminders based on the schedule.

[0404] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time, for example, a pop-up notification or a vibration alert.

[0405] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[0406] Specific examples

[0407] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[0408] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[0409] 2. Device: Sends collected heart rate data and emotional state data to the server.

[0410] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[0411] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[0412] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[0413] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[0414] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[0415] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[0416] When inputting a prompt sentence into a generative AI model, the following prompt could be used, for example:

[0417] "Please provide a detailed description of the system that monitors the user's health and emotional state, notifies them appropriately when abnormalities are detected, and provides support tools and reminders. Please also include specific examples of the hardware and software used."

[0418] Using this prompt, the generative AI model can automatically generate the system details.

[0419] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0420] Step 1:

[0421] User: Wears a smartwatch or health management sensor, which collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. The specific input is the user's biometric information, and the output is the collected data.

[0422] Step 2:

[0423] Device: Temporarily stores the biometric information collected from the sensor. The input is the biometric information acquired in step 1, and the output is the data stored in the device's internal storage.

[0424] Step 3:

[0425] Device: The temporarily stored data is periodically sent to the server via Bluetooth or Wi-Fi. The input is the biometric information stored in the device's internal storage, and the output is the data sent to the server.

[0426] Step 4:

[0427] Server: Stores the received data in a database. The input is the biometric information sent from the device, and the output is the data stored in the database.

[0428] Step 5:

[0429] Server: Analyzes the biometric information using an analysis program. This uses Python or R libraries (e.g., Pandas or NumPy). The input is the biometric information stored in the database, and the output is the analysis results.

[0430] Step 6:

[0431] Server: Generates a notification message based on the analysis results if an anomaly is detected. The input is the analysis results, and the output is the generated notification message.

[0432] Step 7:

[0433] Server: Sends generated notification messages to users and designated parties. The input is the generated notification message and the output is the sent notification.

[0434] Step 8:

[0435] Terminal: Receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification received from the server, and the output is the alert displayed on the user's terminal.

[0436] Step 9:

[0437] User: Checks the alert and contacts a doctor if necessary. The input is the alert displayed on the user's device, and the output is the user's action (e.g., contacting a doctor).

[0438] Step 10:

[0439] User: Requests a support tool by operating a terminal. The input is the user's request, and the output is the request data generated by the terminal.

[0440] Step 11:

[0441] Terminal: Sends the user's request and emotion information to the server. The input is the user's request and emotion information, and the output is the request data sent to the server.

[0442] Step 12:

[0443] Server: Retrieves the appropriate support tool from the database and generates the content to be provided. The input is the request data sent to the server and the content of the database, and the output is the content of the generated support tool.

[0444] Step 13:

[0445] Terminal: The support tool is displayed on the user's smartphone or tablet. The input is the content of the support tool sent from the server, and the output is the support tool displayed on the user's terminal.

[0446] Step 14:

[0447] User: Uses the displayed support tools and acts in accordance with the health guidelines. The input is the support tools displayed on the terminal, and the output is the user's actions (e.g., performing physical therapy).

[0448] Step 15:

[0449] User: Sets medication times and exercise schedules on the device. The input is the schedule information set by the user, and the output is the schedule data saved on the device.

[0450] Step 16:

[0451] Server: Saves the set schedule in the database and runs the reminder generation program. The input is the schedule data sent from the terminal, and the output is the schedule saved in the database and the generated reminder notification.

[0452] Step 17:

[0453] Terminal: Receives reminder notifications from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server, and the output is the alarm and notification displayed on the user terminal.

[0454] Step 18:

[0455] User: Checks the reminder notification and acts according to the instructions. The input is the reminder notification displayed on the device, and the output is the user's action (e.g., taking medicine).

[0456] (Application example 2)

[0457] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0458] Conventional health monitoring systems are limited to collecting and analyzing users' biometric information, without taking into account their emotional state. This makes it difficult to respond appropriately in situations that are influenced by emotions such as stress or anxiety. In addition, notification systems that detect abnormalities only notify users of the abnormality and do not provide specific countermeasures or support. This has led to issues that do not adequately improve users' quality of life or ensure their safety.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0460] In this invention, the server includes means for collecting biometric information and emotional information of the user, means for analyzing the collected data to determine abnormalities or emotional states, means for sending notifications to the user and designated parties when abnormalities or specific emotional states are determined, means for providing the user with physical therapy and health guidelines based on their emotional states, and means for sending reminders based on the user's schedule. This makes it possible to monitor the user's health state, respond quickly when abnormalities occur, and provide appropriate support tools according to their emotional states.

[0461] The "sensor means" is a device for collecting biometric and emotional information of a user.

[0462] The "terminal means" is a device for transmitting the biometric information and emotional information collected by the sensor means to the server.

[0463] The "analysis means" is a device or program for analyzing the bio-information and emotional information and determining abnormalities and emotional states.

[0464] The "notification means" is a device or program for sending a notification to the user and designated related parties when the analysis means determines an abnormality or a particular emotional state.

[0465] The "support tool providing means" is a device or program for providing physical therapy or health guidelines based on the user's emotional state.

[0466] A "reminder means" is a device or program for sending reminders based on a user's schedule.

[0467] "Biometric information" refers to data such as the user's heart rate, blood pressure, and body temperature.

[0468] "Emotional information" refers to the emotional state of the user that can be obtained from facial expressions, tone of voice, etc.

[0469] The present invention is a system that monitors a user's biological information and emotional state, promptly notifies the user if an abnormality is detected, and provides support tools according to the user's emotional state. Specific embodiments of the system are described below.

[0470] Overall system configuration

[0471] Data collection

[0472] Sensor means

[0473] The user wears a sensor that collects biometric information such as heart rate, blood pressure, and body temperature. This sensor can be a smartwatch or a dedicated health management sensor. In addition, a camera and microphone can be used to collect emotional information such as the user's facial expressions and tone of voice.

[0474] Terminal means

[0475] The data collected by the sensor means is sent to devices such as smartphones and tablets, allowing the user's biological and emotional information to be monitored in real time.

[0476] Data Analysis and Notification

[0477] server

[0478] The data sent from the device is stored on a server via the Internet. The server analyzes the data using software such as Python, TENSORFLOW (registered trademark), and Keras. Specifically, it analyzes data such as heart rate, blood pressure, and body temperature to detect abnormal values. It also uses an emotion engine to determine the user's emotional state from facial expressions and tone of voice.

[0479] Notification means

[0480] If the server detects an abnormality or a specific emotional state, it will send a notification to the user and designated parties using Firebase Cloud Messaging, Twilio API, etc. The notification will be displayed as an alert on the user's device and will be sent to the parties via email or SMS.

[0481] Providing support tools

[0482] server

[0483] The server provides appropriate physical therapy and health guidelines based on the user's emotional state, including videos and guidelines created using React Native and Rest APIs. For example, if the user is in a high-stress state, a relaxation video will be provided.

[0484] Terminal

[0485] Support tools sent from the server are displayed on the user's smartphone or tablet, allowing the user to act in accordance with the presented physical therapy and health guidelines.

[0486] Reminders

[0487] server

[0488] Based on the schedule set by the user, the server generates reminders and notifies them at the specified times.

[0489] Terminal

[0490] Reminder notifications received from the server are displayed on the user's device, helping the user to remember to take their medicine or to exercise regularly.

[0491] Specific examples

[0492] For example, if a 65-year-old user wears a smartwatch and monitors their heart rate and blood pressure in real time, and the emotion engine detects an abnormally high heart rate and a high stress state, the system will operate as follows:

[0493] 1. Sensor means

[0494] The smartwatch detects abnormal heart rates and senses the user's facial expressions and tone of voice through the camera and microphone.

[0495] 2. Terminal means

[0496] The collected data is sent to a smartphone.

[0497] 3. Server

[0498] The data is analyzed to detect abnormal heart rates and high stress states.

[0499] 4. Means of notification

[0500] Based on the analysis results, notifications are sent to the user and relevant parties. An alert stating "An abnormal heart rate and high stress state have been detected" is displayed on the user's smartphone, and a similar message is sent to relevant parties via email.

[0501] 5. Means of providing support tools

[0502] A relaxing video is displayed on the user's smartphone.

[0503] 6. Reminders

[0504] When it's time for the user to take their medication, a notification will appear on their smartphone.

[0505] Prompt Sentence Examples

[0506] "If my heart rate exceeds 100, show me a stress relief video and send a notification message to my family."

[0507] In this way, the present invention monitors the user's health and emotional state, promptly notifies them in the event of an abnormality, and provides appropriate support, thereby creating an environment in which the user can live with peace of mind.

[0508] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0509] Step 1:

[0510] The user wears a sensor device (e.g., a smart watch). The sensor collects biometric information such as heart rate, blood pressure, and body temperature in real time, and also detects the user's facial expressions and tone of voice. The input is biometric and emotional information collected by the sensor.

[0511] Step 2:

[0512] The terminal means (e.g., a smartphone) temporarily stores the biometric and emotional information received from the sensor and periodically transmits it to a server via the Internet. The input is the biometric and emotional information sent from the sensor, and the output is the data sent to the server. Specifically, it receives data from the sensor using Bluetooth Low Energy (BLE), converts the data into JSON format, and transmits it to the server.

[0513] Step 3:

[0514] The server stores the biometric and emotional information received from the device in a database and performs data analysis using Python, TensorFlow, Keras, etc. The input is the biometric and emotional information sent from the device, and the output is the analysis results (abnormal values ​​and specific emotional states). Specifically, the server organizes the data as time-series data and analyzes it by applying anomaly detection algorithms and emotion recognition models.

[0515] Step 4:

[0516] If the server detects an anomaly or a specific emotional state, it uses Firebase Cloud Messaging or Twilio API to generate a notification and send it to the user and designated stakeholders. The input is the analysis result, and the output is the notification message. Specifically, it displays an alert on the user's smartphone and sends an email or SMS to the relevant stakeholders.

[0517] Step 5:

[0518] Based on the analysis results, the server provides physical therapy and health guidelines tailored to the user's emotional state. The input is the analysis results of biometric and emotional information, and the output is the proposed support tools (videos and guidelines). Specifically, it retrieves relevant videos and guidelines from the database and sends them to the user's device using React Native and a Rest API.

[0519] Step 6:

[0520] The device displays support tools (relaxation videos and health guidelines) received from the server to the user. The input is the support tools sent from the server, and the output is the content displayed on the device display. Specifically, the device displays the received videos and guidelines on the playback screen.

[0521] Step 7:

[0522] The server generates reminders based on the schedule set by the user and sends notifications to the device at the specified time. The input is the user's schedule information, and the output is the reminder notification. Specifically, it calculates the timing of the reminder notification based on the schedule information and sends the notification using Firebase Cloud Messaging.

[0523] Step 8:

[0524] The terminal displays the reminder notification received from the server to the user at the specified time. The input is the reminder notification from the server, and the output is the reminder displayed to the user. Specifically, the terminal generates an alarm and a pop-up notification at the specified time to notify the user.

[0525] In this way, the entire system's processing is coordinated at each step, enabling comprehensive monitoring of the user's health and emotional state and providing appropriate responses and support.

[0526] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0527] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0528] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0529] [Second embodiment]

[0530] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0531] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0532] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0533] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0534] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0535] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0536] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0537] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0538] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0539] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0540] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0541] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0542] This invention is a system for providing a safe and secure living environment for users at home. This system monitors the user's health status in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[0543] System programs and their processing

[0544] 1. Health monitoring

[0545] User: The user wears a smartwatch and health monitoring sensors on a daily basis, which collect biometric information such as heart rate, blood pressure, and body temperature in real time.

[0546] Device: The device periodically transmits the collected biometric information to the server. This device includes any communication-enabled device that the user normally uses, such as a smartphone or tablet.

[0547] Server: The server stores the received biometric information in a database and analyzes it using an analysis program, which includes algorithms that determine whether the heart rate and blood pressure are within normal ranges.

[0548] 2. Notification System

[0549] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties in the event of an emergency or high risk.

[0550] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[0551] User: The user checks the alert displayed on the device and contacts a doctor if necessary.

[0552] 3. Providing support tools

[0553] User: When a user needs to manage their daily health, they operate the device and request a support tool, such as watching a physical therapy video.

[0554] Terminal: The terminal starts the support tool providing program based on the user's request and displays the required content.

[0555] Server: The server retrieves the corresponding support tools (e.g., breathing exercise videos and exercise guidelines) from the database and sends them to the terminal.

[0556] User: Users can access support content at their own pace and practice physical therapy and health guidelines.

[0557] 4. Reminders

[0558] User: The user sets up a schedule for taking medicines and exercising on the device, enabling regular health management.

[0559] Server: The server runs a program that manages the user's schedule and generates reminders.

[0560] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[0561] User: The user follows the reminders to take their medicine and manage their health.

[0562] Specific examples

[0563] For example, assume that User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high, the procedure is as follows:

[0564] 1. User: The smartwatch detects an abnormal heart rate.

[0565] 2. Device: Sends collected heart rate data to the server.

[0566] 3. Server: Analyzes the data and detects anomalies.

[0567] 4. Server: Sends notifications about abnormal heart rate to User B and his family.

[0568] 5. Device: An alert is displayed on User B's smartphone and an email is sent to family members.

[0569] 6. User: Checks the alert and contacts the doctor.

[0570] 7. Support tool provided: The device will play a breathing exercise video to stabilize your heart rate.

[0571] 8. Reminder: User B sets the time to take the medicine, and the device notifies him / her at the specified time.

[0572] This system provides continuous monitoring, prompt notifications, and appropriate support tools to help users create an environment where they can live safely at home.

[0573] The processing flow will be explained below.

[0574] 1. Health monitoring

[0575] Step 1:

[0576] The user wears a smartwatch or health management sensor, which automatically collects biometric information such as heart rate, blood pressure, and body temperature in real time.

[0577] Step 2:

[0578] The device receives the collected biometric information on the user's smartphone or tablet and temporarily stores it.

[0579] Step 3:

[0580] The terminal transmits biometric information to the server at regular intervals.

[0581] Step 4:

[0582] The server stores the received biometric information in a database.

[0583] Step 5:

[0584] The server analyzes the biometric information and runs an analysis program to determine whether there are any abnormal values ​​for heart rate or blood pressure.

[0585] 2. Notification System

[0586] Step 6:

[0587] If the server detects an abnormality based on the analysis results, it generates a message to notify the user and designated parties.

[0588] Step 7:

[0589] The server sends the generated notification message to the user's terminal and the designated parties.

[0590] Step 8:

[0591] Based on the notification message received by the device, an alert is displayed on the user's smartphone or tablet.

[0592] Step 9:

[0593] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[0594] Step 10:

[0595] The user checks the alert on the device and contacts a doctor if necessary.

[0596] 3. Providing support tools

[0597] Step 11:

[0598] When a user needs to manage their daily health, they can operate the device to request a support tool, such as watching a video on breathing exercises.

[0599] Step 12:

[0600] The terminal sends the user's request to the server.

[0601] Step 13:

[0602] The server retrieves the corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database.

[0603] Step 14:

[0604] The server transmits the acquired support tool to the terminal.

[0605] Step 15:

[0606] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[0607] Step 16:

[0608] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[0609] 4. Reminders

[0610] Step 17:

[0611] The user operates the device to set medication times and exercise schedules.

[0612] Step 18:

[0613] The server saves the configured schedule in the database.

[0614] Step 19:

[0615] The server launches a program to generate reminders based on the saved schedule.

[0616] Step 20:

[0617] The server sends a reminder notification to the device at the scheduled time.

[0618] Step 21:

[0619] The device receives the reminder notification, displays a notification on the user's smartphone or tablet, and sounds an alarm.

[0620] Step 22:

[0621] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[0622] In this way, the system monitors the user's health at each step and provides appropriate notifications and support tools, creating an environment where the user can live safely at home.

[0623] Example 1

[0624] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0625] In modern society, home health management is crucial, especially for the elderly and those with chronic illnesses. However, achieving this requires complex health management systems that monitor vital signs in real time and respond quickly to any abnormalities. It also requires a means to support daily health management and provide appropriate advice and reminders. Conventional systems struggle to meet these requirements and often only provide partial solutions. Therefore, there is an urgent need to develop more sophisticated and comprehensive health management systems.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0627] In this invention, the server includes an input means for collecting biometric information of a user, a communication means for transmitting the biometric information collected by the input means to a processing system, an analysis means for analyzing the biometric information to determine whether an abnormality is present, a notification means for sending a notification to the user and designated related parties if the analysis means determines an abnormality, a support means for providing the user with health management and health guidelines, a schedule support means for sending reminders based on the user's schedule, a message generation means for the notification means to send notifications to the user and related parties including specific abnormalities and recommended actions, and a reminder generation means for the schedule support means to generate periodic reminders based on a time set by the user. This enables comprehensive real-time health management, prompt notification of abnormalities, and support for improving quality of life.

[0628] A "user" is an individual who uses the system to manage their health.

[0629] "Biometric information" is data relating to the user's physical condition, such as heart rate, blood pressure, and body temperature.

[0630] "Input means" refers to a device such as a sensor that collects biometric information.

[0631] A "communication means" is a device or method that transmits biometric information collected by an input means to a processing system.

[0632] "Analysis means" refers to algorithms or software that analyze biometric information and determine whether or not there are any abnormalities.

[0633] The "notification means" is a function that sends a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[0634] "Support means" refers to services and tools that provide users with health management and health guidelines.

[0635] The "schedule support means" is a function that sends reminders based on the user's schedule.

[0636] The "message generating means" is an algorithm or software that generates a message including the abnormality details and recommended actions when the notification means sends a notification.

[0637] The "reminder generation means" is a function that generates periodic reminders based on a time set by the user.

[0638] "Processing system" refers to the entire system for receiving, managing, and analyzing biometric information and providing notifications and support based on the results.

[0639] This invention is a health management system that provides a safe and secure environment for users at home. This system monitors the user's biological information in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[0640] Health Status Monitoring

[0641] User: The user wears a smartwatch or health monitoring sensor to measure heart rate, blood pressure, and temperature. The sensor collects real-time biometric information on a daily basis.

[0642] Device: The device periodically transmits the biometric information acquired from the smartwatch to the server. This device is the user's smartphone or tablet. The smartphone or tablet transmits data to the server using Bluetooth, Wi-Fi, or mobile data communication.

[0643] Server: The server stores the received biometric data in a database and analyzes the data using analytical programs. For example, algorithms are used to determine whether heart rate or blood pressure is within normal ranges. The algorithms generate warnings if abnormal values ​​are detected.

[0644] Notification System

[0645] Server: If the server detects an abnormality based on the analysis results, it immediately generates a notification. The notification will include the specific abnormality and a recommended action, such as "Your heart rate is out of the normal range. Contact your doctor immediately."

[0646] Device: The device receives notifications from the server and displays an alert on the user's smartphone or tablet. At the same time, the server also sends notifications to relevant parties (e.g., family members or doctors) via email or SMS.

[0647] User: The user checks the alert displayed on the device and contacts a doctor if necessary. Notifications that can be viewed by the user allow for prompt and appropriate action to be taken.

[0648] Support tools provided

[0649] User: When a user needs health management, they operate their device to request support tools. For example, they may want to view physical therapy videos or breathing exercise guides.

[0650] Terminal: The terminal receives a user request, launches the support tool providing program, retrieves the necessary content from the server, and displays it.

[0651] Server: The server retrieves appropriate support tools (such as breathing exercise videos or exercise guidelines) from the database and sends them to the device.

[0652] Users: Users can use support content to practice physical therapy and health guidelines.

[0653] Reminders

[0654] User: The user sets the time to take medicine and exercise schedule on the device. For example, the user sets the time to take medicine at 9:00 every day using a dedicated app.

[0655] Server: The server manages the user's schedule and generates reminders at the specified time.

[0656] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[0657] User: Users can take their medicines according to the reminders and maintain a regular health regime.

[0658] Examples of prompt statements

[0659] Describe how the following system works: A user wears a smartwatch to monitor their heart rate. If the heart rate reaches an abnormal level, the server generates a notification, displays an alert on the user's smartphone, and provides a video with appropriate breathing exercises.

[0660] Through these functions, the system provides a safe and secure environment for users to live at home. It enables comprehensive and real-time health management, enables rapid response in the event of an abnormality, and provides tools to support daily health management.

[0661] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0662] Specific explanation of processing steps

[0663] Step 1: Collecting biometric information

[0664] User: The user wears a smartwatch or health monitoring sensors that measure heart rate, blood pressure, and body temperature. The input is real-time biometric information from these sensors. The output is the acquired biometric data: heart rate, blood pressure, and body temperature.

[0665] How it works: When a user wakes up in the morning and puts on the smartwatch, the sensors start collecting data on their heart rate and body temperature from that moment on.

[0666] Step 2: Sending biometric information

[0667] Terminal: The terminal periodically transmits biometric information acquired from the smartwatch to the server. The input is biometric information acquired from the smartwatch via Bluetooth connection. The output is biometric information transmitted to the server via Wi-Fi or mobile data communication.

[0668] How it works: The smartwatch connects to the smartphone via Bluetooth, and the smartphone sends data to the server using Wi-Fi.

[0669] Step 3: Analysis of biological information

[0670] Server: The server stores the received biometric information in a database and analyzes the data using an analysis program. The input is the biometric information received from the device. The output is the analysis result, judging whether it is within the normal range or abnormal.

[0671] Specific operation: The server analyzes the received heart rate data, calculates the average and standard deviation, and generates an alert if an abnormal value is detected.

[0672] Step 4: Generate a notification message

[0673] Server: The server immediately generates a notification message if an anomaly is detected based on the analysis results. The input is the anomaly judgment of the analysis results. The output is a notification message containing the specific anomaly content and recommended actions.

[0674] What happens: The server generates the message "Your heart rate is outside the normal range. Contact your doctor immediately."

[0675] Step 5: Sending notifications

[0676] Server: The server generates and sends notification messages to the user and the designated parties. The input is the generated notification message. The output is the notification message sent to the user's smartphone and the designated parties.

[0677] Specific operation: The server sends a notification to the user's smartphone and simultaneously notifies the user's family and doctor by email.

[0678] Step 6: Receive and confirm notifications

[0679] Terminal: The terminal receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification message sent from the server. The output is the alert displayed on the user's smartphone.

[0680] User: The user checks the alert displayed on the device and contacts a doctor if necessary. The input is the alert message displayed on the smartphone. The output is the user's response (e.g., contacting a doctor).

[0681] Specific operation: The user sees the "abnormal heart rate" notification on their smartphone screen and then uses the app to contact their doctor.

[0682] Step 7: Request Support Tools

[0683] User: The user requests the necessary support tools through the terminal. The input is an operation to request a physical therapy video or a breathing exercise guide. The output is the information sent in response to the request.

[0684] Specific operation: The user selects "Breathing exercise video" from the menu of the dedicated app.

[0685] Step 8: Providing support tools

[0686] Terminal: The terminal sends the user's request to the server and starts the support tool providing program. The input is the user's request information. The output is the start of the support tool providing program and the transmission of the request to the server.

[0687] Server: The server retrieves the corresponding support tool (such as a breathing exercise video or exercise guidelines) from the database and sends it to the device. The input is the user's request information. The output is the data of the support tool that was sent.

[0688] Specific operation: The server sends a link to a breathing exercise video to the device, and the device plays the video.

[0689] Step 9: Set reminders

[0690] User: The user sets up schedules such as medication times and exercise schedules on the device. The input is the schedule setting information in the dedicated app. The output is the set schedule information.

[0691] Specific operation: The user uses the calendar function of the dedicated app to schedule taking the medicine at 9:00 every day.

[0692] Step 10: Generate reminders

[0693] Server: The server manages the user's schedule and generates reminders at the specified time. The input is the user's schedule information. The output is the generated reminder notification.

[0694] What happens: The server generates a reminder notification every day at 9:00 AM based on the user's settings.

[0695] Step 11: Reminder Notifications

[0696] Terminal: The terminal receives reminders from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server. The output is the reminder displayed on the terminal.

[0697] User: The user takes medicine according to the reminder and manages their health. The input is the reminder notification displayed on the device. The output is the user's coping behavior (e.g., taking medicine).

[0698] Specific operation: At 9 o'clock, the user's smartphone will sound an alarm and notify them that it's time for their medicine, and they will then take their medicine.

[0699] (Application example 1)

[0700] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] For elderly people and those with chronic illnesses, constant monitoring of their health status and prompt response when an abnormality occurs are extremely important for them to continue living safely at home. However, conventional systems lack the functionality to monitor users' health status in real time or an immediate notification system in case of an emergency, making it difficult to fully ensure user safety. Furthermore, the provision of support tools for daily health management and improving quality of life is also insufficient.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0703] In this invention, the server includes a signal generating means for generating an output signal when an abnormality is detected based on the user's biological information, a support tool providing means, and a schedule management means. This makes it possible to monitor the user's health condition in real time and quickly detect and notify abnormalities. It also provides various functions to support daily health management and can support the user in improving their lifestyle habits through reminders.

[0704] The "sensor means" is a device for collecting biometric information of a user.

[0705] The "terminal means" is a communication device for transmitting the biometric information collected by the sensor means to the server.

[0706] The "analysis means" is a system that analyzes the biometric information sent to the server and determines whether or not there is an abnormality.

[0707] The "notification means" is a function for sending a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[0708] The "support tool providing means" is a function for providing physical therapy and physical guidelines to the user.

[0709] The "schedule management means" is a function for sending reminders based on the user's schedule.

[0710] The "signal generating means" is a device or system that generates an output signal when an abnormality is detected based on the user's biological information.

[0711] "Electronic communication" means a means of communication to send notices to designated parties via email or SMS.

[0712] An "alert" is a warning message that is displayed on the user's terminal when an abnormality occurs.

[0713] 1. System Configuration

[0714] The present invention provides a system that monitors the user's health condition in real time and notifies the user promptly if an abnormality occurs, thereby providing an environment in which the user can live safely at home. Specific embodiments will be described below.

[0715] 2. Hardware and Software Used

[0716] Hardware:

[0717] Smartphone (iOS / Android)

[0718] Smartwatches (Apple Watch, Fitbit, etc.)

[0719] Health management sensors (heart rate sensors, blood pressure monitors, etc.)

[0720] software:

[0721] Mobile application (cross-platform using Flutter)

[0722] Server (using Amazon Web Services)

[0723] Database (using Amazon RDS)

[0724] Notification system (Firebase Cloud Messaging)

[0725] 3. Operation of various means

[0726] Sensor means

[0727] Users wear smartwatches and health monitoring sensors that collect real-time biometric information such as heart rate, blood pressure, and body temperature.

[0728] Terminal means

[0729] Smartphones and tablets function as terminals and send collected biometric information to Amazon Web Services (AWS).

[0730] Analysis means

[0731] The server receives the vital signs and stores them in Amazon RDS, then uses an analysis program (using a generative AI model) to determine whether the heart rate and blood pressure are within normal ranges.

[0732] Notification means

[0733] If an abnormality is detected as a result of the analysis, an alert is sent to the user's smartphone using Firebase Cloud Messaging. At the same time, pre-registered contacts (family members and doctors) are also notified via SMS and email.

[0734] Support tool provision method

[0735] When a user requests a physical therapy video or health guidelines, the server retrieves the corresponding content from Amazon RDS and provides it to the user through a mobile application.

[0736] Schedule management tools

[0737] Users set their medication and exercise schedules in the mobile app. The server manages this schedule and generates reminders at the specified times. The mobile app then notifies the user of these reminders on their smartphone.

[0738] Signal generation means

[0739] When an abnormality is detected based on the user's biometric information, the server generates an output signal that displays an alert on the user's device and sends an electronic communication to designated parties.

[0740] 4. Specific Examples

[0741] For example, if User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring, the behavior is as follows:

[0742] 1. The smartwatch detects an abnormal heart rate.

[0743] 2. The collected heart rate data is sent to the AWS server via a smartphone app.

[0744] 3. The server analyzes the heart rate data and detects abnormalities.

[0745] 4. Use Firebase Cloud Messaging to send an alert to User B's smartphone.

[0746] 5. An abnormality notification is sent to User B's family and doctor via email or SMS.

[0747] 6. Play a breathing exercise video on your smartphone to help stabilize your heart rate.

[0748] 7. Remind me when it's time to take my medicine.

[0749] 5. Examples of prompts

[0750] "A 70-year-old man is wearing a smartwatch for monitoring his heart rate. If his heart rate becomes abnormally high, the AWS server will analyze it and send a notification to his smartphone using Firebase Cloud Messaging. The system will also notify his family of the abnormality via email and SMS, play a video on breathing exercises to stabilize his heart rate on the smartphone, and remind him to take his medication. Please generate appropriate prompts taking these situations into consideration."

[0751] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0752] Step 1:

[0753] The user wears a smartwatch or health management sensor.

[0754] Input: User's biometric information (heart rate, blood pressure, temperature, etc.)

[0755] Data processing: Sensors collect data in real time

[0756] Output: Biometric data

[0757] Step 2:

[0758] The terminal transmits the collected biometric information to the server.

[0759] Input: Biometric data obtained from sensors

[0760] Data processing: Converting data into packets and sending them to a server via the Internet

[0761] Output: Biometric data transmitted to the server

[0762] Step 3:

[0763] The server receives the biometric information and stores it in a database.

[0764] Input: Transmitted biometric data

[0765] Data processing: storing data in a suitable format for the database

[0766] Output: Stored biometric data

[0767] Step 4:

[0768] The server analyzes the stored biometric information and detects any abnormalities.

[0769] Input: Stored biometric data

[0770] Data calculation: Analyze using generative AI models and compare with normal ranges

[0771] Output: Analysis result (normal / abnormal)

[0772] Step 5:

[0773] Generates a notification message if the server detects an abnormality.

[0774] Input: Analysis results (when an abnormality is detected)

[0775] Data processing: Generate notification messages about anomalies

[0776] Output: Information message

[0777] Step 6:

[0778] The server uses Firebase Cloud Messaging to send notifications to the user device.

[0779] Input: Notification message

[0780] Data processing: Encode the message in FMC protocol and send it

[0781] Output: An alert displayed on the user's terminal

[0782] Step 7:

[0783] At the same time, the server sends a notification to pre-registered contacts.

[0784] Input: Notification message, registered contact information

[0785] Data processing: Convert to email or SMS format and send to contacts

[0786] Output: Email or SMS notification sent to your contacts

[0787] Step 8:

[0788] The user terminal provides the support tool in response to a request.

[0789] Input: User request, analysis results

[0790] Data processing: Retrieve and play corresponding video and guideline content

[0791] Output: Supporting content (e.g., video) displayed on the user's device

[0792] Step 9:

[0793] The server generates and sends reminders based on a schedule set by the user.

[0794] Input: User's schedule information

[0795] Data processing: Generate and send reminders at specified times

[0796] Output: Reminder notification displayed on the user's device

[0797] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0798] This invention is a system that provides a safe and secure environment for users at home. By combining it with an emotion engine that recognizes the user's emotions, it provides advanced health management and support. This system monitors the user's health status in real time, promptly notifies the user when an abnormality occurs, and encourages appropriate action. It also provides physical therapy and health guidelines based on the user's emotional state, and reminders to improve quality of life.

[0799] System programs and their processing

[0800] Health Monitoring and Emotion Engine

[0801] User: Users wear smartwatches and health monitoring sensors on a daily basis, which collect real-time data such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[0802] Device: The device temporarily stores biometric and emotional state data collected from sensors and periodically transmits it to the server.

[0803] Server: The server stores the received biometric and emotional state data in a database and analyzes it using an analysis program that identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0804] Notification System

[0805] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties when an emergency call is required or there is a high risk.

[0806] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[0807] User: The user checks the alert on the device and contacts a doctor if necessary.

[0808] Support tools provided

[0809] User: When users need daily health management, they operate their device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[0810] Terminal: The terminal sends the user's request and emotional state information to the server.

[0811] Server: The server retrieves corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database and provides them tailored to the user's emotional state.

[0812] Terminal: The terminal displays the received support tool on the user's smartphone or tablet.

[0813] User: The user utilizes the displayed support tools and acts in accordance with the physical therapy and health guidelines.

[0814] Reminders

[0815] User: The user sets medication times and exercise schedules on the device.

[0816] Server: The server runs a program that stores the configured schedule in a database and generates reminders based on the schedule.

[0817] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time.

[0818] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[0819] Specific examples

[0820] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[0821] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[0822] 2. Device: Sends collected heart rate data and emotional state data to the server.

[0823] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[0824] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[0825] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[0826] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[0827] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[0828] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[0829] This system provides comprehensive monitoring, including emotional state, prompt notification, and emotional support tools to create an environment where users can live safely at home.

[0830] The processing flow will be explained below.

[0831] Health Monitoring and Emotion Engine

[0832] Step 1:

[0833] Users wear smartwatches and health management sensors that automatically collect data in real time, such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[0834] Step 2:

[0835] The device receives the collected biometric and emotional state data on the user's smartphone or tablet and temporarily stores it.

[0836] Step 3:

[0837] The terminal periodically transmits the collected data to the server.

[0838] Step 4:

[0839] The server stores the received biometric information and emotional state data in a database.

[0840] Step 5:

[0841] The server analyzes biometric information to identify abnormal values ​​such as heart rate and blood pressure, and also uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0842] Notification System

[0843] Step 6:

[0844] If the server detects an anomaly based on the analysis, it generates a notification message containing details about the user's health and emotional state.

[0845] Step 7:

[0846] The server generates and sends the notification message to the user's device and the designated parties, for example via email or SMS.

[0847] Step 8:

[0848] The notification message received by the device is displayed as an alert on the user's smartphone or tablet.

[0849] Step 9:

[0850] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[0851] Step 10:

[0852] The user can check the alerts on the device, contact a doctor if necessary, and receive relaxation techniques and support based on their emotional state.

[0853] Support tools provided

[0854] Step 11:

[0855] The user operates the device to request a support tool, for example, to watch a video on relaxation techniques.

[0856] Step 12:

[0857] The terminal transmits the user's request and emotional state information to the server.

[0858] Step 13:

[0859] The server retrieves the corresponding support tools (e.g., physical therapy videos or exercise guidelines) from the database and selects the appropriate content based on the emotion engine.

[0860] Step 14:

[0861] The server transmits the acquired support tool to the terminal.

[0862] Step 15:

[0863] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[0864] Step 16:

[0865] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[0866] Reminders

[0867] Step 17:

[0868] The user operates the device to set medication times and exercise schedules.

[0869] Step 18:

[0870] The server saves the configured schedule in the database.

[0871] Step 19:

[0872] The server launches a program to generate reminders based on the saved schedule.

[0873] Step 20:

[0874] The server sends a reminder notification to the device at the scheduled time.

[0875] Step 21:

[0876] Your device will receive reminder notifications and display alarms and notifications at the specified times.

[0877] Step 22:

[0878] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[0879] Through these steps, the system provides a safe and secure environment for users to live at home. Furthermore, by integrating an emotion engine, the system improves the user's quality of life by providing physical therapy and support tools based on the user's emotional state.

[0880] Example 2

[0881] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0882] The main problem that this invention aims to solve is to effectively manage a user's health and monitor their emotional state in their daily lives, and provide prompt and appropriate responses and support. Conventional health management systems lack the ability to detect abnormal conditions and provide emotional support, which can lead to a decline in the user's health and quality of life. In response, this invention aims to realize an environment in which users can live with peace of mind by comprehensively monitoring a user's biometric information and emotional state and providing appropriate notifications and support.

[0883] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a measurement means for collecting biometric information of the user, a communication means for transmitting the biometric information collected by the measurement means to the server, a diagnosis means for analyzing the biometric information and determining whether or not there is an abnormality, a warning means for sending a notification to the user and designated relevant parties when the diagnosis means determines an abnormality, a support means for providing the user with physical therapy and health guidelines, and a suggestion means for sending reminders based on the user's schedule. This makes it possible to comprehensively manage the user's health and emotional state in real time and provide prompt notification and appropriate support when an abnormality is detected.

[0884] "Measurement means" refers to a device or function that collects biometric information from a user.

[0885] "Communication means" refers to a device or function for transmitting the biometric information collected by the measurement means to a server.

[0886] "Diagnostic means" refers to a device or function that analyzes collected biological information to determine whether or not there is an abnormality.

[0887] "Alert means" refers to a device or function that sends a notification to the user and designated parties when the diagnostic means determines an abnormality.

[0888] "Assistive device" refers to a device or function that provides physical therapy or health guidelines to a user.

[0889] "Suggestion means" refers to a device or function that sends reminders based on the user's schedule.

[0890] "Reminder" refers to an alert or message that notifies a user of a specific action or schedule.

[0891] "Biometric information" refers to data such as a user's heart rate, blood pressure, body temperature, facial expression, and tone of voice.

[0892] "Abnormal" refers to a state in which the user's biological information deviates from a preset normal range.

[0893] "Emotional state" refers to the psychological and emotional state of a user as determined by their facial expression and tone of voice.

[0894] "Support tools" refers to resources such as physical therapy and health guidelines to help users manage their health.

[0895] This invention is a system for providing a user with a safe and secure living environment at home, monitoring the user's health and emotional state in real time, notifying them promptly when an abnormality occurs, and encouraging them to take appropriate action. This system consists of the following components:

[0896] Health Monitoring and Emotion Engine

[0897] User: The user wears a smartwatch or health management sensor on a daily basis. The smartwatch collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. Examples of such devices include the Apple Watch and Fitbit.

[0898] Device: The device temporarily stores the collected biometric and emotional data, which is then periodically sent to a server via Bluetooth or Wi-Fi.

[0899] Server: The server stores the received data in a database and analyzes the biometric information using an analysis program. Programming languages ​​such as Python and R can be used for the analysis. Specific libraries used include Pandas and NumPy. The analysis program identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[0900] Notification System

[0901] Server: Based on the analysis results, the server generates a notification message if an anomaly is detected. This message is sent to the user and designated parties. The message can be sent via email or short message service (SMS).

[0902] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. Notifications can be pop-ups or audio alerts, and can also send emails or SMS to relevant parties such as family members or doctors.

[0903] User: The user checks the alerts on the device and contacts a doctor if necessary. For example, this includes the ability to press an emergency contact button to talk directly to a doctor.

[0904] Support tools provided

[0905] User: When users need daily health management, they operate the device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[0906] Terminal: The terminal sends the user's request and emotional state information to the server.

[0907] Server: The server retrieves the corresponding support tools from the database and provides health guidelines and physical therapy videos tailored to the user's emotional state. Specifically, YouTube and Vimeo are used as healthcare video resources.

[0908] Device: The device displays the received support tool on the user's smartphone or tablet, for example, by playing a physical therapy video using a video playback app.

[0909] User: The user utilizes the displayed support tools and follows the physical therapy and health guidelines.

[0910] Reminders

[0911] User: The user sets medication times and exercise schedules on the device using a dedicated app.

[0912] Server: The server stores the configured schedule in a database and runs a program that generates reminders based on the schedule.

[0913] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time, for example, a pop-up notification or a vibration alert.

[0914] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[0915] Specific examples

[0916] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[0917] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[0918] 2. Device: Sends collected heart rate data and emotional state data to the server.

[0919] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[0920] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[0921] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[0922] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[0923] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[0924] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[0925] When inputting a prompt sentence into a generative AI model, the following prompt could be used, for example:

[0926] "Please provide a detailed description of the system that monitors the user's health and emotional state, notifies them appropriately when abnormalities are detected, and provides support tools and reminders. Please also include specific examples of the hardware and software used."

[0927] Using this prompt, the generative AI model can automatically generate the system details.

[0928] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0929] Step 1:

[0930] User: Wears a smartwatch or health management sensor, which collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. The specific input is the user's biometric information, and the output is the collected data.

[0931] Step 2:

[0932] Device: Temporarily stores the biometric information collected from the sensor. The input is the biometric information acquired in step 1, and the output is the data stored in the device's internal storage.

[0933] Step 3:

[0934] Device: The temporarily stored data is periodically sent to the server via Bluetooth or Wi-Fi. The input is the biometric information stored in the device's internal storage, and the output is the data sent to the server.

[0935] Step 4:

[0936] Server: Stores the received data in a database. The input is the biometric information sent from the device, and the output is the data stored in the database.

[0937] Step 5:

[0938] Server: Analyzes the biometric information using an analysis program. This uses Python or R libraries (e.g., Pandas or NumPy). The input is the biometric information stored in the database, and the output is the analysis results.

[0939] Step 6:

[0940] Server: Generates a notification message based on the analysis results if an anomaly is detected. The input is the analysis results, and the output is the generated notification message.

[0941] Step 7:

[0942] Server: Sends generated notification messages to users and designated parties. The input is the generated notification message and the output is the sent notification.

[0943] Step 8:

[0944] Terminal: Receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification received from the server, and the output is the alert displayed on the user's terminal.

[0945] Step 9:

[0946] User: Checks the alert and contacts a doctor if necessary. The input is the alert displayed on the user's device, and the output is the user's action (e.g., contacting a doctor).

[0947] Step 10:

[0948] User: Requests a support tool by operating a terminal. The input is the user's request, and the output is the request data generated by the terminal.

[0949] Step 11:

[0950] Terminal: Sends the user's request and emotion information to the server. The input is the user's request and emotion information, and the output is the request data sent to the server.

[0951] Step 12:

[0952] Server: Retrieves the appropriate support tool from the database and generates the content to be provided. The input is the request data sent to the server and the content of the database, and the output is the content of the generated support tool.

[0953] Step 13:

[0954] Terminal: The support tool is displayed on the user's smartphone or tablet. The input is the content of the support tool sent from the server, and the output is the support tool displayed on the user's terminal.

[0955] Step 14:

[0956] User: Uses the displayed support tools and acts in accordance with the health guidelines. The input is the support tools displayed on the terminal, and the output is the user's actions (e.g., performing physical therapy).

[0957] Step 15:

[0958] User: Sets medication times and exercise schedules on the device. The input is the schedule information set by the user, and the output is the schedule data saved on the device.

[0959] Step 16:

[0960] Server: Saves the set schedule in the database and runs the reminder generation program. The input is the schedule data sent from the terminal, and the output is the schedule saved in the database and the generated reminder notification.

[0961] Step 17:

[0962] Terminal: Receives reminder notifications from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server, and the output is the alarm and notification displayed on the user terminal.

[0963] Step 18:

[0964] User: Checks the reminder notification and acts according to the instructions. The input is the reminder notification displayed on the device, and the output is the user's action (e.g., taking medicine).

[0965] (Application example 2)

[0966] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0967] Conventional health monitoring systems are limited to collecting and analyzing users' biometric information, without taking into account their emotional state. This makes it difficult to respond appropriately in situations that are influenced by emotions such as stress or anxiety. In addition, notification systems that detect abnormalities only notify users of the abnormality and do not provide specific countermeasures or support. This has led to issues that do not adequately improve users' quality of life or ensure their safety.

[0968] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0969] In this invention, the server includes means for collecting biometric information and emotional information of the user, means for analyzing the collected data to determine abnormalities or emotional states, means for sending notifications to the user and designated parties when abnormalities or specific emotional states are determined, means for providing the user with physical therapy and health guidelines based on their emotional states, and means for sending reminders based on the user's schedule. This makes it possible to monitor the user's health state, respond quickly when abnormalities occur, and provide appropriate support tools according to their emotional states.

[0970] The "sensor means" is a device for collecting biometric and emotional information of a user.

[0971] The "terminal means" is a device for transmitting the biometric information and emotional information collected by the sensor means to the server.

[0972] The "analysis means" is a device or program for analyzing the bio-information and emotional information and determining abnormalities and emotional states.

[0973] The "notification means" is a device or program for sending a notification to the user and designated related parties when the analysis means determines an abnormality or a particular emotional state.

[0974] The "support tool providing means" is a device or program for providing physical therapy or health guidelines based on the user's emotional state.

[0975] A "reminder means" is a device or program for sending reminders based on a user's schedule.

[0976] "Biometric information" refers to data such as the user's heart rate, blood pressure, and body temperature.

[0977] "Emotional information" refers to the emotional state of the user that can be obtained from facial expressions, tone of voice, etc.

[0978] The present invention is a system that monitors a user's biological information and emotional state, promptly notifies the user if an abnormality is detected, and provides support tools according to the user's emotional state. Specific embodiments of the system are described below.

[0979] Overall system configuration

[0980] Data collection

[0981] Sensor means

[0982] The user wears a sensor that collects biometric information such as heart rate, blood pressure, and body temperature. This sensor can be a smartwatch or a dedicated health management sensor. In addition, a camera and microphone can be used to collect emotional information such as the user's facial expressions and tone of voice.

[0983] Terminal means

[0984] The data collected by the sensor means is sent to devices such as smartphones and tablets, allowing the user's biological and emotional information to be monitored in real time.

[0985] Data Analysis and Notification

[0986] server

[0987] The data sent from the device is stored on a server via the Internet. The server then analyzes the data using software such as Python, TensorFlow, and Keras. Specifically, it analyzes data such as heart rate, blood pressure, and body temperature to detect abnormal values. It also uses an emotion engine to determine the user's emotional state from facial expressions and tone of voice.

[0988] Notification means

[0989] If the server detects an abnormality or a specific emotional state, it will send a notification to the user and designated parties using Firebase Cloud Messaging, Twilio API, etc. The notification will be displayed as an alert on the user's device and will be sent to the parties via email or SMS.

[0990] Providing support tools

[0991] server

[0992] The server provides appropriate physical therapy and health guidelines based on the user's emotional state, including videos and guidelines created using React Native and Rest APIs. For example, if the user is in a high-stress state, a relaxation video will be provided.

[0993] Terminal

[0994] Support tools sent from the server are displayed on the user's smartphone or tablet, allowing the user to act in accordance with the presented physical therapy and health guidelines.

[0995] Reminders

[0996] server

[0997] Based on the schedule set by the user, the server generates reminders and notifies them at the specified times.

[0998] Terminal

[0999] Reminder notifications received from the server are displayed on the user's device, helping the user to remember to take their medicine or to exercise regularly.

[1000] Specific examples

[1001] For example, if a 65-year-old user wears a smartwatch and monitors their heart rate and blood pressure in real time, and the emotion engine detects an abnormally high heart rate and a high stress state, the system will operate as follows:

[1002] 1. Sensor means

[1003] The smartwatch detects abnormal heart rates and senses the user's facial expressions and tone of voice through the camera and microphone.

[1004] 2. Terminal means

[1005] The collected data is sent to a smartphone.

[1006] 3. Server

[1007] The data is analyzed to detect abnormal heart rates and high stress states.

[1008] 4. Means of notification

[1009] Based on the analysis results, notifications are sent to the user and relevant parties. An alert stating "An abnormal heart rate and high stress state have been detected" is displayed on the user's smartphone, and a similar message is sent to relevant parties via email.

[1010] 5. Means of providing support tools

[1011] A relaxing video is displayed on the user's smartphone.

[1012] 6. Reminders

[1013] When it's time for the user to take their medication, a notification will appear on their smartphone.

[1014] Prompt Sentence Examples

[1015] "If my heart rate exceeds 100, show me a stress relief video and send a notification message to my family."

[1016] In this way, the present invention monitors the user's health and emotional state, promptly notifies them in the event of an abnormality, and provides appropriate support, thereby creating an environment in which the user can live with peace of mind.

[1017] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1018] Step 1:

[1019] The user wears a sensor device (e.g., a smart watch). The sensor collects biometric information such as heart rate, blood pressure, and body temperature in real time, and also detects the user's facial expressions and tone of voice. The input is biometric and emotional information collected by the sensor.

[1020] Step 2:

[1021] The terminal means (e.g., a smartphone) temporarily stores the biometric and emotional information received from the sensor and periodically transmits it to a server via the Internet. The input is the biometric and emotional information sent from the sensor, and the output is the data sent to the server. Specifically, it receives data from the sensor using Bluetooth Low Energy (BLE), converts the data into JSON format, and transmits it to the server.

[1022] Step 3:

[1023] The server stores the biometric and emotional information received from the device in a database and performs data analysis using Python, TensorFlow, Keras, etc. The input is the biometric and emotional information sent from the device, and the output is the analysis results (abnormal values ​​and specific emotional states). Specifically, the server organizes the data as time-series data and analyzes it by applying anomaly detection algorithms and emotion recognition models.

[1024] Step 4:

[1025] If the server detects an anomaly or a specific emotional state, it uses Firebase Cloud Messaging or Twilio API to generate a notification and send it to the user and designated stakeholders. The input is the analysis result, and the output is the notification message. Specifically, it displays an alert on the user's smartphone and sends an email or SMS to the relevant stakeholders.

[1026] Step 5:

[1027] Based on the analysis results, the server provides physical therapy and health guidelines tailored to the user's emotional state. The input is the analysis results of biometric and emotional information, and the output is the proposed support tools (videos and guidelines). Specifically, it retrieves relevant videos and guidelines from the database and sends them to the user's device using React Native and a Rest API.

[1028] Step 6:

[1029] The device displays support tools (relaxation videos and health guidelines) received from the server to the user. The input is the support tools sent from the server, and the output is the content displayed on the device display. Specifically, the device displays the received videos and guidelines on the playback screen.

[1030] Step 7:

[1031] The server generates reminders based on the schedule set by the user and sends notifications to the device at the specified time. The input is the user's schedule information, and the output is the reminder notification. Specifically, it calculates the timing of the reminder notification based on the schedule information and sends the notification using Firebase Cloud Messaging.

[1032] Step 8:

[1033] The terminal displays the reminder notification received from the server to the user at the specified time. The input is the reminder notification from the server, and the output is the reminder displayed to the user. Specifically, the terminal generates an alarm and a pop-up notification at the specified time to notify the user.

[1034] In this way, the entire system's processing is coordinated at each step, enabling comprehensive monitoring of the user's health and emotional state and providing appropriate responses and support.

[1035] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1036] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1037] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1038] [Third embodiment]

[1039] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1040] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1041] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1042] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1043] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1044] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1045] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1046] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1047] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1048] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1049] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1050] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1051] This invention is a system for providing a safe and secure living environment for users at home. This system monitors the user's health status in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[1052] System programs and their processing

[1053] 1. Health monitoring

[1054] User: The user wears a smartwatch and health monitoring sensors on a daily basis, which collect biometric information such as heart rate, blood pressure, and body temperature in real time.

[1055] Device: The device periodically transmits the collected biometric information to the server. This device includes any communication-enabled device that the user normally uses, such as a smartphone or tablet.

[1056] Server: The server stores the received biometric information in a database and analyzes it using an analysis program, which includes algorithms that determine whether the heart rate and blood pressure are within normal ranges.

[1057] 2. Notification System

[1058] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties in the event of an emergency or high risk.

[1059] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[1060] User: The user checks the alert displayed on the device and contacts a doctor if necessary.

[1061] 3. Providing support tools

[1062] User: When a user needs to manage their daily health, they operate the device and request a support tool, such as watching a physical therapy video.

[1063] Terminal: The terminal starts the support tool providing program based on the user's request and displays the required content.

[1064] Server: The server retrieves the corresponding support tools (e.g., breathing exercise videos and exercise guidelines) from the database and sends them to the terminal.

[1065] User: Users can access support content at their own pace and practice physical therapy and health guidelines.

[1066] 4. Reminders

[1067] User: The user sets up a schedule for taking medicines and exercising on the device, enabling regular health management.

[1068] Server: The server runs a program that manages the user's schedule and generates reminders.

[1069] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[1070] User: The user follows the reminders to take their medicine and manage their health.

[1071] Specific examples

[1072] For example, assume that User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high, the procedure is as follows:

[1073] 1. User: The smartwatch detects an abnormal heart rate.

[1074] 2. Device: Sends collected heart rate data to the server.

[1075] 3. Server: Analyzes the data and detects anomalies.

[1076] 4. Server: Sends notifications about abnormal heart rate to User B and his family.

[1077] 5. Device: An alert is displayed on User B's smartphone and an email is sent to family members.

[1078] 6. User: Checks the alert and contacts the doctor.

[1079] 7. Support tool provided: The device will play a breathing exercise video to stabilize your heart rate.

[1080] 8. Reminder: User B sets the time to take the medicine, and the device notifies him / her at the specified time.

[1081] This system provides continuous monitoring, prompt notifications, and appropriate support tools to help users create an environment where they can live safely at home.

[1082] The processing flow will be explained below.

[1083] 1. Health monitoring

[1084] Step 1:

[1085] The user wears a smartwatch or health management sensor, which automatically collects biometric information such as heart rate, blood pressure, and body temperature in real time.

[1086] Step 2:

[1087] The device receives the collected biometric information on the user's smartphone or tablet and temporarily stores it.

[1088] Step 3:

[1089] The terminal transmits biometric information to the server at regular intervals.

[1090] Step 4:

[1091] The server stores the received biometric information in a database.

[1092] Step 5:

[1093] The server analyzes the biometric information and runs an analysis program to determine whether there are any abnormal values ​​for heart rate or blood pressure.

[1094] 2. Notification System

[1095] Step 6:

[1096] If the server detects an abnormality based on the analysis results, it generates a message to notify the user and designated parties.

[1097] Step 7:

[1098] The server sends the generated notification message to the user's terminal and the designated parties.

[1099] Step 8:

[1100] Based on the notification message received by the device, an alert is displayed on the user's smartphone or tablet.

[1101] Step 9:

[1102] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[1103] Step 10:

[1104] The user checks the alert on the device and contacts a doctor if necessary.

[1105] 3. Providing support tools

[1106] Step 11:

[1107] When a user needs to manage their daily health, they can operate the device to request a support tool, such as watching a video on breathing exercises.

[1108] Step 12:

[1109] The terminal sends the user's request to the server.

[1110] Step 13:

[1111] The server retrieves the corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database.

[1112] Step 14:

[1113] The server transmits the acquired support tool to the terminal.

[1114] Step 15:

[1115] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[1116] Step 16:

[1117] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[1118] 4. Reminders

[1119] Step 17:

[1120] The user operates the device to set medication times and exercise schedules.

[1121] Step 18:

[1122] The server saves the configured schedule in the database.

[1123] Step 19:

[1124] The server launches a program to generate reminders based on the saved schedule.

[1125] Step 20:

[1126] The server sends a reminder notification to the device at the scheduled time.

[1127] Step 21:

[1128] The device receives the reminder notification, displays a notification on the user's smartphone or tablet, and sounds an alarm.

[1129] Step 22:

[1130] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[1131] In this way, the system monitors the user's health at each step and provides appropriate notifications and support tools, creating an environment where the user can live safely at home.

[1132] Example 1

[1133] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1134] In modern society, home health management is crucial, especially for the elderly and those with chronic illnesses. However, achieving this requires complex health management systems that monitor vital signs in real time and respond quickly to any abnormalities. It also requires a means to support daily health management and provide appropriate advice and reminders. Conventional systems struggle to meet these requirements and often only provide partial solutions. Therefore, there is an urgent need to develop more sophisticated and comprehensive health management systems.

[1135] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1136] In this invention, the server includes an input means for collecting biometric information of a user, a communication means for transmitting the biometric information collected by the input means to a processing system, an analysis means for analyzing the biometric information to determine whether an abnormality is present, a notification means for sending a notification to the user and designated related parties if the analysis means determines an abnormality, a support means for providing the user with health management and health guidelines, a schedule support means for sending reminders based on the user's schedule, a message generation means for the notification means to send notifications to the user and related parties including specific abnormalities and recommended actions, and a reminder generation means for the schedule support means to generate periodic reminders based on a time set by the user. This enables comprehensive real-time health management, prompt notification of abnormalities, and support for improving quality of life.

[1137] A "user" is an individual who uses the system to manage their health.

[1138] "Biometric information" is data relating to the user's physical condition, such as heart rate, blood pressure, and body temperature.

[1139] "Input means" refers to a device such as a sensor that collects biometric information.

[1140] A "communication means" is a device or method that transmits biometric information collected by an input means to a processing system.

[1141] "Analysis means" refers to algorithms or software that analyze biometric information and determine whether or not there are any abnormalities.

[1142] The "notification means" is a function that sends a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[1143] "Support means" refers to services and tools that provide users with health management and health guidelines.

[1144] The "schedule support means" is a function that sends reminders based on the user's schedule.

[1145] The "message generating means" is an algorithm or software that generates a message including the abnormality details and recommended actions when the notification means sends a notification.

[1146] The "reminder generation means" is a function that generates periodic reminders based on a time set by the user.

[1147] "Processing system" refers to the entire system for receiving, managing, and analyzing biometric information and providing notifications and support based on the results.

[1148] This invention is a health management system that provides a safe and secure environment for users at home. This system monitors the user's biological information in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[1149] Health Status Monitoring

[1150] User: The user wears a smartwatch or health monitoring sensor to measure heart rate, blood pressure, and temperature. The sensor collects real-time biometric information on a daily basis.

[1151] Device: The device periodically transmits the biometric information acquired from the smartwatch to the server. This device is the user's smartphone or tablet. The smartphone or tablet transmits data to the server using Bluetooth, Wi-Fi, or mobile data communication.

[1152] Server: The server stores the received biometric data in a database and analyzes the data using analytical programs. For example, algorithms are used to determine whether heart rate or blood pressure is within normal ranges. The algorithms generate warnings if abnormal values ​​are detected.

[1153] Notification System

[1154] Server: If the server detects an abnormality based on the analysis results, it immediately generates a notification. The notification will include the specific abnormality and a recommended action, such as "Your heart rate is out of the normal range. Contact your doctor immediately."

[1155] Device: The device receives notifications from the server and displays an alert on the user's smartphone or tablet. At the same time, the server also sends notifications to relevant parties (e.g., family members or doctors) via email or SMS.

[1156] User: The user checks the alert displayed on the device and contacts a doctor if necessary. Notifications that can be viewed by the user allow for prompt and appropriate action to be taken.

[1157] Support tools provided

[1158] User: When a user needs health management, they operate their device to request support tools. For example, they may want to view physical therapy videos or breathing exercise guides.

[1159] Terminal: The terminal receives a user request, launches the support tool providing program, retrieves the necessary content from the server, and displays it.

[1160] Server: The server retrieves appropriate support tools (such as breathing exercise videos or exercise guidelines) from the database and sends them to the device.

[1161] Users: Users can use support content to practice physical therapy and health guidelines.

[1162] Reminders

[1163] User: The user sets the time to take medicine and exercise schedule on the device. For example, the user sets the time to take medicine at 9:00 every day using a dedicated app.

[1164] Server: The server manages the user's schedule and generates reminders at the specified time.

[1165] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[1166] User: Users can take their medicines according to the reminders and maintain a regular health regime.

[1167] Examples of prompt statements

[1168] Describe how the following system works: A user wears a smartwatch to monitor their heart rate. If the heart rate reaches an abnormal level, the server generates a notification, displays an alert on the user's smartphone, and provides a video with appropriate breathing exercises.

[1169] Through these functions, the system provides a safe and secure environment for users to live at home. It enables comprehensive and real-time health management, enables rapid response in the event of an abnormality, and provides tools to support daily health management.

[1170] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1171] Specific explanation of processing steps

[1172] Step 1: Collecting biometric information

[1173] User: The user wears a smartwatch or health monitoring sensors that measure heart rate, blood pressure, and body temperature. The input is real-time biometric information from these sensors. The output is the acquired biometric data: heart rate, blood pressure, and body temperature.

[1174] How it works: When a user wakes up in the morning and puts on the smartwatch, the sensors start collecting data on their heart rate and body temperature from that moment on.

[1175] Step 2: Sending biometric information

[1176] Terminal: The terminal periodically transmits biometric information acquired from the smartwatch to the server. The input is biometric information acquired from the smartwatch via Bluetooth connection. The output is biometric information transmitted to the server via Wi-Fi or mobile data communication.

[1177] How it works: The smartwatch connects to the smartphone via Bluetooth, and the smartphone sends data to the server using Wi-Fi.

[1178] Step 3: Analysis of biological information

[1179] Server: The server stores the received biometric information in a database and analyzes the data using an analysis program. The input is the biometric information received from the device. The output is the analysis result, judging whether it is within the normal range or abnormal.

[1180] Specific operation: The server analyzes the received heart rate data, calculates the average and standard deviation, and generates an alert if an abnormal value is detected.

[1181] Step 4: Generate a notification message

[1182] Server: The server immediately generates a notification message if an anomaly is detected based on the analysis results. The input is the anomaly judgment of the analysis results. The output is a notification message containing the specific anomaly content and recommended actions.

[1183] What happens: The server generates the message "Your heart rate is outside the normal range. Contact your doctor immediately."

[1184] Step 5: Sending notifications

[1185] Server: The server generates and sends notification messages to the user and the designated parties. The input is the generated notification message. The output is the notification message sent to the user's smartphone and the designated parties.

[1186] Specific operation: The server sends a notification to the user's smartphone and simultaneously notifies the user's family and doctor by email.

[1187] Step 6: Receive and confirm notifications

[1188] Terminal: The terminal receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification message sent from the server. The output is the alert displayed on the user's smartphone.

[1189] User: The user checks the alert displayed on the device and contacts a doctor if necessary. The input is the alert message displayed on the smartphone. The output is the user's response (e.g., contacting a doctor).

[1190] Specific operation: The user sees the "abnormal heart rate" notification on their smartphone screen and then uses the app to contact their doctor.

[1191] Step 7: Request Support Tools

[1192] User: The user requests the necessary support tools through the terminal. The input is an operation to request a physical therapy video or a breathing exercise guide. The output is the information sent in response to the request.

[1193] Specific operation: The user selects "Breathing exercise video" from the menu of the dedicated app.

[1194] Step 8: Providing support tools

[1195] Terminal: The terminal sends the user's request to the server and starts the support tool providing program. The input is the user's request information. The output is the start of the support tool providing program and the transmission of the request to the server.

[1196] Server: The server retrieves the corresponding support tool (such as a breathing exercise video or exercise guidelines) from the database and sends it to the device. The input is the user's request information. The output is the data of the support tool that was sent.

[1197] Specific operation: The server sends a link to a breathing exercise video to the device, and the device plays the video.

[1198] Step 9: Set reminders

[1199] User: The user sets up schedules such as medication times and exercise schedules on the device. The input is the schedule setting information in the dedicated app. The output is the set schedule information.

[1200] Specific operation: The user uses the calendar function of the dedicated app to schedule taking the medicine at 9:00 every day.

[1201] Step 10: Generate reminders

[1202] Server: The server manages the user's schedule and generates reminders at the specified time. The input is the user's schedule information. The output is the generated reminder notification.

[1203] What happens: The server generates a reminder notification every day at 9:00 AM based on the user's settings.

[1204] Step 11: Reminder Notifications

[1205] Terminal: The terminal receives reminders from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server. The output is the reminder displayed on the terminal.

[1206] User: The user takes medicine according to the reminder and manages their health. The input is the reminder notification displayed on the device. The output is the user's coping behavior (e.g., taking medicine).

[1207] Specific operation: At 9 o'clock, the user's smartphone will sound an alarm and notify them that it's time for their medicine, and they will then take their medicine.

[1208] (Application example 1)

[1209] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1210] For elderly people and those with chronic illnesses, constant monitoring of their health status and prompt response when an abnormality occurs are extremely important for them to continue living safely at home. However, conventional systems lack the functionality to monitor users' health status in real time or an immediate notification system in case of an emergency, making it difficult to fully ensure user safety. Furthermore, the provision of support tools for daily health management and improving quality of life is also insufficient.

[1211] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1212] In this invention, the server includes a signal generating means for generating an output signal when an abnormality is detected based on the user's biological information, a support tool providing means, and a schedule management means. This makes it possible to monitor the user's health condition in real time and quickly detect and notify abnormalities. It also provides various functions to support daily health management and can support the user in improving their lifestyle habits through reminders.

[1213] The "sensor means" is a device for collecting biometric information of a user.

[1214] The "terminal means" is a communication device for transmitting the biometric information collected by the sensor means to the server.

[1215] The "analysis means" is a system that analyzes the biometric information sent to the server and determines whether or not there is an abnormality.

[1216] The "notification means" is a function for sending a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[1217] The "support tool providing means" is a function for providing physical therapy and physical guidelines to the user.

[1218] The "schedule management means" is a function for sending reminders based on the user's schedule.

[1219] The "signal generating means" is a device or system that generates an output signal when an abnormality is detected based on the user's biological information.

[1220] "Electronic communication" means a means of communication to send notices to designated parties via email or SMS.

[1221] An "alert" is a warning message that is displayed on the user's terminal when an abnormality occurs.

[1222] 1. System Configuration

[1223] The present invention provides a system that monitors the user's health condition in real time and notifies the user promptly if an abnormality occurs, thereby providing an environment in which the user can live safely at home. Specific embodiments will be described below.

[1224] 2. Hardware and Software Used

[1225] Hardware:

[1226] Smartphone (iOS / Android)

[1227] Smartwatches (Apple Watch, Fitbit, etc.)

[1228] Health management sensors (heart rate sensors, blood pressure monitors, etc.)

[1229] software:

[1230] Mobile application (cross-platform using Flutter)

[1231] Server (using Amazon Web Services)

[1232] Database (using Amazon RDS)

[1233] Notification system (Firebase Cloud Messaging)

[1234] 3. Operation of various means

[1235] Sensor means

[1236] Users wear smartwatches and health monitoring sensors that collect real-time biometric information such as heart rate, blood pressure, and body temperature.

[1237] Terminal means

[1238] Smartphones and tablets function as terminals and send collected biometric information to Amazon Web Services (AWS).

[1239] Analysis means

[1240] The server receives the vital signs and stores them in Amazon RDS, then uses an analysis program (using a generative AI model) to determine whether the heart rate and blood pressure are within normal ranges.

[1241] Notification means

[1242] If an abnormality is detected as a result of the analysis, an alert is sent to the user's smartphone using Firebase Cloud Messaging. At the same time, pre-registered contacts (family members and doctors) are also notified via SMS and email.

[1243] Support tool provision method

[1244] When a user requests a physical therapy video or health guidelines, the server retrieves the corresponding content from Amazon RDS and provides it to the user through a mobile application.

[1245] Schedule management tools

[1246] Users set their medication and exercise schedules in the mobile app. The server manages this schedule and generates reminders at the specified times. The mobile app then notifies the user of these reminders on their smartphone.

[1247] Signal generation means

[1248] When an abnormality is detected based on the user's biometric information, the server generates an output signal that displays an alert on the user's device and sends an electronic communication to designated parties.

[1249] 4. Specific Examples

[1250] For example, if User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring, the behavior is as follows:

[1251] 1. The smartwatch detects an abnormal heart rate.

[1252] 2. The collected heart rate data is sent to the AWS server via a smartphone app.

[1253] 3. The server analyzes the heart rate data and detects abnormalities.

[1254] 4. Use Firebase Cloud Messaging to send an alert to User B's smartphone.

[1255] 5. An abnormality notification is sent to User B's family and doctor via email or SMS.

[1256] 6. Play a breathing exercise video on your smartphone to help stabilize your heart rate.

[1257] 7. Remind me when it's time to take my medicine.

[1258] 5. Examples of prompts

[1259] "A 70-year-old man is wearing a smartwatch for monitoring his heart rate. If his heart rate becomes abnormally high, the AWS server will analyze it and send a notification to his smartphone using Firebase Cloud Messaging. The system will also notify his family of the abnormality via email and SMS, play a video on breathing exercises to stabilize his heart rate on the smartphone, and remind him to take his medication. Please generate appropriate prompts taking these situations into consideration."

[1260] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1261] Step 1:

[1262] The user wears a smartwatch or health management sensor.

[1263] Input: User's biometric information (heart rate, blood pressure, temperature, etc.)

[1264] Data processing: Sensors collect data in real time

[1265] Output: Biometric data

[1266] Step 2:

[1267] The terminal transmits the collected biometric information to the server.

[1268] Input: Biometric data obtained from sensors

[1269] Data processing: Converting data into packets and sending them to a server via the Internet

[1270] Output: Biometric data transmitted to the server

[1271] Step 3:

[1272] The server receives the biometric information and stores it in a database.

[1273] Input: Transmitted biometric data

[1274] Data processing: storing data in a suitable format for the database

[1275] Output: Stored biometric data

[1276] Step 4:

[1277] The server analyzes the stored biometric information and detects any abnormalities.

[1278] Input: Stored biometric data

[1279] Data calculation: Analyze using generative AI models and compare with normal ranges

[1280] Output: Analysis result (normal / abnormal)

[1281] Step 5:

[1282] Generates a notification message if the server detects an abnormality.

[1283] Input: Analysis results (when an abnormality is detected)

[1284] Data processing: Generate notification messages about anomalies

[1285] Output: Information message

[1286] Step 6:

[1287] The server uses Firebase Cloud Messaging to send notifications to the user device.

[1288] Input: Notification message

[1289] Data processing: Encode the message in FMC protocol and send it

[1290] Output: An alert displayed on the user's terminal

[1291] Step 7:

[1292] At the same time, the server sends a notification to pre-registered contacts.

[1293] Input: Notification message, registered contact information

[1294] Data processing: Convert to email or SMS format and send to contacts

[1295] Output: Email or SMS notification sent to your contacts

[1296] Step 8:

[1297] The user terminal provides the support tool in response to a request.

[1298] Input: User request, analysis results

[1299] Data processing: Retrieve and play corresponding video and guideline content

[1300] Output: Supporting content (e.g., video) displayed on the user's device

[1301] Step 9:

[1302] The server generates and sends reminders based on a schedule set by the user.

[1303] Input: User's schedule information

[1304] Data processing: Generate and send reminders at specified times

[1305] Output: Reminder notification displayed on the user's device

[1306] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1307] This invention is a system that provides a safe and secure environment for users at home. By combining it with an emotion engine that recognizes the user's emotions, it provides advanced health management and support. This system monitors the user's health status in real time, promptly notifies the user when an abnormality occurs, and encourages appropriate action. It also provides physical therapy and health guidelines based on the user's emotional state, and reminders to improve quality of life.

[1308] System programs and their processing

[1309] Health Monitoring and Emotion Engine

[1310] User: Users wear smartwatches and health monitoring sensors on a daily basis, which collect real-time data such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[1311] Device: The device temporarily stores biometric and emotional state data collected from sensors and periodically transmits it to the server.

[1312] Server: The server stores the received biometric and emotional state data in a database and analyzes it using an analysis program that identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1313] Notification System

[1314] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties when an emergency call is required or there is a high risk.

[1315] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[1316] User: The user checks the alert on the device and contacts a doctor if necessary.

[1317] Support tools provided

[1318] User: When users need daily health management, they operate their device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[1319] Terminal: The terminal sends the user's request and emotional state information to the server.

[1320] Server: The server retrieves corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database and provides them tailored to the user's emotional state.

[1321] Terminal: The terminal displays the received support tool on the user's smartphone or tablet.

[1322] User: The user utilizes the displayed support tools and acts in accordance with the physical therapy and health guidelines.

[1323] Reminders

[1324] User: The user sets medication times and exercise schedules on the device.

[1325] Server: The server runs a program that stores the configured schedule in a database and generates reminders based on the schedule.

[1326] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time.

[1327] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[1328] Specific examples

[1329] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[1330] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[1331] 2. Device: Sends collected heart rate data and emotional state data to the server.

[1332] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[1333] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[1334] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[1335] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[1336] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[1337] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[1338] This system provides comprehensive monitoring, including emotional state, prompt notification, and emotional support tools to create an environment where users can live safely at home.

[1339] The processing flow will be explained below.

[1340] Health Monitoring and Emotion Engine

[1341] Step 1:

[1342] Users wear smartwatches and health management sensors that automatically collect data in real time, such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[1343] Step 2:

[1344] The device receives the collected biometric and emotional state data on the user's smartphone or tablet and temporarily stores it.

[1345] Step 3:

[1346] The terminal periodically transmits the collected data to the server.

[1347] Step 4:

[1348] The server stores the received biometric information and emotional state data in a database.

[1349] Step 5:

[1350] The server analyzes biometric information to identify abnormal values ​​such as heart rate and blood pressure, and also uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1351] Notification System

[1352] Step 6:

[1353] If the server detects an anomaly based on the analysis, it generates a notification message containing details about the user's health and emotional state.

[1354] Step 7:

[1355] The server generates and sends the notification message to the user's device and the designated parties, for example via email or SMS.

[1356] Step 8:

[1357] The notification message received by the device is displayed as an alert on the user's smartphone or tablet.

[1358] Step 9:

[1359] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[1360] Step 10:

[1361] The user can check the alerts on the device, contact a doctor if necessary, and receive relaxation techniques and support based on their emotional state.

[1362] Support tools provided

[1363] Step 11:

[1364] The user operates the device to request a support tool, for example, to watch a video on relaxation techniques.

[1365] Step 12:

[1366] The terminal transmits the user's request and emotional state information to the server.

[1367] Step 13:

[1368] The server retrieves the corresponding support tools (e.g., physical therapy videos or exercise guidelines) from the database and selects the appropriate content based on the emotion engine.

[1369] Step 14:

[1370] The server transmits the acquired support tool to the terminal.

[1371] Step 15:

[1372] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[1373] Step 16:

[1374] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[1375] Reminders

[1376] Step 17:

[1377] The user operates the device to set medication times and exercise schedules.

[1378] Step 18:

[1379] The server saves the configured schedule in the database.

[1380] Step 19:

[1381] The server launches a program to generate reminders based on the saved schedule.

[1382] Step 20:

[1383] The server sends a reminder notification to the device at the scheduled time.

[1384] Step 21:

[1385] Your device will receive reminder notifications and display alarms and notifications at the specified times.

[1386] Step 22:

[1387] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[1388] Through these steps, the system provides a safe and secure environment for users to live at home. Furthermore, by integrating an emotion engine, the system improves the user's quality of life by providing physical therapy and support tools based on the user's emotional state.

[1389] Example 2

[1390] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] The main problem that this invention aims to solve is to effectively manage a user's health and monitor their emotional state in their daily lives, and provide prompt and appropriate responses and support. Conventional health management systems lack the ability to detect abnormal conditions and provide emotional support, which can lead to a decline in the user's health and quality of life. In response, this invention aims to realize an environment in which users can live with peace of mind by comprehensively monitoring a user's biometric information and emotional state and providing appropriate notifications and support.

[1392] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a measurement means for collecting biometric information of the user, a communication means for transmitting the biometric information collected by the measurement means to the server, a diagnosis means for analyzing the biometric information and determining whether or not there is an abnormality, a warning means for sending a notification to the user and designated relevant parties when the diagnosis means determines an abnormality, a support means for providing the user with physical therapy and health guidelines, and a suggestion means for sending reminders based on the user's schedule. This makes it possible to comprehensively manage the user's health and emotional state in real time and provide prompt notification and appropriate support when an abnormality is detected.

[1393] "Measurement means" refers to a device or function that collects biometric information from a user.

[1394] "Communication means" refers to a device or function for transmitting the biometric information collected by the measurement means to a server.

[1395] "Diagnostic means" refers to a device or function that analyzes collected biological information to determine whether or not there is an abnormality.

[1396] "Alert means" refers to a device or function that sends a notification to the user and designated parties when the diagnostic means determines an abnormality.

[1397] "Assistive device" refers to a device or function that provides physical therapy or health guidelines to a user.

[1398] "Suggestion means" refers to a device or function that sends reminders based on the user's schedule.

[1399] "Reminder" refers to an alert or message that notifies a user of a specific action or schedule.

[1400] "Biometric information" refers to data such as a user's heart rate, blood pressure, body temperature, facial expression, and tone of voice.

[1401] "Abnormal" refers to a state in which the user's biological information deviates from a preset normal range.

[1402] "Emotional state" refers to the psychological and emotional state of a user as determined by their facial expression and tone of voice.

[1403] "Support tools" refers to resources such as physical therapy and health guidelines to help users manage their health.

[1404] This invention is a system for providing a user with a safe and secure living environment at home, monitoring the user's health and emotional state in real time, notifying them promptly when an abnormality occurs, and encouraging them to take appropriate action. This system consists of the following components:

[1405] Health Monitoring and Emotion Engine

[1406] User: The user wears a smartwatch or health management sensor on a daily basis. The smartwatch collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. Examples of such devices include the Apple Watch and Fitbit.

[1407] Device: The device temporarily stores the collected biometric and emotional data, which is then periodically sent to a server via Bluetooth or Wi-Fi.

[1408] Server: The server stores the received data in a database and analyzes the biometric information using an analysis program. Programming languages ​​such as Python and R can be used for the analysis. Specific libraries used include Pandas and NumPy. The analysis program identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1409] Notification System

[1410] Server: Based on the analysis results, the server generates a notification message if an anomaly is detected. This message is sent to the user and designated parties. The message can be sent via email or short message service (SMS).

[1411] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. Notifications can be pop-ups or audio alerts, and can also send emails or SMS to relevant parties such as family members or doctors.

[1412] User: The user checks the alerts on the device and contacts a doctor if necessary. For example, this includes the ability to press an emergency contact button to talk directly to a doctor.

[1413] Support tools provided

[1414] User: When users need daily health management, they operate the device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[1415] Terminal: The terminal sends the user's request and emotional state information to the server.

[1416] Server: The server retrieves the corresponding support tools from the database and provides health guidelines and physical therapy videos tailored to the user's emotional state. Specifically, YouTube and Vimeo are used as healthcare video resources.

[1417] Device: The device displays the received support tool on the user's smartphone or tablet, for example, by playing a physical therapy video using a video playback app.

[1418] User: The user utilizes the displayed support tools and follows the physical therapy and health guidelines.

[1419] Reminders

[1420] User: The user sets medication times and exercise schedules on the device using a dedicated app.

[1421] Server: The server stores the configured schedule in a database and runs a program that generates reminders based on the schedule.

[1422] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time, for example, a pop-up notification or a vibration alert.

[1423] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[1424] Specific examples

[1425] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[1426] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[1427] 2. Device: Sends collected heart rate data and emotional state data to the server.

[1428] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[1429] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[1430] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[1431] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[1432] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[1433] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[1434] When inputting a prompt sentence into a generative AI model, the following prompt could be used, for example:

[1435] "Please provide a detailed description of the system that monitors the user's health and emotional state, notifies them appropriately when abnormalities are detected, and provides support tools and reminders. Please also include specific examples of the hardware and software used."

[1436] Using this prompt, the generative AI model can automatically generate the system details.

[1437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1438] Step 1:

[1439] User: Wears a smartwatch or health management sensor, which collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. The specific input is the user's biometric information, and the output is the collected data.

[1440] Step 2:

[1441] Device: Temporarily stores the biometric information collected from the sensor. The input is the biometric information acquired in step 1, and the output is the data stored in the device's internal storage.

[1442] Step 3:

[1443] Device: The temporarily stored data is periodically sent to the server via Bluetooth or Wi-Fi. The input is the biometric information stored in the device's internal storage, and the output is the data sent to the server.

[1444] Step 4:

[1445] Server: Stores the received data in a database. The input is the biometric information sent from the device, and the output is the data stored in the database.

[1446] Step 5:

[1447] Server: Analyzes the biometric information using an analysis program. This uses Python or R libraries (e.g., Pandas or NumPy). The input is the biometric information stored in the database, and the output is the analysis results.

[1448] Step 6:

[1449] Server: Generates a notification message based on the analysis results if an anomaly is detected. The input is the analysis results, and the output is the generated notification message.

[1450] Step 7:

[1451] Server: Sends generated notification messages to users and designated parties. The input is the generated notification message and the output is the sent notification.

[1452] Step 8:

[1453] Terminal: Receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification received from the server, and the output is the alert displayed on the user's terminal.

[1454] Step 9:

[1455] User: Checks the alert and contacts a doctor if necessary. The input is the alert displayed on the user's device, and the output is the user's action (e.g., contacting a doctor).

[1456] Step 10:

[1457] User: Requests a support tool by operating a terminal. The input is the user's request, and the output is the request data generated by the terminal.

[1458] Step 11:

[1459] Terminal: Sends the user's request and emotion information to the server. The input is the user's request and emotion information, and the output is the request data sent to the server.

[1460] Step 12:

[1461] Server: Retrieves the appropriate support tool from the database and generates the content to be provided. The input is the request data sent to the server and the content of the database, and the output is the content of the generated support tool.

[1462] Step 13:

[1463] Terminal: The support tool is displayed on the user's smartphone or tablet. The input is the content of the support tool sent from the server, and the output is the support tool displayed on the user's terminal.

[1464] Step 14:

[1465] User: Uses the displayed support tools and acts in accordance with the health guidelines. The input is the support tools displayed on the terminal, and the output is the user's actions (e.g., performing physical therapy).

[1466] Step 15:

[1467] User: Sets medication times and exercise schedules on the device. The input is the schedule information set by the user, and the output is the schedule data saved on the device.

[1468] Step 16:

[1469] Server: Saves the set schedule in the database and runs the reminder generation program. The input is the schedule data sent from the terminal, and the output is the schedule saved in the database and the generated reminder notification.

[1470] Step 17:

[1471] Terminal: Receives reminder notifications from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server, and the output is the alarm and notification displayed on the user terminal.

[1472] Step 18:

[1473] User: Checks the reminder notification and acts according to the instructions. The input is the reminder notification displayed on the device, and the output is the user's action (e.g., taking medicine).

[1474] (Application example 2)

[1475] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1476] Conventional health monitoring systems are limited to collecting and analyzing users' biometric information, without taking into account their emotional state. This makes it difficult to respond appropriately in situations that are influenced by emotions such as stress or anxiety. In addition, notification systems that detect abnormalities only notify users of the abnormality and do not provide specific countermeasures or support. This has led to issues that do not adequately improve users' quality of life or ensure their safety.

[1477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1478] In this invention, the server includes means for collecting biometric information and emotional information of the user, means for analyzing the collected data to determine abnormalities or emotional states, means for sending notifications to the user and designated parties when abnormalities or specific emotional states are determined, means for providing the user with physical therapy and health guidelines based on their emotional states, and means for sending reminders based on the user's schedule. This makes it possible to monitor the user's health state, respond quickly when abnormalities occur, and provide appropriate support tools according to their emotional states.

[1479] The "sensor means" is a device for collecting biometric and emotional information of a user.

[1480] The "terminal means" is a device for transmitting the biometric information and emotional information collected by the sensor means to the server.

[1481] The "analysis means" is a device or program for analyzing the bio-information and emotional information and determining abnormalities and emotional states.

[1482] The "notification means" is a device or program for sending a notification to the user and designated related parties when the analysis means determines an abnormality or a particular emotional state.

[1483] The "support tool providing means" is a device or program for providing physical therapy or health guidelines based on the user's emotional state.

[1484] A "reminder means" is a device or program for sending reminders based on a user's schedule.

[1485] "Biometric information" refers to data such as the user's heart rate, blood pressure, and body temperature.

[1486] "Emotional information" refers to the emotional state of the user that can be obtained from facial expressions, tone of voice, etc.

[1487] The present invention is a system that monitors a user's biological information and emotional state, promptly notifies the user if an abnormality is detected, and provides support tools according to the user's emotional state. Specific embodiments of the system are described below.

[1488] Overall system configuration

[1489] Data collection

[1490] Sensor means

[1491] The user wears a sensor that collects biometric information such as heart rate, blood pressure, and body temperature. This sensor can be a smartwatch or a dedicated health management sensor. In addition, a camera and microphone can be used to collect emotional information such as the user's facial expressions and tone of voice.

[1492] Terminal means

[1493] The data collected by the sensor means is sent to devices such as smartphones and tablets, allowing the user's biological and emotional information to be monitored in real time.

[1494] Data Analysis and Notification

[1495] server

[1496] The data sent from the device is stored on a server via the Internet. The server then analyzes the data using software such as Python, TensorFlow, and Keras. Specifically, it analyzes data such as heart rate, blood pressure, and body temperature to detect abnormal values. It also uses an emotion engine to determine the user's emotional state from facial expressions and tone of voice.

[1497] Notification means

[1498] If the server detects an abnormality or a specific emotional state, it will send a notification to the user and designated parties using Firebase Cloud Messaging, Twilio API, etc. The notification will be displayed as an alert on the user's device and will be sent to the parties via email or SMS.

[1499] Providing support tools

[1500] server

[1501] The server provides appropriate physical therapy and health guidelines based on the user's emotional state, including videos and guidelines created using React Native and Rest APIs. For example, if the user is in a high-stress state, a relaxation video will be provided.

[1502] Terminal

[1503] Support tools sent from the server are displayed on the user's smartphone or tablet, allowing the user to act in accordance with the presented physical therapy and health guidelines.

[1504] Reminders

[1505] server

[1506] Based on the schedule set by the user, the server generates reminders and notifies them at the specified times.

[1507] Terminal

[1508] Reminder notifications received from the server are displayed on the user's device, helping the user to remember to take their medicine or to exercise regularly.

[1509] Specific examples

[1510] For example, if a 65-year-old user wears a smartwatch and monitors their heart rate and blood pressure in real time, and the emotion engine detects an abnormally high heart rate and a high stress state, the system will operate as follows:

[1511] 1. Sensor means

[1512] The smartwatch detects abnormal heart rates and senses the user's facial expressions and tone of voice through the camera and microphone.

[1513] 2. Terminal means

[1514] The collected data is sent to a smartphone.

[1515] 3. Server

[1516] The data is analyzed to detect abnormal heart rates and high stress states.

[1517] 4. Means of notification

[1518] Based on the analysis results, notifications are sent to the user and relevant parties. An alert stating "An abnormal heart rate and high stress state have been detected" is displayed on the user's smartphone, and a similar message is sent to relevant parties via email.

[1519] 5. Means of providing support tools

[1520] A relaxing video is displayed on the user's smartphone.

[1521] 6. Reminders

[1522] When it's time for the user to take their medication, a notification will appear on their smartphone.

[1523] Prompt Sentence Examples

[1524] "If my heart rate exceeds 100, show me a stress relief video and send a notification message to my family."

[1525] In this way, the present invention monitors the user's health and emotional state, promptly notifies them in the event of an abnormality, and provides appropriate support, thereby creating an environment in which the user can live with peace of mind.

[1526] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1527] Step 1:

[1528] The user wears a sensor device (e.g., a smart watch). The sensor collects biometric information such as heart rate, blood pressure, and body temperature in real time, and also detects the user's facial expressions and tone of voice. The input is biometric and emotional information collected by the sensor.

[1529] Step 2:

[1530] The terminal means (e.g., a smartphone) temporarily stores the biometric and emotional information received from the sensor and periodically transmits it to a server via the Internet. The input is the biometric and emotional information sent from the sensor, and the output is the data sent to the server. Specifically, it receives data from the sensor using Bluetooth Low Energy (BLE), converts the data into JSON format, and transmits it to the server.

[1531] Step 3:

[1532] The server stores the biometric and emotional information received from the device in a database and performs data analysis using Python, TensorFlow, Keras, etc. The input is the biometric and emotional information sent from the device, and the output is the analysis results (abnormal values ​​and specific emotional states). Specifically, the server organizes the data as time-series data and analyzes it by applying anomaly detection algorithms and emotion recognition models.

[1533] Step 4:

[1534] If the server detects an anomaly or a specific emotional state, it uses Firebase Cloud Messaging or Twilio API to generate a notification and send it to the user and designated stakeholders. The input is the analysis result, and the output is the notification message. Specifically, it displays an alert on the user's smartphone and sends an email or SMS to the relevant stakeholders.

[1535] Step 5:

[1536] Based on the analysis results, the server provides physical therapy and health guidelines tailored to the user's emotional state. The input is the analysis results of biometric and emotional information, and the output is the proposed support tools (videos and guidelines). Specifically, it retrieves relevant videos and guidelines from the database and sends them to the user's device using React Native and a Rest API.

[1537] Step 6:

[1538] The device displays support tools (relaxation videos and health guidelines) received from the server to the user. The input is the support tools sent from the server, and the output is the content displayed on the device display. Specifically, the device displays the received videos and guidelines on the playback screen.

[1539] Step 7:

[1540] The server generates reminders based on the schedule set by the user and sends notifications to the device at the specified time. The input is the user's schedule information, and the output is the reminder notification. Specifically, it calculates the timing of the reminder notification based on the schedule information and sends the notification using Firebase Cloud Messaging.

[1541] Step 8:

[1542] The terminal displays the reminder notification received from the server to the user at the specified time. The input is the reminder notification from the server, and the output is the reminder displayed to the user. Specifically, the terminal generates an alarm and a pop-up notification at the specified time to notify the user.

[1543] In this way, the entire system's processing is coordinated at each step, enabling comprehensive monitoring of the user's health and emotional state and providing appropriate responses and support.

[1544] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1545] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1546] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1547] [Fourth embodiment]

[1548] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1549] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1550] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1551] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1552] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1553] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1554] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1555] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1556] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1557] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1558] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1559] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1560] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1561] This invention is a system for providing a safe and secure living environment for users at home. This system monitors the user's health status in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[1562] System programs and their processing

[1563] 1. Health monitoring

[1564] User: The user wears a smartwatch and health monitoring sensors on a daily basis, which collect biometric information such as heart rate, blood pressure, and body temperature in real time.

[1565] Device: The device periodically transmits the collected biometric information to the server. This device includes any communication-enabled device that the user normally uses, such as a smartphone or tablet.

[1566] Server: The server stores the received biometric information in a database and analyzes it using an analysis program, which includes algorithms that determine whether the heart rate and blood pressure are within normal ranges.

[1567] 2. Notification System

[1568] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties in the event of an emergency or high risk.

[1569] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[1570] User: The user checks the alert displayed on the device and contacts a doctor if necessary.

[1571] 3. Providing support tools

[1572] User: When a user needs to manage their daily health, they operate the device and request a support tool, such as watching a physical therapy video.

[1573] Terminal: The terminal starts the support tool providing program based on the user's request and displays the required content.

[1574] Server: The server retrieves the corresponding support tools (e.g., breathing exercise videos and exercise guidelines) from the database and sends them to the terminal.

[1575] User: Users can access support content at their own pace and practice physical therapy and health guidelines.

[1576] 4. Reminders

[1577] User: The user sets up a schedule for taking medicines and exercising on the device, enabling regular health management.

[1578] Server: The server runs a program that manages the user's schedule and generates reminders.

[1579] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[1580] User: The user follows the reminders to take their medicine and manage their health.

[1581] Specific examples

[1582] For example, assume that User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high, the procedure is as follows:

[1583] 1. User: The smartwatch detects an abnormal heart rate.

[1584] 2. Device: Sends collected heart rate data to the server.

[1585] 3. Server: Analyzes the data and detects anomalies.

[1586] 4. Server: Sends notifications about abnormal heart rate to User B and his family.

[1587] 5. Device: An alert is displayed on User B's smartphone and an email is sent to family members.

[1588] 6. User: Checks the alert and contacts the doctor.

[1589] 7. Support tool provided: The device will play a breathing exercise video to stabilize your heart rate.

[1590] 8. Reminder: User B sets the time to take the medicine, and the device notifies him / her at the specified time.

[1591] This system provides continuous monitoring, prompt notifications, and appropriate support tools to help users create an environment where they can live safely at home.

[1592] The processing flow will be explained below.

[1593] 1. Health monitoring

[1594] Step 1:

[1595] The user wears a smartwatch or health management sensor, which automatically collects biometric information such as heart rate, blood pressure, and body temperature in real time.

[1596] Step 2:

[1597] The device receives the collected biometric information on the user's smartphone or tablet and temporarily stores it.

[1598] Step 3:

[1599] The terminal transmits biometric information to the server at regular intervals.

[1600] Step 4:

[1601] The server stores the received biometric information in a database.

[1602] Step 5:

[1603] The server analyzes the biometric information and runs an analysis program to determine whether there are any abnormal values ​​for heart rate or blood pressure.

[1604] 2. Notification System

[1605] Step 6:

[1606] If the server detects an abnormality based on the analysis results, it generates a message to notify the user and designated parties.

[1607] Step 7:

[1608] The server sends the generated notification message to the user's terminal and the designated parties.

[1609] Step 8:

[1610] Based on the notification message received by the device, an alert is displayed on the user's smartphone or tablet.

[1611] Step 9:

[1612] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[1613] Step 10:

[1614] The user checks the alert on the device and contacts a doctor if necessary.

[1615] 3. Providing support tools

[1616] Step 11:

[1617] When a user needs to manage their daily health, they can operate the device to request a support tool, such as watching a video on breathing exercises.

[1618] Step 12:

[1619] The terminal sends the user's request to the server.

[1620] Step 13:

[1621] The server retrieves the corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database.

[1622] Step 14:

[1623] The server transmits the acquired support tool to the terminal.

[1624] Step 15:

[1625] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[1626] Step 16:

[1627] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[1628] 4. Reminders

[1629] Step 17:

[1630] The user operates the device to set medication times and exercise schedules.

[1631] Step 18:

[1632] The server saves the configured schedule in the database.

[1633] Step 19:

[1634] The server launches a program to generate reminders based on the saved schedule.

[1635] Step 20:

[1636] The server sends a reminder notification to the device at the scheduled time.

[1637] Step 21:

[1638] The device receives the reminder notification, displays a notification on the user's smartphone or tablet, and sounds an alarm.

[1639] Step 22:

[1640] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[1641] In this way, the system monitors the user's health at each step and provides appropriate notifications and support tools, creating an environment where the user can live safely at home.

[1642] Example 1

[1643] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1644] In modern society, home health management is crucial, especially for the elderly and those with chronic illnesses. However, achieving this requires complex health management systems that monitor vital signs in real time and respond quickly to any abnormalities. It also requires a means to support daily health management and provide appropriate advice and reminders. Conventional systems struggle to meet these requirements and often only provide partial solutions. Therefore, there is an urgent need to develop more sophisticated and comprehensive health management systems.

[1645] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1646] In this invention, the server includes an input means for collecting biometric information of a user, a communication means for transmitting the biometric information collected by the input means to a processing system, an analysis means for analyzing the biometric information to determine whether an abnormality is present, a notification means for sending a notification to the user and designated related parties if the analysis means determines an abnormality, a support means for providing the user with health management and health guidelines, a schedule support means for sending reminders based on the user's schedule, a message generation means for the notification means to send notifications to the user and related parties including specific abnormalities and recommended actions, and a reminder generation means for the schedule support means to generate periodic reminders based on a time set by the user. This enables comprehensive real-time health management, prompt notification of abnormalities, and support for improving quality of life.

[1647] A "user" is an individual who uses the system to manage their health.

[1648] "Biometric information" is data relating to the user's physical condition, such as heart rate, blood pressure, and body temperature.

[1649] "Input means" refers to a device such as a sensor that collects biometric information.

[1650] A "communication means" is a device or method that transmits biometric information collected by an input means to a processing system.

[1651] "Analysis means" refers to algorithms or software that analyze biometric information and determine whether or not there are any abnormalities.

[1652] The "notification means" is a function that sends a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[1653] "Support means" refers to services and tools that provide users with health management and health guidelines.

[1654] The "schedule support means" is a function that sends reminders based on the user's schedule.

[1655] The "message generating means" is an algorithm or software that generates a message including the abnormality details and recommended actions when the notification means sends a notification.

[1656] The "reminder generation means" is a function that generates periodic reminders based on a time set by the user.

[1657] "Processing system" refers to the entire system for receiving, managing, and analyzing biometric information and providing notifications and support based on the results.

[1658] This invention is a health management system that provides a safe and secure environment for users at home. This system monitors the user's biological information in real time and promptly notifies the user when an abnormality occurs, encouraging appropriate action. It also provides support tools and reminders for daily health management and improving quality of life.

[1659] Health Status Monitoring

[1660] User: The user wears a smartwatch or health monitoring sensor to measure heart rate, blood pressure, and temperature. The sensor collects real-time biometric information on a daily basis.

[1661] Device: The device periodically transmits the biometric information acquired from the smartwatch to the server. This device is the user's smartphone or tablet. The smartphone or tablet transmits data to the server using Bluetooth, Wi-Fi, or mobile data communication.

[1662] Server: The server stores the received biometric data in a database and analyzes the data using analytical programs. For example, algorithms are used to determine whether heart rate or blood pressure is within normal ranges. The algorithms generate warnings if abnormal values ​​are detected.

[1663] Notification System

[1664] Server: If the server detects an abnormality based on the analysis results, it immediately generates a notification. The notification will include the specific abnormality and a recommended action, such as "Your heart rate is out of the normal range. Contact your doctor immediately."

[1665] Device: The device receives notifications from the server and displays an alert on the user's smartphone or tablet. At the same time, the server also sends notifications to relevant parties (e.g., family members or doctors) via email or SMS.

[1666] User: The user checks the alert displayed on the device and contacts a doctor if necessary. Notifications that can be viewed by the user allow for prompt and appropriate action to be taken.

[1667] Support tools provided

[1668] User: When a user needs health management, they operate their device to request support tools. For example, they may want to view physical therapy videos or breathing exercise guides.

[1669] Terminal: The terminal receives a user request, launches the support tool providing program, retrieves the necessary content from the server, and displays it.

[1670] Server: The server retrieves appropriate support tools (such as breathing exercise videos or exercise guidelines) from the database and sends them to the device.

[1671] Users: Users can use support content to practice physical therapy and health guidelines.

[1672] Reminders

[1673] User: The user sets the time to take medicine and exercise schedule on the device. For example, the user sets the time to take medicine at 9:00 every day using a dedicated app.

[1674] Server: The server manages the user's schedule and generates reminders at the specified time.

[1675] Device: The device receives reminders from the server and displays alarms and notifications at the specified time.

[1676] User: Users can take their medicines according to the reminders and maintain a regular health regime.

[1677] Examples of prompt statements

[1678] Describe how the following system works: A user wears a smartwatch to monitor their heart rate. If the heart rate reaches an abnormal level, the server generates a notification, displays an alert on the user's smartphone, and provides a video with appropriate breathing exercises.

[1679] Through these functions, the system provides a safe and secure environment for users to live at home. It enables comprehensive and real-time health management, enables rapid response in the event of an abnormality, and provides tools to support daily health management.

[1680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1681] Specific explanation of processing steps

[1682] Step 1: Collecting biometric information

[1683] User: The user wears a smartwatch or health monitoring sensors that measure heart rate, blood pressure, and body temperature. The input is real-time biometric information from these sensors. The output is the acquired biometric data: heart rate, blood pressure, and body temperature.

[1684] How it works: When a user wakes up in the morning and puts on the smartwatch, the sensors start collecting data on their heart rate and body temperature from that moment on.

[1685] Step 2: Sending biometric information

[1686] Terminal: The terminal periodically transmits biometric information acquired from the smartwatch to the server. The input is biometric information acquired from the smartwatch via Bluetooth connection. The output is biometric information transmitted to the server via Wi-Fi or mobile data communication.

[1687] How it works: The smartwatch connects to the smartphone via Bluetooth, and the smartphone sends data to the server using Wi-Fi.

[1688] Step 3: Analysis of biological information

[1689] Server: The server stores the received biometric information in a database and analyzes the data using an analysis program. The input is the biometric information received from the device. The output is the analysis result, judging whether it is within the normal range or abnormal.

[1690] Specific operation: The server analyzes the received heart rate data, calculates the average and standard deviation, and generates an alert if an abnormal value is detected.

[1691] Step 4: Generate a notification message

[1692] Server: The server immediately generates a notification message if an anomaly is detected based on the analysis results. The input is the anomaly judgment of the analysis results. The output is a notification message containing the specific anomaly content and recommended actions.

[1693] What happens: The server generates the message "Your heart rate is outside the normal range. Contact your doctor immediately."

[1694] Step 5: Sending notifications

[1695] Server: The server generates and sends notification messages to the user and the designated parties. The input is the generated notification message. The output is the notification message sent to the user's smartphone and the designated parties.

[1696] Specific operation: The server sends a notification to the user's smartphone and simultaneously notifies the user's family and doctor by email.

[1697] Step 6: Receive and confirm notifications

[1698] Terminal: The terminal receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification message sent from the server. The output is the alert displayed on the user's smartphone.

[1699] User: The user checks the alert displayed on the device and contacts a doctor if necessary. The input is the alert message displayed on the smartphone. The output is the user's response (e.g., contacting a doctor).

[1700] Specific operation: The user sees the "abnormal heart rate" notification on their smartphone screen and then uses the app to contact their doctor.

[1701] Step 7: Request Support Tools

[1702] User: The user requests the necessary support tools through the terminal. The input is an operation to request a physical therapy video or a breathing exercise guide. The output is the information sent in response to the request.

[1703] Specific operation: The user selects "Breathing exercise video" from the menu of the dedicated app.

[1704] Step 8: Providing support tools

[1705] Terminal: The terminal sends the user's request to the server and starts the support tool providing program. The input is the user's request information. The output is the start of the support tool providing program and the transmission of the request to the server.

[1706] Server: The server retrieves the corresponding support tool (such as a breathing exercise video or exercise guidelines) from the database and sends it to the device. The input is the user's request information. The output is the data of the support tool that was sent.

[1707] Specific operation: The server sends a link to a breathing exercise video to the device, and the device plays the video.

[1708] Step 9: Set reminders

[1709] User: The user sets up schedules such as medication times and exercise schedules on the device. The input is the schedule setting information in the dedicated app. The output is the set schedule information.

[1710] Specific operation: The user uses the calendar function of the dedicated app to schedule taking the medicine at 9:00 every day.

[1711] Step 10: Generate reminders

[1712] Server: The server manages the user's schedule and generates reminders at the specified time. The input is the user's schedule information. The output is the generated reminder notification.

[1713] What happens: The server generates a reminder notification every day at 9:00 AM based on the user's settings.

[1714] Step 11: Reminder Notifications

[1715] Terminal: The terminal receives reminders from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server. The output is the reminder displayed on the terminal.

[1716] User: The user takes medicine according to the reminder and manages their health. The input is the reminder notification displayed on the device. The output is the user's coping behavior (e.g., taking medicine).

[1717] Specific operation: At 9 o'clock, the user's smartphone will sound an alarm and notify them that it's time for their medicine, and they will then take their medicine.

[1718] (Application example 1)

[1719] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1720] For elderly people and those with chronic illnesses, constant monitoring of their health status and prompt response when an abnormality occurs are extremely important for them to continue living safely at home. However, conventional systems lack the functionality to monitor users' health status in real time or an immediate notification system in case of an emergency, making it difficult to fully ensure user safety. Furthermore, the provision of support tools for daily health management and improving quality of life is also insufficient.

[1721] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1722] In this invention, the server includes a signal generating means for generating an output signal when an abnormality is detected based on the user's biological information, a support tool providing means, and a schedule management means. This makes it possible to monitor the user's health condition in real time and quickly detect and notify abnormalities. It also provides various functions to support daily health management and can support the user in improving their lifestyle habits through reminders.

[1723] The "sensor means" is a device for collecting biometric information of a user.

[1724] The "terminal means" is a communication device for transmitting the biometric information collected by the sensor means to the server.

[1725] The "analysis means" is a system that analyzes the biometric information sent to the server and determines whether or not there is an abnormality.

[1726] The "notification means" is a function for sending a notification to the user and designated related parties when the analysis means determines that an abnormality has occurred.

[1727] The "support tool providing means" is a function for providing physical therapy and physical guidelines to the user.

[1728] The "schedule management means" is a function for sending reminders based on the user's schedule.

[1729] The "signal generating means" is a device or system that generates an output signal when an abnormality is detected based on the user's biological information.

[1730] "Electronic communication" means a means of communication to send notices to designated parties via email or SMS.

[1731] An "alert" is a warning message that is displayed on the user's terminal when an abnormality occurs.

[1732] 1. System Configuration

[1733] The present invention provides a system that monitors the user's health condition in real time and notifies the user promptly if an abnormality occurs, thereby providing an environment in which the user can live safely at home. Specific embodiments will be described below.

[1734] 2. Hardware and Software Used

[1735] Hardware:

[1736] Smartphone (iOS / Android)

[1737] Smartwatches (Apple Watch, Fitbit, etc.)

[1738] Health management sensors (heart rate sensors, blood pressure monitors, etc.)

[1739] software:

[1740] Mobile application (cross-platform using Flutter)

[1741] Server (using Amazon Web Services)

[1742] Database (using Amazon RDS)

[1743] Notification system (Firebase Cloud Messaging)

[1744] 3. Operation of various means

[1745] Sensor means

[1746] Users wear smartwatches and health monitoring sensors that collect real-time biometric information such as heart rate, blood pressure, and body temperature.

[1747] Terminal means

[1748] Smartphones and tablets function as terminals and send collected biometric information to Amazon Web Services (AWS).

[1749] Analysis means

[1750] The server receives the vital signs and stores them in Amazon RDS, then uses an analysis program (using a generative AI model) to determine whether the heart rate and blood pressure are within normal ranges.

[1751] Notification means

[1752] If an abnormality is detected as a result of the analysis, an alert is sent to the user's smartphone using Firebase Cloud Messaging. At the same time, pre-registered contacts (family members and doctors) are also notified via SMS and email.

[1753] Support tool provision method

[1754] When a user requests a physical therapy video or health guidelines, the server retrieves the corresponding content from Amazon RDS and provides it to the user through a mobile application.

[1755] Schedule management tools

[1756] Users set their medication and exercise schedules in the mobile app. The server manages this schedule and generates reminders at the specified times. The mobile app then notifies the user of these reminders on their smartphone.

[1757] Signal generation means

[1758] When an abnormality is detected based on the user's biometric information, the server generates an output signal that displays an alert on the user's device and sends an electronic communication to designated parties.

[1759] 4. Specific Examples

[1760] For example, if User B, who is 70 years old, is wearing a smartwatch for heart rate monitoring, the behavior is as follows:

[1761] 1. The smartwatch detects an abnormal heart rate.

[1762] 2. The collected heart rate data is sent to the AWS server via a smartphone app.

[1763] 3. The server analyzes the heart rate data and detects abnormalities.

[1764] 4. Use Firebase Cloud Messaging to send an alert to User B's smartphone.

[1765] 5. An abnormality notification is sent to User B's family and doctor via email or SMS.

[1766] 6. Play a breathing exercise video on your smartphone to help stabilize your heart rate.

[1767] 7. Remind me when it's time to take my medicine.

[1768] 5. Examples of prompts

[1769] "A 70-year-old man is wearing a smartwatch for monitoring his heart rate. If his heart rate becomes abnormally high, the AWS server will analyze it and send a notification to his smartphone using Firebase Cloud Messaging. The system will also notify his family of the abnormality via email and SMS, play a video on breathing exercises to stabilize his heart rate on the smartphone, and remind him to take his medication. Please generate appropriate prompts taking these situations into consideration."

[1770] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1771] Step 1:

[1772] The user wears a smartwatch or health management sensor.

[1773] Input: User's biometric information (heart rate, blood pressure, temperature, etc.)

[1774] Data processing: Sensors collect data in real time

[1775] Output: Biometric data

[1776] Step 2:

[1777] The terminal transmits the collected biometric information to the server.

[1778] Input: Biometric data obtained from sensors

[1779] Data processing: Converting data into packets and sending them to a server via the Internet

[1780] Output: Biometric data transmitted to the server

[1781] Step 3:

[1782] The server receives the biometric information and stores it in a database.

[1783] Input: Transmitted biometric data

[1784] Data processing: storing data in a suitable format for the database

[1785] Output: Stored biometric data

[1786] Step 4:

[1787] The server analyzes the stored biometric information and detects any abnormalities.

[1788] Input: Stored biometric data

[1789] Data calculation: Analyze using generative AI models and compare with normal ranges

[1790] Output: Analysis result (normal / abnormal)

[1791] Step 5:

[1792] Generates a notification message if the server detects an abnormality.

[1793] Input: Analysis results (when an abnormality is detected)

[1794] Data processing: Generate notification messages about anomalies

[1795] Output: Information message

[1796] Step 6:

[1797] The server uses Firebase Cloud Messaging to send notifications to the user device.

[1798] Input: Notification message

[1799] Data processing: Encode the message in FMC protocol and send it

[1800] Output: An alert displayed on the user's terminal

[1801] Step 7:

[1802] At the same time, the server sends a notification to pre-registered contacts.

[1803] Input: Notification message, registered contact information

[1804] Data processing: Convert to email or SMS format and send to contacts

[1805] Output: Email or SMS notification sent to your contacts

[1806] Step 8:

[1807] The user terminal provides the support tool in response to a request.

[1808] Input: User request, analysis results

[1809] Data processing: Retrieve and play corresponding video and guideline content

[1810] Output: Supporting content (e.g., video) displayed on the user's device

[1811] Step 9:

[1812] The server generates and sends reminders based on a schedule set by the user.

[1813] Input: User's schedule information

[1814] Data processing: Generate and send reminders at specified times

[1815] Output: Reminder notification displayed on the user's device

[1816] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1817] This invention is a system that provides a safe and secure environment for users at home. By combining it with an emotion engine that recognizes the user's emotions, it provides advanced health management and support. This system monitors the user's health status in real time, promptly notifies the user when an abnormality occurs, and encourages appropriate action. It also provides physical therapy and health guidelines based on the user's emotional state, and reminders to improve quality of life.

[1818] System programs and their processing

[1819] Health Monitoring and Emotion Engine

[1820] User: Users wear smartwatches and health monitoring sensors on a daily basis, which collect real-time data such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[1821] Device: The device temporarily stores biometric and emotional state data collected from sensors and periodically transmits it to the server.

[1822] Server: The server stores the received biometric and emotional state data in a database and analyzes it using an analysis program that identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1823] Notification System

[1824] Server: If an anomaly is detected based on the analysis results, the server generates a notification message that is sent to the user and designated parties when an emergency call is required or there is a high risk.

[1825] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. It can also send emails or SMS to relevant parties such as family members or doctors.

[1826] User: The user checks the alert on the device and contacts a doctor if necessary.

[1827] Support tools provided

[1828] User: When users need daily health management, they operate their device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[1829] Terminal: The terminal sends the user's request and emotional state information to the server.

[1830] Server: The server retrieves corresponding support tools (e.g., physical therapy videos and exercise guidelines) from the database and provides them tailored to the user's emotional state.

[1831] Terminal: The terminal displays the received support tool on the user's smartphone or tablet.

[1832] User: The user utilizes the displayed support tools and acts in accordance with the physical therapy and health guidelines.

[1833] Reminders

[1834] User: The user sets medication times and exercise schedules on the device.

[1835] Server: The server runs a program that stores the configured schedule in a database and generates reminders based on the schedule.

[1836] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time.

[1837] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[1838] Specific examples

[1839] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[1840] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[1841] 2. Device: Sends collected heart rate data and emotional state data to the server.

[1842] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[1843] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[1844] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[1845] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[1846] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[1847] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[1848] This system provides comprehensive monitoring, including emotional state, prompt notification, and emotional support tools to create an environment where users can live safely at home.

[1849] The processing flow will be explained below.

[1850] Health Monitoring and Emotion Engine

[1851] Step 1:

[1852] Users wear smartwatches and health management sensors that automatically collect data in real time, such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice.

[1853] Step 2:

[1854] The device receives the collected biometric and emotional state data on the user's smartphone or tablet and temporarily stores it.

[1855] Step 3:

[1856] The terminal periodically transmits the collected data to the server.

[1857] Step 4:

[1858] The server stores the received biometric information and emotional state data in a database.

[1859] Step 5:

[1860] The server analyzes biometric information to identify abnormal values ​​such as heart rate and blood pressure, and also uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1861] Notification System

[1862] Step 6:

[1863] If the server detects an anomaly based on the analysis, it generates a notification message containing details about the user's health and emotional state.

[1864] Step 7:

[1865] The server generates and sends the notification message to the user's device and the designated parties, for example via email or SMS.

[1866] Step 8:

[1867] The notification message received by the device is displayed as an alert on the user's smartphone or tablet.

[1868] Step 9:

[1869] Designated stakeholders (family members or doctors) will receive email and SMS notifications.

[1870] Step 10:

[1871] The user can check the alerts on the device, contact a doctor if necessary, and receive relaxation techniques and support based on their emotional state.

[1872] Support tools provided

[1873] Step 11:

[1874] The user operates the device to request a support tool, for example, to watch a video on relaxation techniques.

[1875] Step 12:

[1876] The terminal transmits the user's request and emotional state information to the server.

[1877] Step 13:

[1878] The server retrieves the corresponding support tools (e.g., physical therapy videos or exercise guidelines) from the database and selects the appropriate content based on the emotion engine.

[1879] Step 14:

[1880] The server transmits the acquired support tool to the terminal.

[1881] Step 15:

[1882] The support tool received by the terminal is displayed on the user's smartphone or tablet.

[1883] Step 16:

[1884] The user utilizes the displayed support tools and acts in accordance with physical therapy and health guidelines.

[1885] Reminders

[1886] Step 17:

[1887] The user operates the device to set medication times and exercise schedules.

[1888] Step 18:

[1889] The server saves the configured schedule in the database.

[1890] Step 19:

[1891] The server launches a program to generate reminders based on the saved schedule.

[1892] Step 20:

[1893] The server sends a reminder notification to the device at the scheduled time.

[1894] Step 21:

[1895] Your device will receive reminder notifications and display alarms and notifications at the specified times.

[1896] Step 22:

[1897] The user checks the reminder notification and takes action as instructed, for example, taking medication.

[1898] Through these steps, the system provides a safe and secure environment for users to live at home. Furthermore, by integrating an emotion engine, the system improves the user's quality of life by providing physical therapy and support tools based on the user's emotional state.

[1899] Example 2

[1900] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1901] The main problem that this invention aims to solve is to effectively manage a user's health and monitor their emotional state in their daily lives, and provide prompt and appropriate responses and support. Conventional health management systems lack the ability to detect abnormal conditions and provide emotional support, which can lead to a decline in the user's health and quality of life. In response, this invention aims to realize an environment in which users can live with peace of mind by comprehensively monitoring a user's biometric information and emotional state and providing appropriate notifications and support.

[1902] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a measurement means for collecting biometric information of the user, a communication means for transmitting the biometric information collected by the measurement means to the server, a diagnosis means for analyzing the biometric information and determining whether or not there is an abnormality, a warning means for sending a notification to the user and designated relevant parties when the diagnosis means determines an abnormality, a support means for providing the user with physical therapy and health guidelines, and a suggestion means for sending reminders based on the user's schedule. This makes it possible to comprehensively manage the user's health and emotional state in real time and provide prompt notification and appropriate support when an abnormality is detected.

[1903] "Measurement means" refers to a device or function that collects biometric information from a user.

[1904] "Communication means" refers to a device or function for transmitting the biometric information collected by the measurement means to a server.

[1905] "Diagnostic means" refers to a device or function that analyzes collected biological information to determine whether or not there is an abnormality.

[1906] "Alert means" refers to a device or function that sends a notification to the user and designated parties when the diagnostic means determines an abnormality.

[1907] "Assistive device" refers to a device or function that provides physical therapy or health guidelines to a user.

[1908] "Suggestion means" refers to a device or function that sends reminders based on the user's schedule.

[1909] "Reminder" refers to an alert or message that notifies a user of a specific action or schedule.

[1910] "Biometric information" refers to data such as a user's heart rate, blood pressure, body temperature, facial expression, and tone of voice.

[1911] "Abnormal" refers to a state in which the user's biological information deviates from a preset normal range.

[1912] "Emotional state" refers to the psychological and emotional state of a user as determined by their facial expression and tone of voice.

[1913] "Support tools" refers to resources such as physical therapy and health guidelines to help users manage their health.

[1914] This invention is a system for providing a user with a safe and secure living environment at home, monitoring the user's health and emotional state in real time, notifying them promptly when an abnormality occurs, and encouraging them to take appropriate action. This system consists of the following components:

[1915] Health Monitoring and Emotion Engine

[1916] User: The user wears a smartwatch or health management sensor on a daily basis. The smartwatch collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. Examples of such devices include the Apple Watch and Fitbit.

[1917] Device: The device temporarily stores the collected biometric and emotional data, which is then periodically sent to a server via Bluetooth or Wi-Fi.

[1918] Server: The server stores the received data in a database and analyzes the biometric information using an analysis program. Programming languages ​​such as Python and R can be used for the analysis. Specific libraries used include Pandas and NumPy. The analysis program identifies abnormal values ​​in the user's heart rate, blood pressure, and body temperature, and uses an emotion engine to determine the user's emotions from facial expressions and tone of voice.

[1919] Notification System

[1920] Server: Based on the analysis results, the server generates a notification message if an anomaly is detected. This message is sent to the user and designated parties. The message can be sent via email or short message service (SMS).

[1921] Device: The device receives notifications from the server and displays alerts on the user's smartphone or tablet. Notifications can be pop-ups or audio alerts, and can also send emails or SMS to relevant parties such as family members or doctors.

[1922] User: The user checks the alerts on the device and contacts a doctor if necessary. For example, this includes the ability to press an emergency contact button to talk directly to a doctor.

[1923] Support tools provided

[1924] User: When users need daily health management, they operate the device to request support tools. Based on the emotion engine, appropriate physical therapy and health guidelines are suggested.

[1925] Terminal: The terminal sends the user's request and emotional state information to the server.

[1926] Server: The server retrieves the corresponding support tools from the database and provides health guidelines and physical therapy videos tailored to the user's emotional state. Specifically, YouTube and Vimeo are used as healthcare video resources.

[1927] Device: The device displays the received support tool on the user's smartphone or tablet, for example, by playing a physical therapy video using a video playback app.

[1928] User: The user utilizes the displayed support tools and follows the physical therapy and health guidelines.

[1929] Reminders

[1930] User: The user sets medication times and exercise schedules on the device using a dedicated app.

[1931] Server: The server stores the configured schedule in a database and runs a program that generates reminders based on the schedule.

[1932] Device: The device receives reminder notifications from the server and displays alarms and notifications at the specified time, for example, a pop-up notification or a vibration alert.

[1933] User: The user checks the reminder notification and acts as instructed, for example, taking medication.

[1934] Specific examples

[1935] For example, assume that User C, aged 65, is wearing a smartwatch for heart rate monitoring. If his heart rate becomes abnormally high and at the same time the emotion engine detects a state of stress, the procedure is as follows:

[1936] 1. User: The smartwatch detects abnormal heart rate and determines the user's emotional state from their facial expressions and tone of voice.

[1937] 2. Device: Sends collected heart rate data and emotional state data to the server.

[1938] 3. Server: Analyzes the data and detects anomalies and high-stress situations.

[1939] 4. Server: Sends notifications about abnormal heart rate and high stress state to user C and his family.

[1940] 5. Device: An alert is displayed on User C's smartphone and an email is sent to his / her family.

[1941] 6. User: Checks the alert, contacts a doctor, and watches a video on relaxation techniques suggested by the server.

[1942] 7. Providing support tools: The device plays physical therapy videos adapted to the emotion engine.

[1943] 8. Reminder: User C sets the time to take medicine, and the device notifies him / her at the specified time.

[1944] When inputting a prompt sentence into a generative AI model, the following prompt could be used, for example:

[1945] "Please provide a detailed description of the system that monitors the user's health and emotional state, notifies them appropriately when abnormalities are detected, and provides support tools and reminders. Please also include specific examples of the hardware and software used."

[1946] Using this prompt, the generative AI model can automatically generate the system details.

[1947] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1948] Step 1:

[1949] User: Wears a smartwatch or health management sensor, which collects biometric information such as heart rate, blood pressure, body temperature, facial expressions, and tone of voice in real time. The specific input is the user's biometric information, and the output is the collected data.

[1950] Step 2:

[1951] Device: Temporarily stores the biometric information collected from the sensor. The input is the biometric information acquired in step 1, and the output is the data stored in the device's internal storage.

[1952] Step 3:

[1953] Device: The temporarily stored data is periodically sent to the server via Bluetooth or Wi-Fi. The input is the biometric information stored in the device's internal storage, and the output is the data sent to the server.

[1954] Step 4:

[1955] Server: Stores the received data in a database. The input is the biometric information sent from the device, and the output is the data stored in the database.

[1956] Step 5:

[1957] Server: Analyzes the biometric information using an analysis program. This uses Python or R libraries (e.g., Pandas or NumPy). The input is the biometric information stored in the database, and the output is the analysis results.

[1958] Step 6:

[1959] Server: Generates a notification message based on the analysis results if an anomaly is detected. The input is the analysis results, and the output is the generated notification message.

[1960] Step 7:

[1961] Server: Sends generated notification messages to users and designated parties. The input is the generated notification message and the output is the sent notification.

[1962] Step 8:

[1963] Terminal: Receives notifications from the server and displays alerts on the user's smartphone or tablet. The input is the notification received from the server, and the output is the alert displayed on the user's terminal.

[1964] Step 9:

[1965] User: Checks the alert and contacts a doctor if necessary. The input is the alert displayed on the user's device, and the output is the user's action (e.g., contacting a doctor).

[1966] Step 10:

[1967] User: Requests a support tool by operating a terminal. The input is the user's request, and the output is the request data generated by the terminal.

[1968] Step 11:

[1969] Terminal: Sends the user's request and emotion information to the server. The input is the user's request and emotion information, and the output is the request data sent to the server.

[1970] Step 12:

[1971] Server: Retrieves the appropriate support tool from the database and generates the content to be provided. The input is the request data sent to the server and the content of the database, and the output is the content of the generated support tool.

[1972] Step 13:

[1973] Terminal: The support tool is displayed on the user's smartphone or tablet. The input is the content of the support tool sent from the server, and the output is the support tool displayed on the user's terminal.

[1974] Step 14:

[1975] User: Uses the displayed support tools and acts in accordance with the health guidelines. The input is the support tools displayed on the terminal, and the output is the user's actions (e.g., performing physical therapy).

[1976] Step 15:

[1977] User: Sets medication times and exercise schedules on the device. The input is the schedule information set by the user, and the output is the schedule data saved on the device.

[1978] Step 16:

[1979] Server: Saves the set schedule in the database and runs the reminder generation program. The input is the schedule data sent from the terminal, and the output is the schedule saved in the database and the generated reminder notification.

[1980] Step 17:

[1981] Terminal: Receives reminder notifications from the server and displays alarms and notifications at the specified time. The input is the reminder notification sent from the server, and the output is the alarm and notification displayed on the user terminal.

[1982] Step 18:

[1983] User: Checks the reminder notification and acts according to the instructions. The input is the reminder notification displayed on the device, and the output is the user's action (e.g., taking medicine).

[1984] (Application example 2)

[1985] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1986] Conventional health monitoring systems are limited to collecting and analyzing users' biometric information, without taking into account their emotional state. This makes it difficult to respond appropriately in situations that are influenced by emotions such as stress or anxiety. In addition, notification systems that detect abnormalities only notify users of the abnormality and do not provide specific countermeasures or support. This has led to issues that do not adequately improve users' quality of life or ensure their safety.

[1987] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1988] In this invention, the server includes means for collecting biometric information and emotional information of the user, means for analyzing the collected data to determine abnormalities or emotional states, means for sending notifications to the user and designated parties when abnormalities or specific emotional states are determined, means for providing the user with physical therapy and health guidelines based on their emotional states, and means for sending reminders based on the user's schedule. This makes it possible to monitor the user's health state, respond quickly when abnormalities occur, and provide appropriate support tools according to their emotional states.

[1989] The "sensor means" is a device for collecting biometric and emotional information of a user.

[1990] The "terminal means" is a device for transmitting the biometric information and emotional information collected by the sensor means to the server.

[1991] The "analysis means" is a device or program for analyzing the bio-information and emotional information and determining abnormalities and emotional states.

[1992] The "notification means" is a device or program for sending a notification to the user and designated related parties when the analysis means determines an abnormality or a particular emotional state.

[1993] The "support tool providing means" is a device or program for providing physical therapy or health guidelines based on the user's emotional state.

[1994] A "reminder means" is a device or program for sending reminders based on a user's schedule.

[1995] "Biometric information" refers to data such as the user's heart rate, blood pressure, and body temperature.

[1996] "Emotional information" refers to the emotional state of the user that can be obtained from facial expressions, tone of voice, etc.

[1997] The present invention is a system that monitors a user's biological information and emotional state, promptly notifies the user if an abnormality is detected, and provides support tools according to the user's emotional state. Specific embodiments of the system are described below.

[1998] Overall system configuration

[1999] Data collection

[2000] Sensor means

[2001] The user wears a sensor that collects biometric information such as heart rate, blood pressure, and body temperature. This sensor can be a smartwatch or a dedicated health management sensor. In addition, a camera and microphone can be used to collect emotional information such as the user's facial expressions and tone of voice.

[2002] Terminal means

[2003] The data collected by the sensor means is sent to devices such as smartphones and tablets, allowing the user's biological and emotional information to be monitored in real time.

[2004] Data Analysis and Notification

[2005] server

[2006] The data sent from the device is stored on a server via the Internet. The server then analyzes the data using software such as Python, TensorFlow, and Keras. Specifically, it analyzes data such as heart rate, blood pressure, and body temperature to detect abnormal values. It also uses an emotion engine to determine the user's emotional state from facial expressions and tone of voice.

[2007] Notification means

[2008] If the server detects an abnormality or a specific emotional state, it will send a notification to the user and designated parties using Firebase Cloud Messaging, Twilio API, etc. The notification will be displayed as an alert on the user's device and will be sent to the parties via email or SMS.

[2009] Providing support tools

[2010] server

[2011] The server provides appropriate physical therapy and health guidelines based on the user's emotional state, including videos and guidelines created using React Native and Rest APIs. For example, if the user is in a high-stress state, a relaxation video will be provided.

[2012] Terminal

[2013] Support tools sent from the server are displayed on the user's smartphone or tablet, allowing the user to act in accordance with the presented physical therapy and health guidelines.

[2014] Reminders

[2015] server

[2016] Based on the schedule set by the user, the server generates reminders and notifies them at the specified times.

[2017] Terminal

[2018] Reminder notifications received from the server are displayed on the user's device, helping the user to remember to take their medicine or to exercise regularly.

[2019] Specific examples

[2020] For example, if a 65-year-old user wears a smartwatch and monitors their heart rate and blood pressure in real time, and the emotion engine detects an abnormally high heart rate and a high stress state, the system will operate as follows:

[2021] 1. Sensor means

[2022] The smartwatch detects abnormal heart rates and senses the user's facial expressions and tone of voice through the camera and microphone.

[2023] 2. Terminal means

[2024] The collected data is sent to a smartphone.

[2025] 3. Server

[2026] The data is analyzed to detect abnormal heart rates and high stress states.

[2027] 4. Means of notification

[2028] Based on the analysis results, notifications are sent to the user and relevant parties. An alert stating "An abnormal heart rate and high stress state have been detected" is displayed on the user's smartphone, and a similar message is sent to relevant parties via email.

[2029] 5. Means of providing support tools

[2030] A relaxing video is displayed on the user's smartphone.

[2031] 6. Reminders

[2032] When it's time for the user to take their medication, a notification will appear on their smartphone.

[2033] Prompt Sentence Examples

[2034] "If my heart rate exceeds 100, show me a stress relief video and send a notification message to my family."

[2035] In this way, the present invention monitors the user's health and emotional state, promptly notifies them in the event of an abnormality, and provides appropriate support, thereby creating an environment in which the user can live with peace of mind.

[2036] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2037] Step 1:

[2038] The user wears a sensor device (e.g., a smart watch). The sensor collects biometric information such as heart rate, blood pressure, and body temperature in real time, and also detects the user's facial expressions and tone of voice. The input is biometric and emotional information collected by the sensor.

[2039] Step 2:

[2040] The terminal means (e.g., a smartphone) temporarily stores the biometric and emotional information received from the sensor and periodically transmits it to a server via the Internet. The input is the biometric and emotional information sent from the sensor, and the output is the data sent to the server. Specifically, it receives data from the sensor using Bluetooth Low Energy (BLE), converts the data into JSON format, and transmits it to the server.

[2041] Step 3:

[2042] The server stores the biometric and emotional information received from the device in a database and performs data analysis using Python, TensorFlow, Keras, etc. The input is the biometric and emotional information sent from the device, and the output is the analysis results (abnormal values ​​and specific emotional states). Specifically, the server organizes the data as time-series data and analyzes it by applying anomaly detection algorithms and emotion recognition models.

[2043] Step 4:

[2044] If the server detects an anomaly or a specific emotional state, it uses Firebase Cloud Messaging or Twilio API to generate a notification and send it to the user and designated stakeholders. The input is the analysis result, and the output is the notification message. Specifically, it displays an alert on the user's smartphone and sends an email or SMS to the relevant stakeholders.

[2045] Step 5:

[2046] Based on the analysis results, the server provides physical therapy and health guidelines tailored to the user's emotional state. The input is the analysis results of biometric and emotional information, and the output is the proposed support tools (videos and guidelines). Specifically, it retrieves relevant videos and guidelines from the database and sends them to the user's device using React Native and a Rest API.

[2047] Step 6:

[2048] The device displays support tools (relaxation videos and health guidelines) received from the server to the user. The input is the support tools sent from the server, and the output is the content displayed on the device display. Specifically, the device displays the received videos and guidelines on the playback screen.

[2049] Step 7:

[2050] The server generates reminders based on the schedule set by the user and sends notifications to the device at the specified time. The input is the user's schedule information, and the output is the reminder notification. Specifically, it calculates the timing of the reminder notification based on the schedule information and sends the notification using Firebase Cloud Messaging.

[2051] Step 8:

[2052] The terminal displays the reminder notification received from the server to the user at the specified time. The input is the reminder notification from the server, and the output is the reminder displayed to the user. Specifically, the terminal generates an alarm and a pop-up notification at the specified time to notify the user.

[2053] In this way, the entire system's processing is coordinated at each step, enabling comprehensive monitoring of the user's health and emotional state and providing appropriate responses and support.

[2054] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2055] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2056] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2057] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2058] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2059] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2060] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2061] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2062] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2063] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2064] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2065] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2066] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2067] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2068] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2069] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2070] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2071] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2072] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2073] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2074] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2075] The following is further disclosed regarding the above embodiment.

[2076] (Claim 1)

[2077] a sensor means for collecting biometric information of a user;

[2078] a terminal means for transmitting the biological information collected by the sensor means to a server;

[2079] an analysis means for analyzing the biological information and determining whether or not there is an abnormality;

[2080] a notification means for sending a notification to the user and designated parties when the analysis means determines an abnormality;

[2081] A support tool providing means for providing physical therapy and health guidelines to users;

[2082] a reminder means for sending reminders based on the user's schedule;

[2083] A system including:

[2084] (Claim 2)

[2085] 2. The system according to claim 1, wherein the support tool providing means includes means for displaying a video of breathing exercises and various health guidelines.

[2086] (Claim 3)

[2087] 2. The system according to claim 1, wherein the notification means includes means for displaying an alert on a user's terminal and means for sending an email or SMS to designated parties when an abnormality is determined by the analysis means.

[2088] "Example 1"

[2089] (Claim 1)

[2090] an input means for collecting biometric information of a user;

[2091] a communication means for transmitting the biological information collected by the input means to a processing system;

[2092] an analysis means for analyzing the biological information and determining whether or not there is an abnormality;

[2093] a notification means for sending a notification to the user and designated parties when the analysis means determines an abnormality;

[2094] Support means for providing users with health management and health guidelines;

[2095] a schedule support means for sending reminders based on the user's sch...

Claims

1. a sensor means for collecting biometric information of a user; a terminal means for transmitting the biological information collected by the sensor means to a server; an analysis means for analyzing the biological information and determining whether or not there is an abnormality; a notification means for sending a notification to the user and designated parties when the analysis means determines an abnormality; A support tool providing means for providing physical therapy and health guidelines to users; a reminder means for sending reminders based on the user's schedule; A system including:

2. The system according to claim 1 , wherein the support tool providing means includes means for displaying a video of a breathing technique and various health guidelines.

3. The system according to claim 1, wherein the notification means includes means for displaying an alert on the user's terminal and means for sending an email or SMS to designated parties when an abnormality is determined by the analysis means.

Images