system

Abstract

We provide the system. [Solution] Means for collecting biometric information from users, A means of transmitting the collected biometric information to a server, A means by which a server analyzes received biometric information using a generative model and evaluates health status, A means of providing health guidance to users based on analysis results, A means of sending alerts to users and medical institutions when an anomaly is detected, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern society, with the progress of aging, an extension of the healthy life span is demanded. However, there is not a sufficient system for early detection of diseases and appropriate responses. Also, it is necessary to effectively utilize individual health data and provide optimal health guidance for each individual to reduce health risks in advance. However, existing systems have limitations in data collection and analysis and have not achieved effective health management.

Means for Solving the Problems

[0005] This invention provides a system that collects biometric information from users' smart devices and analyzes that information using a generative model on a server to provide users with highly accurate health guidance. This system immediately alerts users and medical institutions when it detects abnormalities in the collected biometric information, thereby enabling early disease detection and prevention. Furthermore, it provides specific lifestyle improvement advice based on the analysis results, supporting individual health management. In this way, the system aims to extend healthy life expectancy and reduce medical costs.

[0006] A "user" refers to an individual who uses the system to manage their own health status.

[0007] "Biometric information" refers to data that indicates the physical state of an organism, including heart rate, steps taken, and sleep patterns.

[0008] "Means" refers to methods or devices used to achieve a specific purpose, and in this invention, it refers to a mechanism for realizing a function.

[0009] A "server" refers to a computing system that receives, stores, and analyzes data, and is responsible for receiving data from terminals via a network.

[0010] A "generative model" refers to a mathematical or algorithmic model used to analyze data using machine learning and to predict or estimate specific outcomes.

[0011] "Health guidance" refers to advice and recommendations regarding lifestyle improvements and health maintenance, provided based on the user's health status.

[0012] An "alert" refers to an urgent notification sent to users or relevant organizations when an anomaly is detected.

[0013] "Medical institution" refers to hospitals, clinics, and other facilities or organizations that provide medical services. [Brief explanation of the drawing]

[0014] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system that supports users' health management using smartphones and wearable devices. This system monitors and provides guidance on the user's health status based on biometric information that the user continuously acquires in their daily life.

[0036] First, the device uses built-in sensors to acquire biometric information, such as the user's heart rate, steps taken, and sleep patterns. This information is recorded in real time or at specified time intervals and temporarily stored within the device.

[0037] The device then transmits biometric information to a server via the internet. This communication is encrypted, ensuring the security of the transmitted data. At this point, the user's lifestyle and past health data are also stored in a database, forming a dataset that forms the basis for analysis.

[0038] The server processes the received biometric information and uses a generative model to analyze the user's current health status. The model is trained on previously collected health data and can predict health risks and detect anomalies. For example, it can estimate stress levels from heart rate variability patterns or warn of sleep deprivation.

[0039] Based on the analysis results, the server generates personalized health guidance for each user. This guidance includes, for example, suggestions for dietary improvements, exercise recommendations, and advice for maintaining mental health. The generated guidance is sent to the device, allowing users to receive alert notifications or check it through a dedicated app.

[0040] Furthermore, if a serious abnormality is detected, the server immediately sends an alert to the user and registered medical institutions. Upon receiving the alert, the terminal notifies the user via voice and vibration to encourage a quick response. This function enables early detection of illness and emergency response.

[0041] Through this system, users can monitor their daily health status and improve their lifestyle habits according to the guidance provided. This allows users to proactively maintain their health, reduce the frequency of medical visits, and aim to extend their healthy lifespan. For example, if a user is told that their sleep quality is deteriorating, they can create an action plan to develop better sleep habits and work towards improvement.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The device uses the user's smartphone or wearable device to collect biometric information. Information such as heart rate, steps taken, and sleep patterns is acquired in real time and temporarily stored in local storage.

[0045] Step 2:

[0046] The device sends the collected biometric information to the server as encrypted data at regular intervals (e.g., every hour). The transmission is performed via a secure protocol, ensuring data security.

[0047] Step 3:

[0048] The server stores the received data in a database for analysis. The stored data is standardized, and outliers are removed through data cleansing.

[0049] Step 4:

[0050] The server analyzes biometric information using generative models. It uses specific algorithms to assess the user's health status and determine whether there are any abnormalities or health risks.

[0051] Step 5:

[0052] Based on the analysis results, the server generates personalized health guidance for the user. For example, it might include suggestions for meals, exercise recommendations, and advice on improving sleep.

[0053] Step 6:

[0054] The server sends the generated health guidance to the user's device. The device notifies the user of this information and allows them to view the details using a dedicated app.

[0055] Step 7:

[0056] The server will send an alert to the user and registered medical institutions if it detects a serious abnormality in the biometric data analysis. The alert will include details of the health risks.

[0057] Step 8:

[0058] When the device receives an alert, it notifies the user with sound and vibration. This allows the user to immediately understand the situation and take appropriate action.

[0059] Step 9:

[0060] Users improve their daily lives based on health guidance provided through their devices. By following the advice, they can strive to maintain their health and prevent illness.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In modern society, lifestyle-related diseases and health problems caused by stress are on the rise, making it crucial for individuals to monitor and manage their own health on a daily basis. However, conventional methods make it difficult to efficiently collect and analyze individual health data and provide specific health guidance. Therefore, there is a need for a system that can easily and effectively collect health information in daily life, and, after expert analysis, provide specific guidance for improving lifestyle habits.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes a device for acquiring biometric information, a device for accumulating the information, and a device for analyzing the information using a generative artificial intelligence model. This enables users to continuously monitor their health status in their daily lives and receive specific health guidance based on rapid analysis.

[0066] A "user" refers to an individual who uses this system to manage their own health status.

[0067] "Biometric information" refers to information that includes data related to life activities, such as an individual's heart rate, steps taken, and sleep patterns.

[0068] "Device" refers to equipment or components that have a function and perform a specific operation or process.

[0069] "Communication methods" refer to the means and protocols used to securely transmit data.

[0070] "Integrated equipment" refers to a computing system for receiving, storing, and analyzing biological information.

[0071] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs specific analyses or predictions.

[0072] "Health status" refers to the overall state of an individual's physical and mental health.

[0073] "Health guidance" refers to advice and plans provided to individuals with the aim of improving or maintaining their health.

[0074] "Abnormal" refers to a condition that exhibits behavior or measurements that deviate from a normal state of health.

[0075] A "warning" refers to a message or signal used to notify a user or relevant organization of an abnormal event and to draw their attention to it.

[0076] A "terminal" refers to an electronic device used by a user to receive and verify information.

[0077] This invention is an information processing system configured to support users in managing their health in their daily lives. The main components of the system include a terminal worn by the user, a server that processes data, and a generative artificial intelligence model that supports them.

[0078] First, the device functions as a device that acquires biometric information such as heart rate, steps taken, and sleep patterns, specifically including smartphones and wearable devices. These devices acquire data from the user using built-in sensors and temporarily store the collected information within the device.

[0079] The acquired biometric information is transmitted to the server in an encrypted state via internet-based communication, either in real time or at specified time intervals. The server receives this data and stores it in a data storage facility. A generative artificial intelligence model on the server is trained using a large amount of health data and is used to analyze the user's health status in detail. This model predicts health risks and determines whether or not there are abnormalities based on changes in heart rate and sleep data.

[0080] Based on the analysis results, the server generates optimized health guidance. This guidance includes dietary suggestions, exercise recommendations, and stress management advice. This information is then transmitted back to the terminal via communication means and notified to the user. Specifically, the user can receive health guidance through an application on the terminal, which helps them review their lifestyle habits.

[0081] For example, if sleep deprivation is detected based on sleep data measured by a user's device, the server can use a generative AI model to generate instructions such as, "To get enough sleep, refrain from using your smartphone before bed," and the user can then create an action plan based on this advice.

[0082] An example of a prompt message would be, "Predict the user's stress level based on heart rate and sleep data, and generate appropriate health advice." This would allow the user to continuously monitor their health status and receive advice in an easy-to-understand format.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] The device uses built-in sensors to acquire biometric information such as the user's heart rate, steps taken, and sleep patterns. Specifically, biometric information is continuously read from the sensors as digital data and stored in the device in real time. The input is real-time data from the sensors, and the output is temporarily stored digital biometric information.

[0086] Step 2:

[0087] The device organizes temporarily stored biometric information after a certain period or when it reaches a specified data volume. It then transmits the data to the server in an encrypted state using an internet connection. The SSL / TLS protocol is used for encryption. The input is temporarily stored biometric data, and the output is encrypted data prepared for transmission.

[0088] Step 3:

[0089] The server receives biometric information transmitted from the terminal and stores it in a database. The received data is formatted and transformed into an analyzable form. The input is encrypted biometric information, and the output is analyzable data stored in the database.

[0090] Step 4:

[0091] The server provides biometric information stored in the database to a generating AI model, which analyzes the user's health status. The generating AI model performs risk assessment and anomaly detection based on past health data. For example, it measures stress levels from patterns of heart rate increase and decrease. The input is analyzable biometric data, and the output is a health status assessment using the AI ​​model.

[0092] Step 5:

[0093] The server generates personalized health guidance based on the analysis results. This guidance includes specific suggestions regarding diet and exercise. The input is the health status assessment results, and the output is health guidance optimized for the user.

[0094] Step 6:

[0095] The server sends the generated health guidance back to the terminal. The terminal notifies the user of the received guidance information using a dedicated application. The terminal uses alert and reminder functions to keep the user informed. The input is the health guidance content, and the output is user instructions via the terminal screen and notification functions.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] In today's busy lifestyle, there is a need to accurately analyze an individual's health status and stress levels in real time and to provide appropriate guidance and safety measures quickly. However, conventional systems struggle to do this efficiently, and are particularly inadequate in addressing the timely management of health risks caused by stress. Therefore, this invention aims to solve these problems and provide a system that supports users in leading a safer and healthier life.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for collecting biometric information from the user, means for transmitting the collected biometric information via a network, means for an external device to analyze the received biometric information using a generative model and evaluate the user's health status, means for providing health guidance to the user based on the analysis results, means for sending warnings to the user and medical institutions if an abnormality is detected, and means for adjusting safety protection measures according to the stress level obtained from the analysis results. This enables real-time monitoring of the user's health status and stress level, and allows for prompt and accurate guidance and protection measures.

[0101] "Biometric information" refers to data collected to understand an individual's health status, and includes heart rate, steps taken, sleep patterns, and other similar data.

[0102] A "network" is a communication infrastructure used to send and receive information, and includes multiple communication technologies such as the internet and wireless communication.

[0103] An "external device" refers to a device that has a hardware or software configuration that receives and analyzes data transmitted from the user's terminal.

[0104] A "generative model" is a program or algorithm built to analyze data using machine learning or artificial intelligence techniques, and is used to predict health conditions and assess risks.

[0105] "Health guidance" refers to specific advice and policies regarding health provided to improve the user's quality of life.

[0106] A "warning" is a message sent to users or medical institutions to alert them when an emergency or abnormality occurs.

[0107] "Safety protection measures" refer to policies and measures implemented to ensure the health and safety of users, based on their health status and stress levels.

[0108] Modes for carrying out the invention

[0109] The system for realizing this application uses devices such as smartphones and smart glasses to generate programs for collecting and analyzing the user's biometric information in real time. These devices incorporate heart rate and accelerometer sensors, which are used to acquire the user's biometric data. The acquired data is then transmitted to a server via the network in an encrypted format.

[0110] The server performs analysis using a generative model based on the received biometric information. The software environment used utilizes TENSORFLOW® as the machine learning library, and a general network service with security considerations is used as the cloud platform. The server uses the generative AI model to evaluate stress levels and health status, and provides health guidance and safety protection measures based on these results.

[0111] Users can view analysis results and guidance sent from the server on their device. For example, if a user's heart rate suddenly increases, the application will issue a warning via voice or vibration and suggest relaxation techniques.

[0112] As a concrete example, when a user on a business trip experiences stress in an unfamiliar environment, the app can detect fluctuations in heart rate and immediately provide guidance such as, "Take a 10-minute meditation to relax; we'll guide you through it within the app." This feature allows users to instantly address health risks and efficiently manage their health.

[0113] Examples of prompt statements are as follows:

[0114] "Design a clever way to use user heart rate data to assess stress levels and automatically strengthen security protocols when stress levels are high."

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The device uses built-in heart rate and accelerometer sensors to collect the user's biometric information in real time. This process acquires data such as heart rate, steps taken, and body movements, and temporarily stores it within the device. The input is biometric data acquired from the sensors, and the output is biometric information in a data format.

[0118] Step 2:

[0119] The device encrypts the collected biometric information and transmits it to the server via the network. Encryption ensures data security during transfer to the cloud server. The input is the unencrypted biometric data, and the output is the encrypted data packet.

[0120] Step 3:

[0121] The server decodes the received biometric information and analyzes the data using a generative AI model. This analysis evaluates the user's health status and stress level. The input is the decoded biometric data, and the output is the result of the health status evaluation.

[0122] Step 4:

[0123] The server generates health guidance and safety measures tailored to the user's health status and stress level based on the analysis results. This guidance includes suggestions for dietary improvements and relaxation methods, and is optimized for the user. The input is the analysis results data, and the output is personalized guidance content.

[0124] Step 5:

[0125] The user receives health guidance transmitted from the server on their device and displays it using a dedicated app. The device also provides voice and vibration notifications to the user to encourage quick confirmation. The input is guidance data from the server, and the output is the message displayed to the user.

[0126] Step 6:

[0127] The server sends a warning to the user and registered healthcare institutions if an anomaly is detected. This warning is intended to respond to changes in health conditions that require immediate attention. The input is the analysis result of the anomaly detection, and the output is the warning message.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] This invention relates to a health management support system that incorporates an emotion engine that collects the user's biometric information and recognizes emotions based on that information. The system aims to acquire the user's biometric information through a smart device and comprehensively evaluate the user's physical and mental health.

[0130] First, the device uses a smartphone or wearable device to acquire biometric information such as heart rate, body temperature, and skin potential response in real time. This information is collected naturally during the user's daily life and temporarily stored on the device.

[0131] Next, the device encrypts the collected biometric information and sends it to a server via the internet. This data is then analyzed by an emotion engine. The emotion engine combines multiple biometric data points to recognize the user's emotional state. For example, if the heart rate is high and the skin potential response is strong, it is determined that the user is experiencing stress or excitement.

[0132] The server comprehensively assesses the user's health status based on biometric information and the results of the emotion engine analysis. This includes predicting health risks and evaluating the impact of emotions on health. For example, if a user experiences high levels of stress over a long period, a risk to their mental health may be identified.

[0133] Based on the analysis results, the server provides users with specific health guidance. This includes advice on stress reduction tailored to their emotional state, suggestions for relaxation-promoting exercises, and suggestions for improving their diet. This guidance is sent to the user's device, and the user can implement the guidance through a dedicated app.

[0134] Furthermore, if the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if an extreme stress state persists, an alert will be issued, and the user will be advised to seek professional medical support as soon as possible.

[0135] Users can utilize this system to better understand the impact of their emotions on their health and adjust their lifestyle based on the guidance provided. For example, the emotional engine can detect work-related stress and provide advice on how to refresh oneself accordingly, enabling users to take actions to maintain their mental and physical health.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The device collects biometric information in real time from the user's smartphone or wearable device. Specifically, it acquires heart rate, body temperature, skin potential response, etc., through sensors and stores the data locally.

[0139] Step 2:

[0140] The device encrypts biometric information collected at regular intervals and sends it to the server. The data is transmitted using a secure protocol, protecting user privacy.

[0141] Step 3:

[0142] The server stores the received biometric information in a database. At this point, the data is cleansed, and missing or outlier values ​​are removed.

[0143] Step 4:

[0144] The server uses an emotion engine to recognize the user's emotions from biometric data. For example, if a high heart rate and a strong skin potential response are detected simultaneously, the server determines that the user is experiencing stress.

[0145] Step 5:

[0146] The server comprehensively evaluates the user's health status based on analysis results, including emotional state. This includes taking into account the impact of emotions such as stress and anxiety on health.

[0147] Step 6:

[0148] The server generates personalized health guidance based on the analysis results. It creates advice on stress reduction exercises and mental care based on the user's emotional state.

[0149] Step 7:

[0150] The server sends the generated health guidance to the user's device, and the device notifies the user. Through a dedicated app, the user can review and implement the detailed guidance.

[0151] Step 8:

[0152] If the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if prolonged high stress is detected, action will be prompted.

[0153] Step 9:

[0154] Users will work to improve their daily lives based on health guidance provided by their devices. Specific actions include activities to reduce stress and plans for getting adequate rest.

[0155] (Example 2)

[0156] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0157] In modern times, users are required to accurately understand their own emotions and physical health and to manage their health appropriately. However, conventional technology has faced the challenge of not being able to effectively and immediately assess the situation and provide appropriate guidance. This challenge makes it difficult for users to foresee health risks and to mitigate long-term health impacts.

[0158] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0159] In this invention, the server includes a device for acquiring information about the user's physical condition, a device for encoding the acquired information and transmitting it to a remote computing device, and a device for the computing device to analyze the received physical condition information using a generative model and estimate the emotional state. This makes it possible to comprehensively and in real time evaluate the user's emotional state and health condition and provide appropriate health guidance.

[0160] "Information regarding the user's physical condition" refers to data that indicates the user's physiological state, such as heart rate, body temperature, and skin potential response.

[0161] "Acquisition devices" refer to devices such as smartphones and wearable devices that use sensors to collect the user's biometric information.

[0162] "Encoding" is an encryption process used to securely transmit collected biometric information.

[0163] A "remote computing device" refers to a server or data processing system that communicates via the internet to process data sent from a user terminal.

[0164] A "generative model" is an algorithm that uses artificial intelligence or machine learning to analyze a user's biometric information.

[0165] A "device for estimating emotional state" is equipment or a function that uses an emotion engine to process a large amount of biometric information and identify the user's emotions.

[0166] "Health guidance" refers to specific advice provided to users based on analysis results, aimed at maintaining or improving their health.

[0167] A "warning" is an alert message sent to a user when an abnormal emotion or health condition is detected.

[0168] A "healthcare provider" is an institution or professional that provides health management or medical care.

[0169] This invention is a system for evaluating a user's emotions and health status and supporting health management. The embodiments of this system are described in detail below.

[0170] The device uses a smartphone or wearable device to collect the user's biometric information, such as heart rate, body temperature, and skin potential response. These devices use sensors to acquire data in real time and convert it to a standard format through built-in software. The collected data is temporarily stored in the device's memory.

[0171] The device encrypts the collected biometric information using the AES encryption algorithm and sends it to the server via the HTTPS protocol. This method ensures secure data transfer while protecting user privacy.

[0172] After decoding the received data, the server analyzes the biometric information using a generative AI model. The emotion engine uses machine learning algorithms to estimate the user's emotional state and calculates an emotion score from multiple biometric data points. For example, if the heart rate is elevated and the skin potential response is strong, it is determined that the user is in a stressed state.

[0173] The server comprehensively assesses the user's health status based on estimated emotional states and analysis results from a generative AI model. It predicts health risks and evaluates the physical impact of emotions, generating health guidance as needed. For example, if a prolonged high-stress emotional state is detected, relaxation methods and stress management advice will be provided.

[0174] Based on the analysis results, the server sends health guidance to the user's device, which the user can view and implement through a dedicated app. If an abnormal emotional state is detected, a warning is immediately displayed on the user's device, and healthcare providers are notified if necessary.

[0175] This system allows users to gain a deeper understanding of how their emotional state impacts their health and improve their lifestyle based on the advice provided. For example, a user could input a command into the AI ​​model such as, "Please tell me how to provide stress management advice based on the user's emotional state," and use this information for daily health management.

[0176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0177] Step 1:

[0178] The device uses sensors from smartphones or wearable devices to acquire biometric information such as the user's heart rate, body temperature, and skin potential response. The input is these physiological indicators, and the output is the collected biometric data. The device formats this data using a dedicated application and temporarily stores it in memory.

[0179] Step 2:

[0180] The device encrypts the collected biometric information using the AES encryption algorithm and prepares it for secure data transfer. The input is formatted biometric data, and the output is encrypted data. This encrypted data is sent to the server via the HTTPS protocol.

[0181] Step 3:

[0182] The server receives encrypted data sent from the terminal and decrypts it. The input is encrypted data, and the output is raw biometric data. Next, the server uses a generated AI model to analyze the biometric data.

[0183] Step 4:

[0184] The server uses a generative AI model to estimate the user's emotional state. The input is decoded biometric data, and the output is a score or category indicating the emotional state. For example, a high heart rate and a large skin potential response would be classified as a stressed state. This analysis applies machine learning algorithms, and the model identifies emotions by comparing them with the data it was trained on.

[0185] Step 5:

[0186] The server comprehensively assesses the user's health status based on analyzed emotional states and health evaluations, and generates health guidance. The input is an emotional state score and associated biometric data, and the output is health guidance advice. For example, it can create documents containing relaxation suggestions and stress reduction methods.

[0187] Step 6:

[0188] The server sends the generated health guidance to the terminal, and the user receives the evaluation results and guidance content through a dedicated app. The input is health guidance advice, and the output is the evaluation content obtained by the user through the app. The user can use this information to work towards improving their lifestyle.

[0189] Step 7:

[0190] The server immediately sends a warning to the user's terminal if an abnormal emotional state is detected. The input is the result of the detected abnormal emotional state, and the output is the warning notification received by the user. If necessary, healthcare providers are also notified and support is provided for the user to take corrective action.

[0191] (Application Example 2)

[0192] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0193] In physical stores, there is a need to improve customer satisfaction by accurately understanding the emotional state of customers and providing optimal service based on that understanding. Traditional methods often rely on customer service staff to intuitively guess customer emotions, which has limitations in accuracy. Furthermore, there is currently a lack of systems to provide appropriate service at the right time, tailored to the customer's state.

[0194] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0195] In this invention, the server includes a device means for collecting biometric information from a user, a device means for transmitting the collected biometric information to an information processing device, and a device means for analyzing the biometric information of a customer and analyzing their emotional state. This makes it possible to analyze the emotional state of customers in a physical store in real time and provide optimal service based on the results.

[0196] A "user" refers to an individual who provides biometric information and receives health guidance and services from the system.

[0197] "Biometric information" is a general term for data that indicates the user's physical condition, such as heart rate, body temperature, and skin potential response.

[0198] "Device means" refers to a system that combines hardware and software for collecting, transmitting, and analyzing a user's biometric information.

[0199] An "information processing device" refers to a computing device that analyzes biometric information received from a user and evaluates their health and emotional state using a generative machine learning model.

[0200] A "generative machine learning model" refers to a data analysis algorithm used to analyze a user's health and emotional state based on received biometric information.

[0201] An "alert" refers to a notification issued to users or, if necessary, to draw attention to or alert an abnormality.

[0202] "Visitors" refers to customers who visit a physical store and are the recipients of services based on their biometric information.

[0203] "Emotional state" refers to the psychological and emotional state analyzed based on the user's physical and mental responses.

[0204] To implement this invention, the following system is necessary. First, the user wears a device such as a wearable sensor or smart glasses to collect biometric information such as heart rate, body temperature, and skin potential response in real time. The device also temporarily stores this biometric data, encrypts it, and transmits it to a server via the internet.

[0205] The server uses a generative machine learning model to analyze the received biometric information. This model utilizes a mature machine learning library such as TensorFlow. Based on these analyses, the server comprehensively assesses the user's health status, particularly identifying their emotional state. The analysis results are then sent to the device as specific health guidance. This guidance includes advice on stress-relieving activities and diet.

[0206] Furthermore, the server performs emotion analysis, which is also useful in physical stores, to understand the emotional state of customers in the store. Based on this information, sales staff and other staff members can be given suggestions for customer service methods tailored to each individual customer. As an example of this implementation, if a customer is determined to be tense, the staff can suggest products that have a relaxing effect based on the analysis results.

[0207] As a concrete example, a possible prompt message could be something like, "Please select an appropriate machine learning model to obtain the customer's heart rate data and analyze their current emotional state," which could be input to a generative AI model. This prompt would allow the server to perform a rapid and accurate analysis, enabling real-time service improvements.

[0208] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0209] Step 1:

[0210] The device acquires the user's biometric information, such as heart rate, body temperature, and skin potential response, in real time from wearable sensors and temporarily stores it in local storage. This process periodically samples data from the sensors and converts it into a data processing format (e.g., JSON). The input is analog signals from each biosensor, and the output is digital data.

[0211] Step 2:

[0212] The device encrypts the stored biometric information using encryption technology (e.g., AES encryption) and sends it to the server via the internet. The input is locally stored biometric data, and the output is encrypted data.

[0213] Step 3:

[0214] The server decrypts the encrypted data received from the terminal, performs the necessary preprocessing for formatting, and then prepares it for analysis. The input is encrypted biometric data, and the output is the decrypted data. In this step, the raw data is normalized and filtered for analysis.

[0215] Step 4:

[0216] The server analyzes decoded data using a generative AI model (e.g., a machine learning algorithm) to evaluate the user's health and emotional state. The input is decoded biometric data, and the output is a health assessment and emotional analysis result. This analysis detects abnormal indicators and emotional changes.

[0217] Step 5:

[0218] The server generates health guidance for the user based on the analysis results and sends it to the terminal. The input is a health assessment and emotion analysis results, and the output is specific health guidance content. As an implementation example, it generates recommendations for relaxation methods and dietary improvement advice.

[0219] Step 6:

[0220] In physical stores, the server analyzes the customer's emotional state and notifies store staff in real time, suggesting customer service based on that analysis as needed. The input is the emotional assessment result, and the output is a customer service suggestion. For example, if tension is detected in a customer, a notification is sent suggesting products that have a relaxing effect.

[0221] Step 7:

[0222] Users follow the health guidance received from their device and make lifestyle changes as needed. They also receive alerts if abnormalities are detected, allowing them to take corrective action. Inputs are health guidance and alerts, while output is the user's behavioral changes.

[0223] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0224] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0225] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0226] [Second Embodiment]

[0227] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0228] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0229] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0230] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0231] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0232] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0233] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0234] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0235] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0236] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0237] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0238] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0239] This invention is a system that supports users' health management using smartphones and wearable devices. This system monitors and provides guidance on the user's health status based on biometric information that the user continuously acquires in their daily life.

[0240] First, the device uses built-in sensors to acquire biometric information, such as the user's heart rate, steps taken, and sleep patterns. This information is recorded in real time or at specified time intervals and temporarily stored within the device.

[0241] The device then transmits biometric information to a server via the internet. This communication is encrypted, ensuring the security of the transmitted data. At this point, the user's lifestyle and past health data are also stored in a database, forming a dataset that forms the basis for analysis.

[0242] The server processes the received biometric information and uses a generative model to analyze the user's current health status. The model is trained on previously collected health data and can predict health risks and detect anomalies. For example, it can estimate stress levels from heart rate variability patterns or warn of sleep deprivation.

[0243] Based on the analysis results, the server generates personalized health guidance for each user. This guidance includes, for example, suggestions for dietary improvements, exercise recommendations, and advice for maintaining mental health. The generated guidance is sent to the device, allowing users to receive alert notifications or check it through a dedicated app.

[0244] Furthermore, if a serious abnormality is detected, the server immediately sends an alert to the user and registered medical institutions. Upon receiving the alert, the terminal notifies the user via voice and vibration to encourage a quick response. This function enables early detection of illness and emergency response.

[0245] Through this system, users can monitor their daily health status and improve their lifestyle habits according to the guidance provided. This allows users to proactively maintain their health, reduce the frequency of medical visits, and aim to extend their healthy lifespan. For example, if a user is told that their sleep quality is deteriorating, they can create an action plan to develop better sleep habits and work towards improvement.

[0246] The following describes the processing flow.

[0247] Step 1:

[0248] The device uses the user's smartphone or wearable device to collect biometric information. Information such as heart rate, steps taken, and sleep patterns is acquired in real time and temporarily stored in local storage.

[0249] Step 2:

[0250] The device sends the collected biometric information to the server as encrypted data at regular intervals (e.g., every hour). The transmission is performed via a secure protocol, ensuring data security.

[0251] Step 3:

[0252] The server stores the received data in a database for analysis. The stored data is standardized, and outliers are removed through data cleansing.

[0253] Step 4:

[0254] The server analyzes biometric information using generative models. It uses specific algorithms to assess the user's health status and determine whether there are any abnormalities or health risks.

[0255] Step 5:

[0256] Based on the analysis results, the server generates personalized health guidance for the user. For example, it might include suggestions for meals, exercise recommendations, and advice on improving sleep.

[0257] Step 6:

[0258] The server sends the generated health guidance to the user's device. The device notifies the user of this information and allows them to view the details using a dedicated app.

[0259] Step 7:

[0260] The server will send an alert to the user and registered medical institutions if it detects a serious abnormality in the biometric data analysis. The alert will include details of the health risks.

[0261] Step 8:

[0262] When the device receives an alert, it notifies the user with sound and vibration. This allows the user to immediately understand the situation and take appropriate action.

[0263] Step 9:

[0264] Users improve their daily lives based on health guidance provided through their devices. By following the advice, they can strive to maintain their health and prevent illness.

[0265] (Example 1)

[0266] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0267] In modern society, lifestyle-related diseases and health problems caused by stress are on the rise, making it crucial for individuals to monitor and manage their own health on a daily basis. However, conventional methods make it difficult to efficiently collect and analyze individual health data and provide specific health guidance. Therefore, there is a need for a system that can easily and effectively collect health information in daily life, and, after expert analysis, provide specific guidance for improving lifestyle habits.

[0268] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0269] In this invention, the server includes a device for acquiring biometric information, a device for accumulating the information, and a device for analyzing the information using a generative artificial intelligence model. This enables users to continuously monitor their health status in their daily lives and receive specific health guidance based on rapid analysis.

[0270] A "user" refers to an individual who uses this system to manage their own health status.

[0271] "Biometric information" refers to information that includes data related to life activities, such as an individual's heart rate, steps taken, and sleep patterns.

[0272] "Device" refers to equipment or components that have a function and perform a specific operation or process.

[0273] "Communication methods" refer to the means and protocols used to securely transmit data.

[0274] "Integrated equipment" refers to a computing system for receiving, storing, and analyzing biological information.

[0275] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs specific analyses or predictions.

[0276] "Health status" refers to the overall state of an individual's physical and mental health.

[0277] "Health guidance" refers to advice and plans provided to individuals with the aim of improving or maintaining their health.

[0278] "Abnormal" refers to a condition that exhibits behavior or measurements that deviate from a normal state of health.

[0279] A "warning" refers to a message or signal used to notify a user or relevant organization of an abnormal event and to draw their attention to it.

[0280] The "terminal" refers to an electronic device for a user to receive and view information.

[0281] This invention is an information processing system configured to assist in the health management in a user's daily life. As the main components of the system, it includes a terminal worn by the user, a server for processing data, and a generative artificial intelligence model to support them.

[0282] First, the terminal, specifically such as a smartphone or a wearable device, functions as a device for acquiring biometric information such as heart rate, number of steps, and sleep pattern. These terminals acquire data from the user by built-in sensors and temporarily store the collected information in the terminal.

[0283] The acquired biometric information is transmitted to the server in an encrypted state in real time or at specified time intervals by means of communication using an Internet connection. The server receives this and stores the data in the data integration facility. And the generative artificial intelligence model on the server is trained using a large amount of health data and is used to analyze the user's health condition in detail. This model predicts health risks and determines the presence or absence of abnormalities based on changes in heart rate and sleep data.

[0284] Based on the analysis results, the server generates optimized health guidance. This guidance includes diet suggestions, exercise recommendations, stress management advice, etc. This information is transmitted to the terminal again using the communication means and notified to the user. Specifically, the user can receive health guidance through the application on the terminal, which helps to review their own lifestyle.

[0285] B As an example, when it is detected from the sleep data measured by the terminal that a certain user is sleep deprived, the server uses the generative AI model to generate an instruction such as "To get enough sleep, refrain from using the smartphone before going to bed", and the user can make an action plan based on this advice.

[0286] As an example of a prompt sentence, "Predict the user's stress level based on heart rate and sleep data and generate appropriate health advice." can be considered. This enables the user to continuously monitor their health status and receive advice in an understandable format.

[0287] The flow of the specific process in Example 1 will be described using FIG. 11.

[0288] Step 1:

[0289] The terminal acquires biometric information such as the user's heart rate, step count, and sleep pattern using built-in sensors. As a specific operation, the biometric information is continuously read from the sensors as digital data and stored in the terminal in real time. The input is the real-time data of the sensors, and the output is the temporarily stored digital biometric information.

[0290] Step 2:

[0291] The terminal organizes the temporarily stored biometric information when a certain period of time has passed or when a specified amount of data has been reached. Then, it uses an Internet connection to transmit it to the server in an encrypted state. At this time, the SSL / TLS protocol is used for encryption. The input is the temporarily stored biometric data, and the output is the data encrypted for data transmission preparation.

[0292] Step 3:

[0293] The server receives the biometric information transmitted from the terminal and stores it in the database. The received data is formatted and shaped into an analyzable form. The input is the encrypted biometric information, and the output is the analyzable data stored in the database.

[0294] Step 4:

[0295] The server provides biometric information stored in the database to a generating AI model, which analyzes the user's health status. The generating AI model performs risk assessment and anomaly detection based on past health data. For example, it measures stress levels from patterns of heart rate increase and decrease. The input is analyzable biometric data, and the output is a health status assessment using the AI ​​model.

[0296] Step 5:

[0297] The server generates personalized health guidance based on the analysis results. This guidance includes specific suggestions regarding diet and exercise. The input is the health status assessment results, and the output is health guidance optimized for the user.

[0298] Step 6:

[0299] The server sends the generated health guidance back to the terminal. The terminal notifies the user of the received guidance information using a dedicated application. The terminal uses alert and reminder functions to keep the user informed. The input is the health guidance content, and the output is user instructions via the terminal screen and notification functions.

[0300] (Application Example 1)

[0301] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0302] In today's busy lifestyle, there is a need to accurately analyze an individual's health status and stress levels in real time and to provide appropriate guidance and safety measures quickly. However, conventional systems struggle to do this efficiently, and are particularly inadequate in addressing the timely management of health risks caused by stress. Therefore, this invention aims to solve these problems and provide a system that supports users in leading a safer and healthier life.

[0303] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 1 is realized by the following means.

[0304] In this invention, the server includes means for collecting biometric information from a user, means for transmitting the collected biometric information via a network, means for analyzing the biometric information received by an external device using a generation model and evaluating the health status, means for providing health guidance to the user based on the analysis result, means for transmitting a warning to the user and a medical institution when an abnormality is detected, and means for adjusting safety protection measures according to the stress level obtained from the analysis result. Thereby, it becomes possible to monitor the health status and stress level of the user in real time, and to provide prompt and accurate guidance and protection measures.

[0305] "Biometric information" is data collected to grasp an individual's health status, and includes items such as heart rate, number of steps, and sleep pattern.

[0306] "Network" is a communication infrastructure used for transmitting and receiving information, and includes multiple communication technologies such as the Internet and wireless communication.

[0307] "External device" refers to a device having a hardware or software configuration that receives data transmitted from a user's terminal and performs analysis.

[0308] "Generation model" is a program or algorithm constructed for analyzing data using machine learning or artificial intelligence techniques, and is used for predicting the health status and risk assessment.

[0309] "Health guidance" refers to specific health-related advice and guidelines provided to improve the quality of a user's life.

[0310] "Warning" is a message for alerting sent to a user or a medical institution when an emergency or abnormality occurs.

[0311] "Safety protection measures" refer to policies and measures implemented to ensure the health and safety of users, based on their health status and stress levels.

[0312] Modes for carrying out the invention

[0313] The system for realizing this application uses devices such as smartphones and smart glasses to generate programs for collecting and analyzing the user's biometric information in real time. These devices incorporate heart rate and accelerometer sensors, which are used to acquire the user's biometric data. The acquired data is then transmitted to a server via the network in an encrypted format.

[0314] The server performs analysis using a generative model based on the received biometric information. The software environment used utilizes TensorFlow as the machine learning library, and a secure, general network service is used as the cloud platform. The server leverages the generative AI model to evaluate stress levels and health status, and provides health guidance and safety protection measures based on these results.

[0315] Users can view analysis results and guidance sent from the server on their device. For example, if a user's heart rate suddenly increases, the application will issue a warning via voice or vibration and suggest relaxation techniques.

[0316] As a concrete example, when a user on a business trip experiences stress in an unfamiliar environment, the app can detect fluctuations in heart rate and immediately provide guidance such as, "Take a 10-minute meditation to relax; we'll guide you through it within the app." This feature allows users to instantly address health risks and efficiently manage their health.

[0317] Examples of prompt statements are as follows:

[0318] "Design a clever way to use user heart rate data to assess stress levels and automatically strengthen security protocols when stress levels are high."

[0319] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0320] Step 1:

[0321] The device uses built-in heart rate and accelerometer sensors to collect the user's biometric information in real time. This process acquires data such as heart rate, steps taken, and body movements, and temporarily stores it within the device. The input is biometric data acquired from the sensors, and the output is biometric information in a data format.

[0322] Step 2:

[0323] The device encrypts the collected biometric information and transmits it to the server via the network. Encryption ensures data security during transfer to the cloud server. The input is the unencrypted biometric data, and the output is the encrypted data packet.

[0324] Step 3:

[0325] The server decodes the received biometric information and analyzes the data using a generative AI model. This analysis evaluates the user's health status and stress level. The input is the decoded biometric data, and the output is the result of the health status evaluation.

[0326] Step 4:

[0327] The server generates health guidance and safety measures tailored to the user's health status and stress level based on the analysis results. This guidance includes suggestions for dietary improvements and relaxation methods, and is optimized for the user. The input is the analysis results data, and the output is personalized guidance content.

[0328] Step 5:

[0329] The user receives health guidance transmitted from the server on their device and displays it using a dedicated app. The device also provides voice and vibration notifications to the user to encourage quick confirmation. The input is guidance data from the server, and the output is the message displayed to the user.

[0330] Step 6:

[0331] The server sends a warning to the user and registered healthcare institutions if an anomaly is detected. This warning is intended to respond to changes in health conditions that require immediate attention. The input is the analysis result of the anomaly detection, and the output is the warning message.

[0332] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0333] This invention relates to a health management support system that incorporates an emotion engine that collects the user's biometric information and recognizes emotions based on that information. The system aims to acquire the user's biometric information through a smart device and comprehensively evaluate the user's physical and mental health.

[0334] First, the device uses a smartphone or wearable device to acquire biometric information such as heart rate, body temperature, and skin potential response in real time. This information is collected naturally during the user's daily life and temporarily stored on the device.

[0335] Next, the device encrypts the collected biometric information and sends it to a server via the internet. This data is then analyzed by an emotion engine. The emotion engine combines multiple biometric data points to recognize the user's emotional state. For example, if the heart rate is high and the skin potential response is strong, it is determined that the user is experiencing stress or excitement.

[0336] The server comprehensively assesses the user's health status based on biometric information and the results of the emotion engine analysis. This includes predicting health risks and evaluating the impact of emotions on health. For example, if a user experiences high levels of stress over a long period, a risk to their mental health may be identified.

[0337] Based on the analysis results, the server provides users with specific health guidance. This includes advice on stress reduction tailored to their emotional state, suggestions for relaxation-promoting exercises, and suggestions for improving their diet. This guidance is sent to the user's device, and the user can implement the guidance through a dedicated app.

[0338] Furthermore, if the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if an extreme stress state persists, an alert will be issued, and the user will be advised to seek professional medical support as soon as possible.

[0339] Users can utilize this system to better understand the impact of their emotions on their health and adjust their lifestyle based on the guidance provided. For example, the emotional engine can detect work-related stress and provide advice on how to refresh oneself accordingly, enabling users to take actions to maintain their mental and physical health.

[0340] The following describes the processing flow.

[0341] Step 1:

[0342] The device collects biometric information in real time from the user's smartphone or wearable device. Specifically, it acquires heart rate, body temperature, skin potential response, etc., through sensors and stores the data locally.

[0343] Step 2:

[0344] The device encrypts biometric information collected at regular intervals and sends it to the server. The data is transmitted using a secure protocol, protecting user privacy.

[0345] Step 3:

[0346] The server stores the received biometric information in a database. At this point, the data is cleansed, and missing or outlier values ​​are removed.

[0347] Step 4:

[0348] The server uses an emotion engine to recognize the user's emotions from biometric data. For example, if a high heart rate and a strong skin potential response are detected simultaneously, the server determines that the user is experiencing stress.

[0349] Step 5:

[0350] The server comprehensively evaluates the user's health status based on analysis results, including emotional state. This includes taking into account the impact of emotions such as stress and anxiety on health.

[0351] Step 6:

[0352] The server generates personalized health guidance based on the analysis results. It creates advice on stress reduction exercises and mental care based on the user's emotional state.

[0353] Step 7:

[0354] The server sends the generated health guidance to the user's device, and the device notifies the user. Through a dedicated app, the user can review and implement the detailed guidance.

[0355] Step 8:

[0356] If the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if prolonged high stress is detected, action will be prompted.

[0357] Step 9:

[0358] Users will work to improve their daily lives based on health guidance provided by their devices. Specific actions include activities to reduce stress and plans for getting adequate rest.

[0359] (Example 2)

[0360] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0361] In modern times, users are required to accurately understand their own emotions and physical health and to manage their health appropriately. However, conventional technology has faced the challenge of not being able to effectively and immediately assess the situation and provide appropriate guidance. This challenge makes it difficult for users to foresee health risks and to mitigate long-term health impacts.

[0362] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0363] In this invention, the server includes a device for acquiring information about the user's physical condition, a device for encoding the acquired information and transmitting it to a remote computing device, and a device for the computing device to analyze the received physical condition information using a generative model and estimate the emotional state. This makes it possible to comprehensively and in real time evaluate the user's emotional state and health condition and provide appropriate health guidance.

[0364] "Information regarding the user's physical condition" refers to data that indicates the user's physiological state, such as heart rate, body temperature, and skin potential response.

[0365] "Acquisition devices" refer to devices such as smartphones and wearable devices that use sensors to collect the user's biometric information.

[0366] "Encoding" is an encryption process used to securely transmit collected biometric information.

[0367] A "remote computing device" refers to a server or data processing system that communicates via the internet to process data sent from a user terminal.

[0368] A "generative model" is an algorithm that uses artificial intelligence or machine learning to analyze a user's biometric information.

[0369] A "device for estimating emotional state" is equipment or a function that uses an emotion engine to process a large amount of biometric information and identify the user's emotions.

[0370] "Health guidance" refers to specific advice provided to users based on analysis results, aimed at maintaining or improving their health.

[0371] A "warning" is an alert message sent to a user when an abnormal emotion or health condition is detected.

[0372] A "healthcare provider" is an institution or professional that provides health management or medical care.

[0373] This invention is a system for evaluating a user's emotions and health status and supporting health management. The embodiments of this system are described in detail below.

[0374] The device uses a smartphone or wearable device to collect the user's biometric information, such as heart rate, body temperature, and skin potential response. These devices use sensors to acquire data in real time and convert it to a standard format through built-in software. The collected data is temporarily stored in the device's memory.

[0375] The device encrypts the collected biometric information using the AES encryption algorithm and sends it to the server via the HTTPS protocol. This method ensures secure data transfer while protecting user privacy.

[0376] After decoding the received data, the server analyzes the biometric information using a generative AI model. The emotion engine uses machine learning algorithms to estimate the user's emotional state and calculates an emotion score from multiple biometric data points. For example, if the heart rate is elevated and the skin potential response is strong, it is determined that the user is in a stressed state.

[0377] The server comprehensively assesses the user's health status based on estimated emotional states and analysis results from a generative AI model. It predicts health risks and evaluates the physical impact of emotions, generating health guidance as needed. For example, if a prolonged high-stress emotional state is detected, relaxation methods and stress management advice will be provided.

[0378] Based on the analysis results, the server sends health guidance to the user's device, which the user can view and implement through a dedicated app. If an abnormal emotional state is detected, a warning is immediately displayed on the user's device, and healthcare providers are notified if necessary.

[0379] This system allows users to gain a deeper understanding of how their emotional state impacts their health and improve their lifestyle based on the advice provided. For example, a user could input a command into the AI ​​model such as, "Please tell me how to provide stress management advice based on the user's emotional state," and use this information for daily health management.

[0380] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0381] Step 1:

[0382] The device uses sensors from smartphones or wearable devices to acquire biometric information such as the user's heart rate, body temperature, and skin potential response. The input is these physiological indicators, and the output is the collected biometric data. The device formats this data using a dedicated application and temporarily stores it in memory.

[0383] Step 2:

[0384] The device encrypts the collected biometric information using the AES encryption algorithm and prepares it for secure data transfer. The input is formatted biometric data, and the output is encrypted data. This encrypted data is sent to the server via the HTTPS protocol.

[0385] Step 3:

[0386] The server receives encrypted data sent from the terminal and decrypts it. The input is encrypted data, and the output is raw biometric data. Next, the server uses a generated AI model to analyze the biometric data.

[0387] Step 4:

[0388] The server uses a generative AI model to estimate the user's emotional state. The input is decoded biometric data, and the output is a score or category indicating the emotional state. For example, a high heart rate and a large skin potential response would be classified as a stressed state. This analysis applies machine learning algorithms, and the model identifies emotions by comparing them with the data it was trained on.

[0389] Step 5:

[0390] The server comprehensively assesses the user's health status based on analyzed emotional states and health evaluations, and generates health guidance. The input is an emotional state score and associated biometric data, and the output is health guidance advice. For example, it can create documents containing relaxation suggestions and stress reduction methods.

[0391] Step 6:

[0392] The server sends the generated health guidance to the terminal, and the user receives the evaluation results and guidance content through a dedicated app. The input is health guidance advice, and the output is the evaluation content obtained by the user through the app. The user can use this information to work towards improving their lifestyle.

[0393] Step 7:

[0394] The server immediately sends a warning to the user's terminal if an abnormal emotional state is detected. The input is the result of the detected abnormal emotional state, and the output is the warning notification received by the user. If necessary, healthcare providers are also notified and support is provided for the user to take corrective action.

[0395] (Application Example 2)

[0396] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0397] In physical stores, there is a need to improve customer satisfaction by accurately understanding the emotional state of customers and providing optimal service based on that understanding. Traditional methods often rely on customer service staff to intuitively guess customer emotions, which has limitations in accuracy. Furthermore, there is currently a lack of systems to provide appropriate service at the right time, tailored to the customer's state.

[0398] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0399] In this invention, the server includes a device means for collecting biometric information from a user, a device means for transmitting the collected biometric information to an information processing device, and a device means for analyzing the biometric information of a customer and analyzing their emotional state. This makes it possible to analyze the emotional state of customers in a physical store in real time and provide optimal service based on the results.

[0400] A "user" refers to an individual who provides biometric information and receives health guidance and services from the system.

[0401] "Biometric information" is a general term for data that indicates the user's physical condition, such as heart rate, body temperature, and skin potential response.

[0402] "Device means" refers to a system that combines hardware and software for collecting, transmitting, and analyzing a user's biometric information.

[0403] An "information processing device" refers to a computing device that analyzes biometric information received from a user and evaluates their health and emotional state using a generative machine learning model.

[0404] A "generative machine learning model" refers to a data analysis algorithm used to analyze a user's health and emotional state based on received biometric information.

[0405] An "alert" refers to a notification issued to users or, if necessary, to draw attention to or alert an abnormality.

[0406] "Visitors" refers to customers who visit a physical store and are the recipients of services based on their biometric information.

[0407] "Emotional state" refers to the psychological and emotional state analyzed based on the user's physical and mental responses.

[0408] To implement this invention, the following system is necessary. First, the user wears a device such as a wearable sensor or smart glasses to collect biometric information such as heart rate, body temperature, and skin potential response in real time. The device also temporarily stores this biometric data, encrypts it, and transmits it to a server via the internet.

[0409] The server uses a generative machine learning model to analyze the received biometric information. This model utilizes a mature machine learning library such as TensorFlow. Based on these analyses, the server comprehensively assesses the user's health status, particularly identifying their emotional state. The analysis results are then sent to the device as specific health guidance. This guidance includes advice on stress-relieving activities and diet.

[0410] Furthermore, the server performs emotion analysis, which is also useful in physical stores, to understand the emotional state of customers in the store. Based on this information, sales staff and other staff members can be given suggestions for customer service methods tailored to each individual customer. As an example of this implementation, if a customer is determined to be tense, the staff can suggest products that have a relaxing effect based on the analysis results.

[0411] As a concrete example, a possible prompt message could be something like, "Please select an appropriate machine learning model to obtain the customer's heart rate data and analyze their current emotional state," which could be input to a generative AI model. This prompt would allow the server to perform a rapid and accurate analysis, enabling real-time service improvements.

[0412] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0413] Step 1:

[0414] The device acquires the user's biometric information, such as heart rate, body temperature, and skin potential response, in real time from wearable sensors and temporarily stores it in local storage. This process periodically samples data from the sensors and converts it into a data processing format (e.g., JSON). The input is analog signals from each biosensor, and the output is digital data.

[0415] Step 2:

[0416] The device encrypts the stored biometric information using encryption technology (e.g., AES encryption) and sends it to the server via the internet. The input is locally stored biometric data, and the output is encrypted data.

[0417] Step 3:

[0418] The server decrypts the encrypted data received from the terminal, performs the necessary preprocessing for formatting, and then prepares it for analysis. The input is encrypted biometric data, and the output is the decrypted data. In this step, the raw data is normalized and filtered for analysis.

[0419] Step 4:

[0420] The server analyzes decoded data using a generative AI model (e.g., a machine learning algorithm) to evaluate the user's health and emotional state. The input is decoded biometric data, and the output is a health assessment and emotional analysis result. This analysis detects abnormal indicators and emotional changes.

[0421] Step 5:

[0422] The server generates health guidance for the user based on the analysis results and sends it to the terminal. The input is a health assessment and emotion analysis results, and the output is specific health guidance content. As an implementation example, it generates recommendations for relaxation methods and dietary improvement advice.

[0423] Step 6:

[0424] In physical stores, the server analyzes the customer's emotional state and notifies store staff in real time, suggesting customer service based on that analysis as needed. The input is the emotional assessment result, and the output is a customer service suggestion. For example, if tension is detected in a customer, a notification is sent suggesting products that have a relaxing effect.

[0425] Step 7:

[0426] Users follow the health guidance received from their device and make lifestyle changes as needed. They also receive alerts if abnormalities are detected, allowing them to take corrective action. Inputs are health guidance and alerts, while output is the user's behavioral changes.

[0427] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0428] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0429] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0430] [Third Embodiment]

[0431] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0432] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0433] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0434] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0435] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0436] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0437] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0438] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0439] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0440] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0441] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0442] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0443] This invention is a system that supports users' health management using smartphones and wearable devices. This system monitors and provides guidance on the user's health status based on biometric information that the user continuously acquires in their daily life.

[0444] First, the device uses built-in sensors to acquire biometric information, such as the user's heart rate, steps taken, and sleep patterns. This information is recorded in real time or at specified time intervals and temporarily stored within the device.

[0445] The device then transmits biometric information to a server via the internet. This communication is encrypted, ensuring the security of the transmitted data. At this point, the user's lifestyle and past health data are also stored in a database, forming a dataset that forms the basis for analysis.

[0446] The server processes the received biometric information and uses a generative model to analyze the user's current health status. The model is trained on previously collected health data and can predict health risks and detect anomalies. For example, it can estimate stress levels from heart rate variability patterns or warn of sleep deprivation.

[0447] Based on the analysis results, the server generates personalized health guidance for each user. This guidance includes, for example, suggestions for dietary improvements, exercise recommendations, and advice for maintaining mental health. The generated guidance is sent to the device, allowing users to receive alert notifications or check it through a dedicated app.

[0448] Furthermore, if a serious abnormality is detected, the server immediately sends an alert to the user and registered medical institutions. Upon receiving the alert, the terminal notifies the user via voice and vibration to encourage a quick response. This function enables early detection of illness and emergency response.

[0449] Through this system, users can monitor their daily health status and improve their lifestyle habits according to the guidance provided. This allows users to proactively maintain their health, reduce the frequency of medical visits, and aim to extend their healthy lifespan. For example, if a user is told that their sleep quality is deteriorating, they can create an action plan to develop better sleep habits and work towards improvement.

[0450] The following describes the processing flow.

[0451] Step 1:

[0452] The device uses the user's smartphone or wearable device to collect biometric information. Information such as heart rate, steps taken, and sleep patterns is acquired in real time and temporarily stored in local storage.

[0453] Step 2:

[0454] The device sends the collected biometric information to the server as encrypted data at regular intervals (e.g., every hour). The transmission is performed via a secure protocol, ensuring data security.

[0455] Step 3:

[0456] The server stores the received data in a database for analysis. The stored data is standardized, and outliers are removed through data cleansing.

[0457] Step 4:

[0458] The server analyzes biometric information using generative models. It uses specific algorithms to assess the user's health status and determine whether there are any abnormalities or health risks.

[0459] Step 5:

[0460] Based on the analysis results, the server generates personalized health guidance for the user. For example, it might include suggestions for meals, exercise recommendations, and advice on improving sleep.

[0461] Step 6:

[0462] The server sends the generated health guidance to the user's device. The device notifies the user of this information and allows them to view the details using a dedicated app.

[0463] Step 7:

[0464] The server will send an alert to the user and registered medical institutions if it detects a serious abnormality in the biometric data analysis. The alert will include details of the health risks.

[0465] Step 8:

[0466] When the device receives an alert, it notifies the user with sound and vibration. This allows the user to immediately understand the situation and take appropriate action.

[0467] Step 9:

[0468] Users improve their daily lives based on health guidance provided through their devices. By following the advice, they can strive to maintain their health and prevent illness.

[0469] (Example 1)

[0470] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0471] In modern society, lifestyle-related diseases and health problems caused by stress are on the rise, making it crucial for individuals to monitor and manage their own health on a daily basis. However, conventional methods make it difficult to efficiently collect and analyze individual health data and provide specific health guidance. Therefore, there is a need for a system that can easily and effectively collect health information in daily life, and, after expert analysis, provide specific guidance for improving lifestyle habits.

[0472] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0473] In this invention, the server includes a device for acquiring biometric information, a device for accumulating the information, and a device for analyzing the information using a generative artificial intelligence model. This enables users to continuously monitor their health status in their daily lives and receive specific health guidance based on rapid analysis.

[0474] A "user" refers to an individual who uses this system to manage their own health status.

[0475] "Biometric information" refers to information that includes data related to life activities, such as an individual's heart rate, steps taken, and sleep patterns.

[0476] "Device" refers to equipment or components that have a function and perform a specific operation or process.

[0477] "Communication methods" refer to the means and protocols used to securely transmit data.

[0478] "Integrated equipment" refers to a computing system for receiving, storing, and analyzing biological information.

[0479] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs specific analyses or predictions.

[0480] "Health status" refers to the overall state of an individual's physical and mental health.

[0481] "Health guidance" refers to advice and plans provided to individuals with the aim of improving or maintaining their health.

[0482] "Abnormal" refers to a condition that exhibits behavior or measurements that deviate from a normal state of health.

[0483] A "warning" refers to a message or signal used to notify a user or relevant organization of an abnormal event and to draw their attention to it.

[0484] A "terminal" refers to an electronic device used by a user to receive and verify information.

[0485] This invention is an information processing system configured to support users in managing their health in their daily lives. The main components of the system include a terminal worn by the user, a server that processes data, and a generative artificial intelligence model that supports them.

[0486] First, the device functions as a device that acquires biometric information such as heart rate, steps taken, and sleep patterns, specifically including smartphones and wearable devices. These devices acquire data from the user using built-in sensors and temporarily store the collected information within the device.

[0487] The acquired biometric information is transmitted to the server in an encrypted state via internet-based communication, either in real time or at specified time intervals. The server receives this data and stores it in a data storage facility. A generative artificial intelligence model on the server is trained using a large amount of health data and is used to analyze the user's health status in detail. This model predicts health risks and determines whether or not there are abnormalities based on changes in heart rate and sleep data.

[0488] Based on the analysis results, the server generates optimized health guidance. This guidance includes dietary suggestions, exercise recommendations, and stress management advice. This information is then transmitted back to the terminal via communication means and notified to the user. Specifically, the user can receive health guidance through an application on the terminal, which helps them review their lifestyle habits.

[0489] For example, if sleep deprivation is detected based on sleep data measured by a user's device, the server can use a generative AI model to generate instructions such as, "To get enough sleep, refrain from using your smartphone before bed," and the user can then create an action plan based on this advice.

[0490] An example of a prompt message would be, "Predict the user's stress level based on heart rate and sleep data, and generate appropriate health advice." This would allow the user to continuously monitor their health status and receive advice in an easy-to-understand format.

[0491] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0492] Step 1:

[0493] The device uses built-in sensors to acquire biometric information such as the user's heart rate, steps taken, and sleep patterns. Specifically, biometric information is continuously read from the sensors as digital data and stored in the device in real time. The input is real-time data from the sensors, and the output is temporarily stored digital biometric information.

[0494] Step 2:

[0495] The device organizes temporarily stored biometric information after a certain period or when it reaches a specified data volume. It then transmits the data to the server in an encrypted state using an internet connection. The SSL / TLS protocol is used for encryption. The input is temporarily stored biometric data, and the output is encrypted data prepared for transmission.

[0496] Step 3:

[0497] The server receives biometric information transmitted from the terminal and stores it in a database. The received data is formatted and transformed into an analyzable form. The input is encrypted biometric information, and the output is analyzable data stored in the database.

[0498] Step 4:

[0499] The server provides biometric information stored in the database to a generating AI model, which analyzes the user's health status. The generating AI model performs risk assessment and anomaly detection based on past health data. For example, it measures stress levels from patterns of heart rate increase and decrease. The input is analyzable biometric data, and the output is a health status assessment using the AI ​​model.

[0500] Step 5:

[0501] The server generates personalized health guidance based on the analysis results. This guidance includes specific suggestions regarding diet and exercise. The input is the health status assessment results, and the output is health guidance optimized for the user.

[0502] Step 6:

[0503] The server sends the generated health guidance back to the terminal. The terminal notifies the user of the received guidance information using a dedicated application. The terminal uses alert and reminder functions to keep the user informed. The input is the health guidance content, and the output is user instructions via the terminal screen and notification functions.

[0504] (Application Example 1)

[0505] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0506] In today's busy lifestyle, there is a need to accurately analyze an individual's health status and stress levels in real time and to provide appropriate guidance and safety measures quickly. However, conventional systems struggle to do this efficiently, and are particularly inadequate in addressing the timely management of health risks caused by stress. Therefore, this invention aims to solve these problems and provide a system that supports users in leading a safer and healthier life.

[0507] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0508] In this invention, the server includes means for collecting biometric information from the user, means for transmitting the collected biometric information via a network, means for an external device to analyze the received biometric information using a generative model and evaluate the user's health status, means for providing health guidance to the user based on the analysis results, means for sending warnings to the user and medical institutions if an abnormality is detected, and means for adjusting safety protection measures according to the stress level obtained from the analysis results. This enables real-time monitoring of the user's health status and stress level, and allows for prompt and accurate guidance and protection measures.

[0509] "Biometric information" refers to data collected to understand an individual's health status, and includes heart rate, steps taken, sleep patterns, and other similar data.

[0510] A "network" is a communication infrastructure used to send and receive information, and includes multiple communication technologies such as the internet and wireless communication.

[0511] An "external device" refers to a device that has a hardware or software configuration that receives and analyzes data transmitted from the user's terminal.

[0512] A "generative model" is a program or algorithm built to analyze data using machine learning or artificial intelligence techniques, and is used to predict health conditions and assess risks.

[0513] "Health guidance" refers to specific advice and policies regarding health provided to improve the user's quality of life.

[0514] A "warning" is a message sent to users or medical institutions to alert them when an emergency or abnormality occurs.

[0515] "Safety protection measures" refer to policies and measures implemented to ensure the health and safety of users, based on their health status and stress levels.

[0516] Modes for carrying out the invention

[0517] The system for realizing this application uses devices such as smartphones and smart glasses to generate programs for collecting and analyzing the user's biometric information in real time. These devices incorporate heart rate and accelerometer sensors, which are used to acquire the user's biometric data. The acquired data is then transmitted to a server via the network in an encrypted format.

[0518] The server performs analysis using a generative model based on the received biometric information. The software environment used utilizes TensorFlow as the machine learning library, and a secure, general network service is used as the cloud platform. The server leverages the generative AI model to evaluate stress levels and health status, and provides health guidance and safety protection measures based on these results.

[0519] Users can view analysis results and guidance sent from the server on their device. For example, if a user's heart rate suddenly increases, the application will issue a warning via voice or vibration and suggest relaxation techniques.

[0520] As a concrete example, when a user on a business trip experiences stress in an unfamiliar environment, the app can detect fluctuations in heart rate and immediately provide guidance such as, "Take a 10-minute meditation to relax; we'll guide you through it within the app." This feature allows users to instantly address health risks and efficiently manage their health.

[0521] Examples of prompt statements are as follows:

[0522] "Design a clever way to use user heart rate data to assess stress levels and automatically strengthen security protocols when stress levels are high."

[0523] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0524] Step 1:

[0525] The device uses built-in heart rate and accelerometer sensors to collect the user's biometric information in real time. This process acquires data such as heart rate, steps taken, and body movements, and temporarily stores it within the device. The input is biometric data acquired from the sensors, and the output is biometric information in a data format.

[0526] Step 2:

[0527] The device encrypts the collected biometric information and transmits it to the server via the network. Encryption ensures data security during transfer to the cloud server. The input is the unencrypted biometric data, and the output is the encrypted data packet.

[0528] Step 3:

[0529] The server decodes the received biometric information and analyzes the data using a generative AI model. This analysis evaluates the user's health status and stress level. The input is the decoded biometric data, and the output is the result of the health status evaluation.

[0530] Step 4:

[0531] The server generates health guidance and safety measures tailored to the user's health status and stress level based on the analysis results. This guidance includes suggestions for dietary improvements and relaxation methods, and is optimized for the user. The input is the analysis results data, and the output is personalized guidance content.

[0532] Step 5:

[0533] The user receives health guidance transmitted from the server on their device and displays it using a dedicated app. The device also provides voice and vibration notifications to the user to encourage quick confirmation. The input is guidance data from the server, and the output is the message displayed to the user.

[0534] Step 6:

[0535] The server sends a warning to the user and registered healthcare institutions if an anomaly is detected. This warning is intended to respond to changes in health conditions that require immediate attention. The input is the analysis result of the anomaly detection, and the output is the warning message.

[0536] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0537] This invention relates to a health management support system that incorporates an emotion engine that collects the user's biometric information and recognizes emotions based on that information. The system aims to acquire the user's biometric information through a smart device and comprehensively evaluate the user's physical and mental health.

[0538] First, the device uses a smartphone or wearable device to acquire biometric information such as heart rate, body temperature, and skin potential response in real time. This information is collected naturally during the user's daily life and temporarily stored on the device.

[0539] Next, the device encrypts the collected biometric information and sends it to a server via the internet. This data is then analyzed by an emotion engine. The emotion engine combines multiple biometric data points to recognize the user's emotional state. For example, if the heart rate is high and the skin potential response is strong, it is determined that the user is experiencing stress or excitement.

[0540] The server comprehensively assesses the user's health status based on biometric information and the results of the emotion engine analysis. This includes predicting health risks and evaluating the impact of emotions on health. For example, if a user experiences high levels of stress over a long period, a risk to their mental health may be identified.

[0541] Based on the analysis results, the server provides users with specific health guidance. This includes advice on stress reduction tailored to their emotional state, suggestions for relaxation-promoting exercises, and suggestions for improving their diet. This guidance is sent to the user's device, and the user can implement the guidance through a dedicated app.

[0542] Furthermore, if the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if an extreme stress state persists, an alert will be issued, and the user will be advised to seek professional medical support as soon as possible.

[0543] Users can utilize this system to better understand the impact of their emotions on their health and adjust their lifestyle based on the guidance provided. For example, the emotional engine can detect work-related stress and provide advice on how to refresh oneself accordingly, enabling users to take actions to maintain their mental and physical health.

[0544] The following describes the processing flow.

[0545] Step 1:

[0546] The device collects biometric information in real time from the user's smartphone or wearable device. Specifically, it acquires heart rate, body temperature, skin potential response, etc., through sensors and stores the data locally.

[0547] Step 2:

[0548] The device encrypts biometric information collected at regular intervals and sends it to the server. The data is transmitted using a secure protocol, protecting user privacy.

[0549] Step 3:

[0550] The server stores the received biometric information in a database. At this point, the data is cleansed, and missing or outlier values ​​are removed.

[0551] Step 4:

[0552] The server uses an emotion engine to recognize the user's emotions from biometric data. For example, if a high heart rate and a strong skin potential response are detected simultaneously, the server determines that the user is experiencing stress.

[0553] Step 5:

[0554] The server comprehensively evaluates the user's health status based on analysis results, including emotional state. This includes taking into account the impact of emotions such as stress and anxiety on health.

[0555] Step 6:

[0556] The server generates personalized health guidance based on the analysis results. It creates advice on stress reduction exercises and mental care based on the user's emotional state.

[0557] Step 7:

[0558] The server sends the generated health guidance to the user's device, and the device notifies the user. Through a dedicated app, the user can review and implement the detailed guidance.

[0559] Step 8:

[0560] If the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if prolonged high stress is detected, action will be prompted.

[0561] Step 9:

[0562] Users will work to improve their daily lives based on health guidance provided by their devices. Specific actions include activities to reduce stress and plans for getting adequate rest.

[0563] (Example 2)

[0564] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0565] In modern times, users are required to accurately understand their own emotions and physical health and to manage their health appropriately. However, conventional technology has faced the challenge of not being able to effectively and immediately assess the situation and provide appropriate guidance. This challenge makes it difficult for users to foresee health risks and to mitigate long-term health impacts.

[0566] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0567] In this invention, the server includes a device for acquiring information about the user's physical condition, a device for encoding the acquired information and transmitting it to a remote computing device, and a device for the computing device to analyze the received physical condition information using a generative model and estimate the emotional state. This makes it possible to comprehensively and in real time evaluate the user's emotional state and health condition and provide appropriate health guidance.

[0568] "Information regarding the user's physical condition" refers to data that indicates the user's physiological state, such as heart rate, body temperature, and skin potential response.

[0569] "Acquisition devices" refer to devices such as smartphones and wearable devices that use sensors to collect the user's biometric information.

[0570] "Encoding" is an encryption process used to securely transmit collected biometric information.

[0571] A "remote computing device" refers to a server or data processing system that communicates via the internet to process data sent from a user terminal.

[0572] A "generative model" is an algorithm that uses artificial intelligence or machine learning to analyze a user's biometric information.

[0573] A "device for estimating emotional state" is equipment or a function that uses an emotion engine to process a large amount of biometric information and identify the user's emotions.

[0574] "Health guidance" refers to specific advice provided to users based on analysis results, aimed at maintaining or improving their health.

[0575] A "warning" is an alert message sent to a user when an abnormal emotion or health condition is detected.

[0576] A "healthcare provider" is an institution or professional that provides health management or medical care.

[0577] This invention is a system for evaluating a user's emotions and health status and supporting health management. The embodiments of this system are described in detail below.

[0578] The device uses a smartphone or wearable device to collect the user's biometric information, such as heart rate, body temperature, and skin potential response. These devices use sensors to acquire data in real time and convert it to a standard format through built-in software. The collected data is temporarily stored in the device's memory.

[0579] The device encrypts the collected biometric information using the AES encryption algorithm and sends it to the server via the HTTPS protocol. This method ensures secure data transfer while protecting user privacy.

[0580] After decoding the received data, the server analyzes the biometric information using a generative AI model. The emotion engine uses machine learning algorithms to estimate the user's emotional state and calculates an emotion score from multiple biometric data points. For example, if the heart rate is elevated and the skin potential response is strong, it is determined that the user is in a stressed state.

[0581] The server comprehensively assesses the user's health status based on estimated emotional states and analysis results from a generative AI model. It predicts health risks and evaluates the physical impact of emotions, generating health guidance as needed. For example, if a prolonged high-stress emotional state is detected, relaxation methods and stress management advice will be provided.

[0582] Based on the analysis results, the server sends health guidance to the user's device, which the user can view and implement through a dedicated app. If an abnormal emotional state is detected, a warning is immediately displayed on the user's device, and healthcare providers are notified if necessary.

[0583] This system allows users to gain a deeper understanding of how their emotional state impacts their health and improve their lifestyle based on the advice provided. For example, a user could input a command into the AI ​​model such as, "Please tell me how to provide stress management advice based on the user's emotional state," and use this information for daily health management.

[0584] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0585] Step 1:

[0586] The device uses sensors from smartphones or wearable devices to acquire biometric information such as the user's heart rate, body temperature, and skin potential response. The input is these physiological indicators, and the output is the collected biometric data. The device formats this data using a dedicated application and temporarily stores it in memory.

[0587] Step 2:

[0588] The device encrypts the collected biometric information using the AES encryption algorithm and prepares it for secure data transfer. The input is formatted biometric data, and the output is encrypted data. This encrypted data is sent to the server via the HTTPS protocol.

[0589] Step 3:

[0590] The server receives encrypted data sent from the terminal and decrypts it. The input is encrypted data, and the output is raw biometric data. Next, the server uses a generated AI model to analyze the biometric data.

[0591] Step 4:

[0592] The server uses a generative AI model to estimate the user's emotional state. The input is decoded biometric data, and the output is a score or category indicating the emotional state. For example, a high heart rate and a large skin potential response would be classified as a stressed state. This analysis applies machine learning algorithms, and the model identifies emotions by comparing them with the data it was trained on.

[0593] Step 5:

[0594] The server comprehensively assesses the user's health status based on analyzed emotional states and health evaluations, and generates health guidance. The input is an emotional state score and associated biometric data, and the output is health guidance advice. For example, it can create documents containing relaxation suggestions and stress reduction methods.

[0595] Step 6:

[0596] The server sends the generated health guidance to the terminal, and the user receives the evaluation results and guidance content through a dedicated app. The input is health guidance advice, and the output is the evaluation content obtained by the user through the app. The user can use this information to work towards improving their lifestyle.

[0597] Step 7:

[0598] The server immediately sends a warning to the user's terminal if an abnormal emotional state is detected. The input is the result of the detected abnormal emotional state, and the output is the warning notification received by the user. If necessary, healthcare providers are also notified and support is provided for the user to take corrective action.

[0599] (Application Example 2)

[0600] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0601] In physical stores, there is a need to improve customer satisfaction by accurately understanding the emotional state of customers and providing optimal service based on that understanding. Traditional methods often rely on customer service staff to intuitively guess customer emotions, which has limitations in accuracy. Furthermore, there is currently a lack of systems to provide appropriate service at the right time, tailored to the customer's state.

[0602] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0603] In this invention, the server includes a device means for collecting biometric information from a user, a device means for transmitting the collected biometric information to an information processing device, and a device means for analyzing the biometric information of a customer and analyzing their emotional state. This makes it possible to analyze the emotional state of customers in a physical store in real time and provide optimal service based on the results.

[0604] A "user" refers to an individual who provides biometric information and receives health guidance and services from the system.

[0605] "Biometric information" is a general term for data that indicates the user's physical condition, such as heart rate, body temperature, and skin potential response.

[0606] "Device means" refers to a system that combines hardware and software for collecting, transmitting, and analyzing a user's biometric information.

[0607] An "information processing device" refers to a computing device that analyzes biometric information received from a user and evaluates their health and emotional state using a generative machine learning model.

[0608] A "generative machine learning model" refers to a data analysis algorithm used to analyze a user's health and emotional state based on received biometric information.

[0609] An "alert" refers to a notification issued to users or, if necessary, to draw attention to or alert an abnormality.

[0610] "Visitors" refers to customers who visit a physical store and are the recipients of services based on their biometric information.

[0611] "Emotional state" refers to the psychological and emotional state analyzed based on the user's physical and mental responses.

[0612] To implement this invention, the following system is necessary. First, the user wears a device such as a wearable sensor or smart glasses to collect biometric information such as heart rate, body temperature, and skin potential response in real time. The device also temporarily stores this biometric data, encrypts it, and transmits it to a server via the internet.

[0613] The server uses a generative machine learning model to analyze the received biometric information. This model utilizes a mature machine learning library such as TensorFlow. Based on these analyses, the server comprehensively assesses the user's health status, particularly identifying their emotional state. The analysis results are then sent to the device as specific health guidance. This guidance includes advice on stress-relieving activities and diet.

[0614] Furthermore, the server performs emotion analysis, which is also useful in physical stores, to understand the emotional state of customers in the store. Based on this information, sales staff and other staff members can be given suggestions for customer service methods tailored to each individual customer. As an example of this implementation, if a customer is determined to be tense, the staff can suggest products that have a relaxing effect based on the analysis results.

[0615] As a concrete example, a possible prompt message could be something like, "Please select an appropriate machine learning model to obtain the customer's heart rate data and analyze their current emotional state," which could be input to a generative AI model. This prompt would allow the server to perform a rapid and accurate analysis, enabling real-time service improvements.

[0616] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0617] Step 1:

[0618] The device acquires the user's biometric information, such as heart rate, body temperature, and skin potential response, in real time from wearable sensors and temporarily stores it in local storage. This process periodically samples data from the sensors and converts it into a data processing format (e.g., JSON). The input is analog signals from each biosensor, and the output is digital data.

[0619] Step 2:

[0620] The device encrypts the stored biometric information using encryption technology (e.g., AES encryption) and sends it to the server via the internet. The input is locally stored biometric data, and the output is encrypted data.

[0621] Step 3:

[0622] The server decrypts the encrypted data received from the terminal, performs the necessary preprocessing for formatting, and then prepares it for analysis. The input is encrypted biometric data, and the output is the decrypted data. In this step, the raw data is normalized and filtered for analysis.

[0623] Step 4:

[0624] The server analyzes decoded data using a generative AI model (e.g., a machine learning algorithm) to evaluate the user's health and emotional state. The input is decoded biometric data, and the output is a health assessment and emotional analysis result. This analysis detects abnormal indicators and emotional changes.

[0625] Step 5:

[0626] The server generates health guidance for the user based on the analysis results and sends it to the terminal. The input is a health assessment and emotion analysis results, and the output is specific health guidance content. As an implementation example, it generates recommendations for relaxation methods and dietary improvement advice.

[0627] Step 6:

[0628] In physical stores, the server analyzes the customer's emotional state and notifies store staff in real time, suggesting customer service based on that analysis as needed. The input is the emotional assessment result, and the output is a customer service suggestion. For example, if tension is detected in a customer, a notification is sent suggesting products that have a relaxing effect.

[0629] Step 7:

[0630] Users follow the health guidance received from their device and make lifestyle changes as needed. They also receive alerts if abnormalities are detected, allowing them to take corrective action. Inputs are health guidance and alerts, while output is the user's behavioral changes.

[0631] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0632] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0633] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0634] [Fourth Embodiment]

[0635] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0636] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0637] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0638] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0639] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0640] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0641] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0642] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0643] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0644] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0645] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0646] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0647] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0648] This invention is a system that supports users' health management using smartphones and wearable devices. This system monitors and provides guidance on the user's health status based on biometric information that the user continuously acquires in their daily life.

[0649] First, the device uses built-in sensors to acquire biometric information, such as the user's heart rate, steps taken, and sleep patterns. This information is recorded in real time or at specified time intervals and temporarily stored within the device.

[0650] The device then transmits biometric information to a server via the internet. This communication is encrypted, ensuring the security of the transmitted data. At this point, the user's lifestyle and past health data are also stored in a database, forming a dataset that forms the basis for analysis.

[0651] The server processes the received biometric information and uses a generative model to analyze the user's current health status. The model is trained on previously collected health data and can predict health risks and detect anomalies. For example, it can estimate stress levels from heart rate variability patterns or warn of sleep deprivation.

[0652] Based on the analysis results, the server generates personalized health guidance for each user. This guidance includes, for example, suggestions for dietary improvements, exercise recommendations, and advice for maintaining mental health. The generated guidance is sent to the device, allowing users to receive alert notifications or check it through a dedicated app.

[0653] Furthermore, if a serious abnormality is detected, the server immediately sends an alert to the user and registered medical institutions. Upon receiving the alert, the terminal notifies the user via voice and vibration to encourage a quick response. This function enables early detection of illness and emergency response.

[0654] Through this system, users can monitor their daily health status and improve their lifestyle habits according to the guidance provided. This allows users to proactively maintain their health, reduce the frequency of medical visits, and aim to extend their healthy lifespan. For example, if a user is told that their sleep quality is deteriorating, they can create an action plan to develop better sleep habits and work towards improvement.

[0655] The following describes the processing flow.

[0656] Step 1:

[0657] The device uses the user's smartphone or wearable device to collect biometric information. Information such as heart rate, steps taken, and sleep patterns is acquired in real time and temporarily stored in local storage.

[0658] Step 2:

[0659] The device sends the collected biometric information to the server as encrypted data at regular intervals (e.g., every hour). The transmission is performed via a secure protocol, ensuring data security.

[0660] Step 3:

[0661] The server stores the received data in a database for analysis. The stored data is standardized, and outliers are removed through data cleansing.

[0662] Step 4:

[0663] The server analyzes biometric information using generative models. It uses specific algorithms to assess the user's health status and determine whether there are any abnormalities or health risks.

[0664] Step 5:

[0665] Based on the analysis results, the server generates personalized health guidance for the user. For example, it might include suggestions for meals, exercise recommendations, and advice on improving sleep.

[0666] Step 6:

[0667] The server sends the generated health guidance to the user's device. The device notifies the user of this information and allows them to view the details using a dedicated app.

[0668] Step 7:

[0669] The server will send an alert to the user and registered medical institutions if it detects a serious abnormality in the biometric data analysis. The alert will include details of the health risks.

[0670] Step 8:

[0671] When the device receives an alert, it notifies the user with sound and vibration. This allows the user to immediately understand the situation and take appropriate action.

[0672] Step 9:

[0673] Users improve their daily lives based on health guidance provided through their devices. By following the advice, they can strive to maintain their health and prevent illness.

[0674] (Example 1)

[0675] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] In modern society, lifestyle-related diseases and health problems caused by stress are on the rise, making it crucial for individuals to monitor and manage their own health on a daily basis. However, conventional methods make it difficult to efficiently collect and analyze individual health data and provide specific health guidance. Therefore, there is a need for a system that can easily and effectively collect health information in daily life, and, after expert analysis, provide specific guidance for improving lifestyle habits.

[0677] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0678] In this invention, the server includes a device for acquiring biometric information, a device for accumulating the information, and a device for analyzing the information using a generative artificial intelligence model. This enables users to continuously monitor their health status in their daily lives and receive specific health guidance based on rapid analysis.

[0679] A "user" refers to an individual who uses this system to manage their own health status.

[0680] "Biometric information" refers to information that includes data related to life activities, such as an individual's heart rate, steps taken, and sleep patterns.

[0681] "Device" refers to equipment or components that have a function and perform a specific operation or process.

[0682] "Communication methods" refer to the means and protocols used to securely transmit data.

[0683] "Integrated equipment" refers to a computing system for receiving, storing, and analyzing biological information.

[0684] A "generative artificial intelligence model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs specific analyses or predictions.

[0685] "Health status" refers to the overall state of an individual's physical and mental health.

[0686] "Health guidance" refers to advice and plans provided to individuals with the aim of improving or maintaining their health.

[0687] "Abnormal" refers to a condition that exhibits behavior or measurements that deviate from a normal state of health.

[0688] A "warning" refers to a message or signal used to notify a user or relevant organization of an abnormal event and to draw their attention to it.

[0689] A "terminal" refers to an electronic device used by a user to receive and verify information.

[0690] This invention is an information processing system configured to support users in managing their health in their daily lives. The main components of the system include a terminal worn by the user, a server that processes data, and a generative artificial intelligence model that supports them.

[0691] First, the device functions as a device that acquires biometric information such as heart rate, steps taken, and sleep patterns, specifically including smartphones and wearable devices. These devices acquire data from the user using built-in sensors and temporarily store the collected information within the device.

[0692] The acquired biometric information is transmitted to the server in an encrypted state via internet-based communication, either in real time or at specified time intervals. The server receives this data and stores it in a data storage facility. A generative artificial intelligence model on the server is trained using a large amount of health data and is used to analyze the user's health status in detail. This model predicts health risks and determines whether or not there are abnormalities based on changes in heart rate and sleep data.

[0693] Based on the analysis results, the server generates optimized health guidance. This guidance includes dietary suggestions, exercise recommendations, and stress management advice. This information is then transmitted back to the terminal via communication means and notified to the user. Specifically, the user can receive health guidance through an application on the terminal, which helps them review their lifestyle habits.

[0694] For example, if sleep deprivation is detected based on sleep data measured by a user's device, the server can use a generative AI model to generate instructions such as, "To get enough sleep, refrain from using your smartphone before bed," and the user can then create an action plan based on this advice.

[0695] An example of a prompt message would be, "Predict the user's stress level based on heart rate and sleep data, and generate appropriate health advice." This would allow the user to continuously monitor their health status and receive advice in an easy-to-understand format.

[0696] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0697] Step 1:

[0698] The device uses built-in sensors to acquire biometric information such as the user's heart rate, steps taken, and sleep patterns. Specifically, biometric information is continuously read from the sensors as digital data and stored in the device in real time. The input is real-time data from the sensors, and the output is temporarily stored digital biometric information.

[0699] Step 2:

[0700] The device organizes temporarily stored biometric information after a certain period or when it reaches a specified data volume. It then transmits the data to the server in an encrypted state using an internet connection. The SSL / TLS protocol is used for encryption. The input is temporarily stored biometric data, and the output is encrypted data prepared for transmission.

[0701] Step 3:

[0702] The server receives biometric information transmitted from the terminal and stores it in a database. The received data is formatted and transformed into an analyzable form. The input is encrypted biometric information, and the output is analyzable data stored in the database.

[0703] Step 4:

[0704] The server provides biometric information stored in the database to a generating AI model, which analyzes the user's health status. The generating AI model performs risk assessment and anomaly detection based on past health data. For example, it measures stress levels from patterns of heart rate increase and decrease. The input is analyzable biometric data, and the output is a health status assessment using the AI ​​model.

[0705] Step 5:

[0706] The server generates personalized health guidance based on the analysis results. This guidance includes specific suggestions regarding diet and exercise. The input is the health status assessment results, and the output is health guidance optimized for the user.

[0707] Step 6:

[0708] The server sends the generated health guidance back to the terminal. The terminal notifies the user of the received guidance information using a dedicated application. The terminal uses alert and reminder functions to keep the user informed. The input is the health guidance content, and the output is user instructions via the terminal screen and notification functions.

[0709] (Application Example 1)

[0710] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0711] In today's busy lifestyle, there is a need to accurately analyze an individual's health status and stress levels in real time and to provide appropriate guidance and safety measures quickly. However, conventional systems struggle to do this efficiently, and are particularly inadequate in addressing the timely management of health risks caused by stress. Therefore, this invention aims to solve these problems and provide a system that supports users in leading a safer and healthier life.

[0712] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0713] In this invention, the server includes means for collecting biometric information from the user, means for transmitting the collected biometric information via a network, means for an external device to analyze the received biometric information using a generative model and evaluate the user's health status, means for providing health guidance to the user based on the analysis results, means for sending warnings to the user and medical institutions if an abnormality is detected, and means for adjusting safety protection measures according to the stress level obtained from the analysis results. This enables real-time monitoring of the user's health status and stress level, and allows for prompt and accurate guidance and protection measures.

[0714] "Biometric information" refers to data collected to understand an individual's health status, and includes heart rate, steps taken, sleep patterns, and other similar data.

[0715] A "network" is a communication infrastructure used to send and receive information, and includes multiple communication technologies such as the internet and wireless communication.

[0716] An "external device" refers to a device that has a hardware or software configuration that receives and analyzes data transmitted from the user's terminal.

[0717] A "generative model" is a program or algorithm built to analyze data using machine learning or artificial intelligence techniques, and is used to predict health conditions and assess risks.

[0718] "Health guidance" refers to specific advice and policies regarding health provided to improve the user's quality of life.

[0719] A "warning" is a message sent to users or medical institutions to alert them when an emergency or abnormality occurs.

[0720] "Safety protection measures" refer to policies and measures implemented to ensure the health and safety of users, based on their health status and stress levels.

[0721] Modes for carrying out the invention

[0722] The system for realizing this application uses devices such as smartphones and smart glasses to generate programs for collecting and analyzing the user's biometric information in real time. These devices incorporate heart rate and accelerometer sensors, which are used to acquire the user's biometric data. The acquired data is then transmitted to a server via the network in an encrypted format.

[0723] The server performs analysis using a generative model based on the received biometric information. The software environment used utilizes TensorFlow as the machine learning library, and a secure, general network service is used as the cloud platform. The server leverages the generative AI model to evaluate stress levels and health status, and provides health guidance and safety protection measures based on these results.

[0724] Users can view analysis results and guidance sent from the server on their device. For example, if a user's heart rate suddenly increases, the application will issue a warning via voice or vibration and suggest relaxation techniques.

[0725] As a concrete example, when a user on a business trip experiences stress in an unfamiliar environment, the app can detect fluctuations in heart rate and immediately provide guidance such as, "Take a 10-minute meditation to relax; we'll guide you through it within the app." This feature allows users to instantly address health risks and efficiently manage their health.

[0726] Examples of prompt statements are as follows:

[0727] "Design a clever way to use user heart rate data to assess stress levels and automatically strengthen security protocols when stress levels are high."

[0728] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0729] Step 1:

[0730] The device uses built-in heart rate and accelerometer sensors to collect the user's biometric information in real time. This process acquires data such as heart rate, steps taken, and body movements, and temporarily stores it within the device. The input is biometric data acquired from the sensors, and the output is biometric information in a data format.

[0731] Step 2:

[0732] The device encrypts the collected biometric information and transmits it to the server via the network. Encryption ensures data security during transfer to the cloud server. The input is the unencrypted biometric data, and the output is the encrypted data packet.

[0733] Step 3:

[0734] The server decodes the received biometric information and analyzes the data using a generative AI model. This analysis evaluates the user's health status and stress level. The input is the decoded biometric data, and the output is the result of the health status evaluation.

[0735] Step 4:

[0736] The server generates health guidance and safety measures tailored to the user's health status and stress level based on the analysis results. This guidance includes suggestions for dietary improvements and relaxation methods, and is optimized for the user. The input is the analysis results data, and the output is personalized guidance content.

[0737] Step 5:

[0738] The user receives health guidance transmitted from the server on their device and displays it using a dedicated app. The device also provides voice and vibration notifications to the user to encourage quick confirmation. The input is guidance data from the server, and the output is the message displayed to the user.

[0739] Step 6:

[0740] The server sends a warning to the user and registered healthcare institutions if an anomaly is detected. This warning is intended to respond to changes in health conditions that require immediate attention. The input is the analysis result of the anomaly detection, and the output is the warning message.

[0741] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0742] This invention relates to a health management support system that incorporates an emotion engine that collects the user's biometric information and recognizes emotions based on that information. The system aims to acquire the user's biometric information through a smart device and comprehensively evaluate the user's physical and mental health.

[0743] First, the device uses a smartphone or wearable device to acquire biometric information such as heart rate, body temperature, and skin potential response in real time. This information is collected naturally during the user's daily life and temporarily stored on the device.

[0744] Next, the device encrypts the collected biometric information and sends it to a server via the internet. This data is then analyzed by an emotion engine. The emotion engine combines multiple biometric data points to recognize the user's emotional state. For example, if the heart rate is high and the skin potential response is strong, it is determined that the user is experiencing stress or excitement.

[0745] The server comprehensively assesses the user's health status based on biometric information and the results of the emotion engine analysis. This includes predicting health risks and evaluating the impact of emotions on health. For example, if a user experiences high levels of stress over a long period, a risk to their mental health may be identified.

[0746] Based on the analysis results, the server provides users with specific health guidance. This includes advice on stress reduction tailored to their emotional state, suggestions for relaxation-promoting exercises, and suggestions for improving their diet. This guidance is sent to the user's device, and the user can implement the guidance through a dedicated app.

[0747] Furthermore, if the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if an extreme stress state persists, an alert will be issued, and the user will be advised to seek professional medical support as soon as possible.

[0748] Users can utilize this system to better understand the impact of their emotions on their health and adjust their lifestyle based on the guidance provided. For example, the emotional engine can detect work-related stress and provide advice on how to refresh oneself accordingly, enabling users to take actions to maintain their mental and physical health.

[0749] The following describes the processing flow.

[0750] Step 1:

[0751] The device collects biometric information in real time from the user's smartphone or wearable device. Specifically, it acquires heart rate, body temperature, skin potential response, etc., through sensors and stores the data locally.

[0752] Step 2:

[0753] The device encrypts biometric information collected at regular intervals and sends it to the server. The data is transmitted using a secure protocol, protecting user privacy.

[0754] Step 3:

[0755] The server stores the received biometric information in a database. At this point, the data is cleansed, and missing or outlier values ​​are removed.

[0756] Step 4:

[0757] The server uses an emotion engine to recognize the user's emotions from biometric data. For example, if a high heart rate and a strong skin potential response are detected simultaneously, the server determines that the user is experiencing stress.

[0758] Step 5:

[0759] The server comprehensively evaluates the user's health status based on analysis results, including emotional state. This includes taking into account the impact of emotions such as stress and anxiety on health.

[0760] Step 6:

[0761] The server generates personalized health guidance based on the analysis results. It creates advice on stress reduction exercises and mental care based on the user's emotional state.

[0762] Step 7:

[0763] The server sends the generated health guidance to the user's device, and the device notifies the user. Through a dedicated app, the user can review and implement the detailed guidance.

[0764] Step 8:

[0765] If the emotion engine detects an abnormal emotional state, the server immediately sends an alert to the user and, if necessary, to a medical institution. For example, if prolonged high stress is detected, action will be prompted.

[0766] Step 9:

[0767] Users will work to improve their daily lives based on health guidance provided by their devices. Specific actions include activities to reduce stress and plans for getting adequate rest.

[0768] (Example 2)

[0769] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0770] In modern times, users are required to accurately understand their own emotions and physical health and to manage their health appropriately. However, conventional technology has faced the challenge of not being able to effectively and immediately assess the situation and provide appropriate guidance. This challenge makes it difficult for users to foresee health risks and to mitigate long-term health impacts.

[0771] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0772] In this invention, the server includes a device for acquiring information about the user's physical condition, a device for encoding the acquired information and transmitting it to a remote computing device, and a device for the computing device to analyze the received physical condition information using a generative model and estimate the emotional state. This makes it possible to comprehensively and in real time evaluate the user's emotional state and health condition and provide appropriate health guidance.

[0773] "Information regarding the user's physical condition" refers to data that indicates the user's physiological state, such as heart rate, body temperature, and skin potential response.

[0774] "Acquisition devices" refer to devices such as smartphones and wearable devices that use sensors to collect the user's biometric information.

[0775] "Encoding" is an encryption process used to securely transmit collected biometric information.

[0776] A "remote computing device" refers to a server or data processing system that communicates via the internet to process data sent from a user terminal.

[0777] A "generative model" is an algorithm that uses artificial intelligence or machine learning to analyze a user's biometric information.

[0778] A "device for estimating emotional state" is equipment or a function that uses an emotion engine to process a large amount of biometric information and identify the user's emotions.

[0779] "Health guidance" refers to specific advice provided to users based on analysis results, aimed at maintaining or improving their health.

[0780] A "warning" is an alert message sent to a user when an abnormal emotion or health condition is detected.

[0781] A "healthcare provider" is an institution or professional that provides health management or medical care.

[0782] This invention is a system for evaluating a user's emotions and health status and supporting health management. The embodiments of this system are described in detail below.

[0783] The device uses a smartphone or wearable device to collect the user's biometric information, such as heart rate, body temperature, and skin potential response. These devices use sensors to acquire data in real time and convert it to a standard format through built-in software. The collected data is temporarily stored in the device's memory.

[0784] The device encrypts the collected biometric information using the AES encryption algorithm and sends it to the server via the HTTPS protocol. This method ensures secure data transfer while protecting user privacy.

[0785] After decoding the received data, the server analyzes the biometric information using a generative AI model. The emotion engine uses machine learning algorithms to estimate the user's emotional state and calculates an emotion score from multiple biometric data points. For example, if the heart rate is elevated and the skin potential response is strong, it is determined that the user is in a stressed state.

[0786] The server comprehensively assesses the user's health status based on estimated emotional states and analysis results from a generative AI model. It predicts health risks and evaluates the physical impact of emotions, generating health guidance as needed. For example, if a prolonged high-stress emotional state is detected, relaxation methods and stress management advice will be provided.

[0787] Based on the analysis results, the server sends health guidance to the user's device, which the user can view and implement through a dedicated app. If an abnormal emotional state is detected, a warning is immediately displayed on the user's device, and healthcare providers are notified if necessary.

[0788] This system allows users to gain a deeper understanding of how their emotional state impacts their health and improve their lifestyle based on the advice provided. For example, a user could input a command into the AI ​​model such as, "Please tell me how to provide stress management advice based on the user's emotional state," and use this information for daily health management.

[0789] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0790] Step 1:

[0791] The device uses sensors from smartphones or wearable devices to acquire biometric information such as the user's heart rate, body temperature, and skin potential response. The input is these physiological indicators, and the output is the collected biometric data. The device formats this data using a dedicated application and temporarily stores it in memory.

[0792] Step 2:

[0793] The device encrypts the collected biometric information using the AES encryption algorithm and prepares it for secure data transfer. The input is formatted biometric data, and the output is encrypted data. This encrypted data is sent to the server via the HTTPS protocol.

[0794] Step 3:

[0795] The server receives encrypted data sent from the terminal and decrypts it. The input is encrypted data, and the output is raw biometric data. Next, the server uses a generated AI model to analyze the biometric data.

[0796] Step 4:

[0797] The server uses a generative AI model to estimate the user's emotional state. The input is decoded biometric data, and the output is a score or category indicating the emotional state. For example, a high heart rate and a large skin potential response would be classified as a stressed state. This analysis applies machine learning algorithms, and the model identifies emotions by comparing them with the data it was trained on.

[0798] Step 5:

[0799] The server comprehensively assesses the user's health status based on analyzed emotional states and health evaluations, and generates health guidance. The input is an emotional state score and associated biometric data, and the output is health guidance advice. For example, it can create documents containing relaxation suggestions and stress reduction methods.

[0800] Step 6:

[0801] The server sends the generated health guidance to the terminal, and the user receives the evaluation results and guidance content through a dedicated app. The input is health guidance advice, and the output is the evaluation content obtained by the user through the app. The user can use this information to work towards improving their lifestyle.

[0802] Step 7:

[0803] The server immediately sends a warning to the user's terminal if an abnormal emotional state is detected. The input is the result of the detected abnormal emotional state, and the output is the warning notification received by the user. If necessary, healthcare providers are also notified and support is provided for the user to take corrective action.

[0804] (Application Example 2)

[0805] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0806] In physical stores, there is a need to improve customer satisfaction by accurately understanding the emotional state of customers and providing optimal service based on that understanding. Traditional methods often rely on customer service staff to intuitively guess customer emotions, which has limitations in accuracy. Furthermore, there is currently a lack of systems to provide appropriate service at the right time, tailored to the customer's state.

[0807] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0808] In this invention, the server includes a device means for collecting biometric information from a user, a device means for transmitting the collected biometric information to an information processing device, and a device means for analyzing the biometric information of a customer and analyzing their emotional state. This makes it possible to analyze the emotional state of customers in a physical store in real time and provide optimal service based on the results.

[0809] A "user" refers to an individual who provides biometric information and receives health guidance and services from the system.

[0810] "Biometric information" is a general term for data that indicates the user's physical condition, such as heart rate, body temperature, and skin potential response.

[0811] "Device means" refers to a system that combines hardware and software for collecting, transmitting, and analyzing a user's biometric information.

[0812] An "information processing device" refers to a computing device that analyzes biometric information received from a user and evaluates their health and emotional state using a generative machine learning model.

[0813] A "generative machine learning model" refers to a data analysis algorithm used to analyze a user's health and emotional state based on received biometric information.

[0814] An "alert" refers to a notification issued to users or, if necessary, to draw attention to or alert an abnormality.

[0815] "Visitors" refers to customers who visit a physical store and are the recipients of services based on their biometric information.

[0816] "Emotional state" refers to the psychological and emotional state analyzed based on the user's physical and mental responses.

[0817] To implement this invention, the following system is necessary. First, the user wears a device such as a wearable sensor or smart glasses to collect biometric information such as heart rate, body temperature, and skin potential response in real time. The device also temporarily stores this biometric data, encrypts it, and transmits it to a server via the internet.

[0818] The server uses a generative machine learning model to analyze the received biometric information. This model utilizes a mature machine learning library such as TensorFlow. Based on these analyses, the server comprehensively assesses the user's health status, particularly identifying their emotional state. The analysis results are then sent to the device as specific health guidance. This guidance includes advice on stress-relieving activities and diet.

[0819] Furthermore, the server performs emotion analysis, which is also useful in physical stores, to understand the emotional state of customers in the store. Based on this information, sales staff and other staff members can be given suggestions for customer service methods tailored to each individual customer. As an example of this implementation, if a customer is determined to be tense, the staff can suggest products that have a relaxing effect based on the analysis results.

[0820] As a concrete example, a possible prompt message could be something like, "Please select an appropriate machine learning model to obtain the customer's heart rate data and analyze their current emotional state," which could be input to a generative AI model. This prompt would allow the server to perform a rapid and accurate analysis, enabling real-time service improvements.

[0821] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0822] Step 1:

[0823] The device acquires the user's biometric information, such as heart rate, body temperature, and skin potential response, in real time from wearable sensors and temporarily stores it in local storage. This process periodically samples data from the sensors and converts it into a data processing format (e.g., JSON). The input is analog signals from each biosensor, and the output is digital data.

[0824] Step 2:

[0825] The device encrypts the stored biometric information using encryption technology (e.g., AES encryption) and sends it to the server via the internet. The input is locally stored biometric data, and the output is encrypted data.

[0826] Step 3:

[0827] The server decrypts the encrypted data received from the terminal, performs the necessary preprocessing for formatting, and then prepares it for analysis. The input is encrypted biometric data, and the output is the decrypted data. In this step, the raw data is normalized and filtered for analysis.

[0828] Step 4:

[0829] The server analyzes decoded data using a generative AI model (e.g., a machine learning algorithm) to evaluate the user's health and emotional state. The input is decoded biometric data, and the output is a health assessment and emotional analysis result. This analysis detects abnormal indicators and emotional changes.

[0830] Step 5:

[0831] The server generates health guidance for the user based on the analysis results and sends it to the terminal. The input is a health assessment and emotion analysis results, and the output is specific health guidance content. As an implementation example, it generates recommendations for relaxation methods and dietary improvement advice.

[0832] Step 6:

[0833] In physical stores, the server analyzes the customer's emotional state and notifies store staff in real time, suggesting customer service based on that analysis as needed. The input is the emotional assessment result, and the output is a customer service suggestion. For example, if tension is detected in a customer, a notification is sent suggesting products that have a relaxing effect.

[0834] Step 7:

[0835] Users follow the health guidance received from their device and make lifestyle changes as needed. They also receive alerts if abnormalities are detected, allowing them to take corrective action. Inputs are health guidance and alerts, while output is the user's behavioral changes.

[0836] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0837] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0838] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0839] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0840] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0841] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0842] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0843] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0844] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0845] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0846] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0847] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0848] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0849] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0850] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0851] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0852] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0853] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0854] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0855] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0856] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0857] The following is further disclosed regarding the embodiments described above.

[0858] (Claim 1)

[0859] Means for collecting biometric information from users,

[0860] A means of transmitting the collected biometric information to a server,

[0861] A means by which a server analyzes received biometric information using a generative model and evaluates health status,

[0862] A means of providing health guidance to users based on analysis results,

[0863] A means of sending alerts to users and medical institutions when an anomaly is detected,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, which provides specific advice for improving the user's lifestyle based on the analysis results.

[0867] (Claim 3)

[0868] The system according to claim 1, wherein the user's terminal notifies the user of an anomaly detection alert with a loud sound or vibration.

[0869] "Example 1"

[0870] (Claim 1)

[0871] A device that acquires biometric information from a user,

[0872] A device that transmits acquired biometric information to a data collection facility using communication means,

[0873] A device that analyzes received biological information using a generative artificial intelligence model and evaluates health status,

[0874] A device that generates health guidance for the user based on analysis results,

[0875] A device that issues a warning to the user and medical institutions when a serious abnormality is detected,

[0876] A device that sends the analysis results to the user's terminal and notifies the user,

[0877] An information processing system that includes this.

[0878] (Claim 2)

[0879] The information processing system according to claim 1, which provides specific guidance to encourage users to improve their lifestyle habits using evaluation results based on analysis.

[0880] (Claim 3)

[0881] The information processing system according to claim 1, wherein the user's terminal notifies the user of an abnormality by loud volume or vibration.

[0882] "Application Example 1"

[0883] (Claim 1)

[0884] Means for collecting biometric information from users,

[0885] A means of transmitting collected biometric information via a network,

[0886] A means for an external device to analyze received biological information using a generative model and evaluate health status,

[0887] A means of providing health guidance to users based on the analysis results,

[0888] A means of sending a warning to the user and medical institution when an abnormality is detected,

[0889] A means of adjusting safety protection measures according to the stress level obtained from the analysis results,

[0890] A system that includes this.

[0891] (Claim 2)

[0892] The system according to claim 1, which provides specific instructions for improving the user's lifestyle patterns based on the analysis results.

[0893] (Claim 3)

[0894] The system according to claim 1, wherein the user's device notifies the user of an anomaly detection warning by voice or vibration.

[0895] "Example 2 of combining an emotion engine"

[0896] (Claim 1)

[0897] A device that acquires information about the user's physical condition,

[0898] A device that encodes the acquired information and transmits it to a remote computing device,

[0899] A device that uses a generative model to analyze received physical state information and estimate emotional state,

[0900] A device that comprehensively evaluates the user's health status based on an analysis of their emotional and physical state,

[0901] A device that provides health guidance to users based on evaluation results,

[0902] A device that sends a warning to the user and healthcare provider when it detects an abnormal emotional state,

[0903] A system that includes this.

[0904] (Claim 2)

[0905] The system according to claim 1, which provides specific advice for improving the user's living environment based on the analysis results.

[0906] (Claim 3)

[0907] The system according to claim 1, wherein the user's information terminal notifies the user of an anomaly detection warning by voice or vibration.

[0908] "Application example 2 when combining with an emotional engine"

[0909] (Claim 1)

[0910] A device and means for collecting biometric information from a user,

[0911] A device means for transmitting collected biological information to an information processing device,

[0912] A device means for evaluating health status by analyzing received biological information using a machine learning model generated by an information processing device,

[0913] A device and means that provides health guidance to the user based on the analysis results,

[0914] A device that transmits an alarm to the user and medical institution when an abnormality is detected,

[0915] A device and means for analyzing the biometric information of customers and their emotional state,

[0916] A device that proposes methods for dealing with customers based on the analysis results,

[0917] A system that includes this.

[0918] (Claim 2)

[0919] The system according to claim 1, which provides specific advice for improving the user's lifestyle based on the analysis results.

[0920] (Claim 3)

[0921] The system according to claim 1, wherein the user's information terminal notifies the user of an abnormality detection alarm with a loud sound or vibration. [Explanation of symbols]

[0922] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting biometric information from users, A means of transmitting the collected biometric information to a server, A means by which a server analyzes received biometric information using a generative model and evaluates health status, A means of providing health guidance to users based on analysis results, A means of sending alerts to users and medical institutions when an anomaly is detected, A system that includes this.

2. The system according to claim 1, which provides specific advice for improving the user's lifestyle based on the analysis results.

3. The system according to claim 1, wherein the user's terminal notifies the user of an anomaly detection alert with a loud sound or vibration.

Images