System

Abstract

A system is provided.SOLUTION: A system, comprising: means for collecting various data of a user's daily life by a wearable device and a mobile terminal; means for accumulating the collected data in a cloud database; means for pre-processing the accumulated data; means for analyzing the pre-processed data and determining and evaluating a user's health condition; means for generating health improvement advice based on the determination and evaluation; and means for notifying the mobile terminal of the generated advice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional health management and disease prevention rely on regular health checkups and comprehensive medical examinations, but these are infrequent, making it difficult to detect illness early or take appropriate preventive measures. Furthermore, users often have difficulty noticing subtle changes in their physical condition or health risks in their daily lives. In today's complex and busy living environment, it is important to constantly monitor users' health status and provide appropriate advice in real time. Therefore, there is a need for the development of a system that can comprehensively collect and analyze various data from daily life and evaluate users' health status. [Means for solving the problem]

[0005] This invention provides a system that includes a means for collecting various data about a user's daily life using a wearable device and a mobile terminal, a means for storing the collected data in a cloud database, a means for preprocessing the stored data, a means for analyzing the preprocessed data to determine and evaluate the user's health status, a means for generating health improvement advice based on the determination and evaluation, and a means for notifying the mobile terminal of the generated advice. This system continuously monitors and analyzes health-related data obtained through daily activities, enabling early detection of changes in physical condition and health risks that the user may not otherwise notice. Furthermore, by using a generative artificial intelligence model to provide advice to the user in an easy-to-understand format, it is possible to suggest specific lifestyle improvements and maintain and improve the user's health status.

[0006] A "wearable device" is an electronic device that can be worn on a user's body and collects biometric data such as weight, heart rate, and number of steps taken.

[0007] A "mobile terminal" is a portable computing device with communication capabilities, such as a smartphone or tablet.

[0008] A "cloud database" is a database system that can be accessed via the Internet and that stores and manages data remotely.

[0009] "Preprocessing" refers to processes such as data cleansing and normalization to prepare collected raw data into an analyzable format.

[0010] "Analysis" is the process of using machine learning models and statistical methods to detect patterns and anomalies in collected and preprocessed data.

[0011] "Health status assessment and evaluation" is the process of evaluating the user's physical condition and potential health risks based on the analysis results, and generating a score and diagnostic results.

[0012] "Health improvement advice" is information that provides specific guidelines and recommendations for improving the user's lifestyle.

[0013] A "generative artificial intelligence model" is an artificial intelligence technology that learns from large amounts of data and generates optimal advice and responses for users.

[0014] "Notification" is the process of sending generated advice or warnings to the user's mobile terminal so that the user can receive and acknowledge them. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life to support health improvement. This system constantly monitors the user's health status and provides appropriate advice to reduce health risks.

[0037] Device settings and integration

[0038] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. Data measured by the wearable device is then sent to the mobile device and stored in a cloud database.

[0039] Data collection and transmission

[0040] Your devices (smart devices and smartphones) automatically collect the following data throughout your daily activities:

[0041] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[0042] The food logging app records the user's meal contents and sends the information to a mobile device.

[0043] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[0044] The smartphone measures distance traveled and walking speed using GPS and integrates the data.

[0045] Data accumulation and preprocessing

[0046] The server stores data sent from mobile devices in a cloud database in real time. Since the data stored in the database is difficult to analyze as is, it undergoes preprocessing. Specifically, this involves data cleansing (complementing missing values ​​and removing outliers), data normalization (scaling values), and sorting time-series data.

[0047] Data analysis

[0048] The server then analyzes the pre-processed data using machine learning models and statistical methods. This analysis evaluates the following:

[0049] Weight Trends: Analyze long-term weight patterns to detect abnormal weight changes.

[0050] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0051] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0052] Health assessment

[0053] The server evaluates the user's health condition based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if there is poor nutritional balance or abnormalities in walking patterns.

[0054] Generate health improvement advice

[0055] The server uses the generative AI model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0056] Advice Notification

[0057] The server notifies the mobile device of the generated advice and warnings, and the user can check these advices through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[0058] User feedback and continuous improvement

[0059] Users improve their lifestyle habits based on the advice they receive, and the server uses a generative AI model to provide appropriate answers to feedback and questions. This allows users to constantly monitor their health status and make continuous improvements.

[0060] As described above, this system utilizes wearable devices and mobile terminals to monitor and manage users' health status in real time and provide specific and appropriate advice, enabling users to detect health risks early and take preventive measures.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The user sets up the smart device (e.g., smart watch, smart scale) and mobile terminal initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.).

[0064] Step 2:

[0065] Terminals (smart devices and smartphones) collect data on daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed.

[0066] Step 3:

[0067] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, etc.) in a cloud database in real time. Specifically, it receives data through an API and stores each data item in the corresponding table.

[0068] Step 4:

[0069] The server preprocesses the accumulated data. Specifically, it performs data cleansing, imputes missing values, and removes outliers. It also performs data normalization and scales the data to a certain range. For time series data, it aligns the data based on timestamps.

[0070] Step 5:

[0071] The server then analyzes the preprocessed data using machine learning models and statistical methods, including trend analysis of weight change, nutritional evaluation of dietary data, and detection of abnormalities in walking patterns.

[0072] Step 6:

[0073] The server evaluates the user's health condition based on the analysis results, generating an abnormality score and calculating the health risk level in the event of weight gain, nutritional imbalance, or abnormal walking patterns.

[0074] Step 7:

[0075] The server uses the generative AI model to generate health improvement advice based on the analysis results. Specific advice might include, "Your calorie intake continues to be high. Include more vegetables and fruits in your diet," or "Your walking pattern is abnormal. We recommend that you see a doctor."

[0076] Step 8:

[0077] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[0078] Step 9:

[0079] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[0080] Step 10:

[0081] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] In modern society, many people lead unhealthy lifestyles, resulting in an increase in lifestyle-related diseases, which has become a serious social problem. The present invention aims to solve these problems and improve the health status of users by providing a system that efficiently collects, analyzes, and evaluates health-related data from daily life and provides appropriate health improvement advice. In particular, highly accurate health management is possible by consistently performing the process from preprocessing, analysis, and evaluation of collected data to generating advice.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a portable information processing device, means for storing the collected data in a remote database, preprocessing means for performing data cleansing, data normalization, and time-series data alignment on the stored data, means for analyzing the preprocessed data using a machine learning model and statistical methods to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation using a generation AI model, and means for notifying the portable information processing device of the generated advice. This makes it possible to comprehensively monitor and evaluate the user's health status and provide individually optimized, specific health improvement advice.

[0087] A "wearable device" is an electronic device that can be worn to collect various data in daily life.

[0088] A "portable information processing device" is an electronic device that can be carried by a user and is used for performing information processing such as collecting, transmitting, and displaying data.

[0089] "Data cleansing" is the process of detecting missing or outliers in collected data and then completing or removing them.

[0090] "Data normalization" is the process of converting data in different units to a common scale to make it easier to analyze.

[0091] "Time series data sorting" is the process of sorting collected data into the correct order based on the time axis.

[0092] A "machine learning model" is an algorithm or mathematical model used to analyze collected data and make predictions or classifications.

[0093] "Statistical methods" are mathematical and statistical techniques used to analyze, evaluate, and interpret data.

[0094] "Assessment and evaluation" is the process of determining the user's health condition based on the analyzed data and making an evaluation of that condition.

[0095] A "generative AI model" is a model that uses artificial intelligence to generate new information and advice from data.

[0096] A "remote database" is a remote data storage system connected to a server for storing and managing collected data.

[0097] "Data preprocessing" is a series of processes performed to convert raw data into an analyzable form.

[0098] MODE FOR CARRYING OUT THE INVENTION

[0099] This invention is a system that collects, analyzes, and evaluates health-related data from a user's daily life and provides appropriate health improvement advice. This system supports the user's health management by collecting data using wearable devices and portable information processing devices and analyzing the data on a server.

[0100] Device settings and integration

[0101] First, the user sets up and links the wearable device and portable information processing device. Specific hardware examples include a smartwatch and a smart weighing scale. The user then installs a dedicated app on their smartphone and enters their personal information. The health-related data measured by the wearable device is then sent to the portable information processing device and stored in a cloud database.

[0102] Data collection and transmission

[0103] Wearable devices and portable information processing devices (e.g., smartwatches, smart scales, smartphones) collect daily activity data. Specifically, smart scales measure weight every morning and send the data to a smartphone, food log apps record meal contents and send the data to a smartphone, and smartwatches measure the number of steps taken, heart rate, and walking patterns and send the data to a smartphone. Smartphones also use GPS information to measure distance traveled and walking speed.

[0104] Data accumulation and preprocessing

[0105] The server accumulates data sent from mobile devices in a cloud database in real time, then performs preprocessing such as data cleansing (filling in missing values ​​and removing outliers), data normalization (scaling values), and time-series data alignment, ensuring the consistency and accuracy of the data and making it suitable for analysis.

[0106] Data analysis

[0107] The server then analyzes the preprocessed data using machine learning models and statistical methods. For example, it analyzes weight trends, dietary habits, and walking patterns. Specific models used include decision trees, random forests, and time series analysis models. The analysis results in an assessment of the user's health status.

[0108] Health assessment

[0109] The server evaluates the user's health status based on the analysis results and calculates an abnormality score and health risk level. For example, if sudden weight gain or nutritional imbalance is detected, a warning score will be generated. Abnormalities in the number of steps taken each day and walking patterns will also be detected.

[0110] Generate health improvement advice

[0111] The server uses the generative AI model to generate specific health improvement advice. This advice is presented in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0112] Advice Notification

[0113] The server notifies the mobile information processing device of the generated advice and warnings. The user can check these advice and warnings through a dedicated app. If necessary, the user can also send feedback and ask follow-up questions.

[0114] User feedback and continuous improvement

[0115] Users improve their lifestyle habits based on the advice they receive. The server uses a generative AI model based on feedback from users to provide appropriate answers. This allows users to constantly monitor their health status and make continuous improvements.

[0116] Examples of prompt statements

[0117] Example prompts to input to a generative AI model:

[0118] "I'm a 30-year-old man and I've recently gained a lot of weight. Can you give me some specific advice on how to manage my weight?"

[0119] The above is a specific embodiment for carrying out the present invention. This system allows users to monitor their own health condition in detail and take appropriate and specific measures to improve their health.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Program processing steps

[0122] Step 1: Device Setup and Integration

[0123] Input: Wearable device, smartphone, dedicated app

[0124] Output: Initial setup completed, device connectivity established

[0125] Specific operation:

[0126] 1. The user has a wearable device (e.g., smart watch, smart scale) at hand.

[0127] 2. The user installs the dedicated app on their smartphone and launches it.

[0128] 3. After launching the app, the user performs initial setup by entering personal information such as name, date of birth, gender, height, and weight.

[0129] 4. The user enables the Bluetooth function on their smartphone and pairs it with the wearable device, establishing a connection between the devices.

[0130] Step 2: Data collection and transmission

[0131] Input: User's daily activities (weight measurement, eating, walking, etc.), sensor data from wearable devices

[0132] Output: Measurement data, transmitted data

[0133] Specific operation:

[0134] 1. A smart scale measures a user's weight every morning and sends the data to a smartphone.

[0135] 2. The food logging app records the user's meal contents and sends the data to their smartphone.

[0136] 3. The smartwatch measures the number of steps, heart rate, and walking patterns and sends the data to your smartphone.

[0137] 4. Smartphones will use GPS to measure distance traveled and walking speed and centralize the data.

[0138] Step 3: Data accumulation and preprocessing

[0139] Input: Measurement data from a smartphone

[0140] Output: Accumulated data, preprocessed data

[0141] Specific operation:

[0142] 1. The server accumulates data sent from the smartphone in a cloud database in real time.

[0143] 2. The server cleanses the accumulated data, imputes missing values, and removes outliers.

[0144] 3. The server performs data normalization, converting data in different units to a common scale.

[0145] 4. The server sorts the time series data and puts it in the correct order.

[0146] Step 4: Data analysis

[0147] Input: Preprocessed data

[0148] Output: Analysis results, evaluation indicators

[0149] Specific operation:

[0150] 1. The server analyzes the preprocessed data using machine learning models and statistical methods.

[0151] 2. Examples of models used: decision trees, random forests, time series analysis models.

[0152] 3. The server analyzes weight trends and detects long-term patterns of gain or loss.

[0153] 4. The server analyzes the dietary data, evaluates nutritional balance, and extracts nutrients that are deficient or in excess.

[0154] 5. The server analyzes walking patterns and evaluates daily variations in step count, speed, and stride length.

[0155] Step 5: Health Assessment

[0156] Input: Analysis results, evaluation indicators

[0157] Output: Health score, risk level

[0158] Specific operation:

[0159] 1. The server comprehensively evaluates the user's health condition based on the analysis results.

[0160] 2. The server calculates a health score based on weight change, nutritional balance, and walking patterns.

[0161] 3. The server generates a high risk score if there is rapid weight gain or nutritional imbalance.

[0162] 4. The server classifies the health risk level based on the assessment results (e.g., low risk, medium risk, high risk).

[0163] Step 6: Generate health improvement advice

[0164] Input: Assessment results, risk level

[0165] Output: Specific health improvement advice

[0166] Specific operation:

[0167] 1. The server uses the generative AI model to generate specific health improvement advice based on the evaluation results.

[0168] 2. Input a prompt sentence into the generative AI model to generate specific advice. For example, "Your recent weight gain needs attention. In addition to limiting your calories, increase your intake of vegetables and fruits."

[0169] Step 7: Advice Notification

[0170] Input: Generated health improvement advice

[0171] Output: Notify user, gather feedback

[0172] Specific operation:

[0173] 1. The server transmits the generated advice or warning to the portable information processing device.

[0174] 2. The user checks the advice through a dedicated app and sends feedback or additional questions as needed.

[0175] Step 8: User feedback and continuous improvement

[0176] Input: User feedback, ongoing data collection

[0177] Output: Advice based on feedback, continuously improving the user's health

[0178] Specific operation:

[0179] 1. Improve your lifestyle based on the advice you receive.

[0180] 2. The server receives feedback from the user, analyzes it, and provides an appropriate answer using a generative AI model.

[0181] 3. The user asks additional questions based on the feedback, and the server answers them, helping to keep track of the user's health status and making continuous improvements.

[0182] (Application example 1)

[0183] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0184] In modern society, many people's busy daily lives make it difficult to effectively manage their health. Providing real-time health management and advice during visits to physical stores is particularly challenging. While there are technologies available to collect health data using wearable devices and mobile terminals, there is a lack of methods to instantly provide this data to users in physical stores. As a result, early detection of health risks and the provision of appropriate health improvement advice may be delayed. Therefore, there is a need for a system that can instantly assess a user's health status in a physical store and provide appropriate advice.

[0185] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0186] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data and determining and evaluating the user's health condition, means for generating health improvement advice based on the determination and evaluation, and means for notifying the generated advice to the mobile terminal and a visual information device that can be viewed in the physical store. This makes it possible to instantly evaluate the user's health condition in the physical store and provide appropriate health improvement advice in real time.

[0187] A "wearable device" is an electronic device worn by a user that collects various data about daily life. Examples include smartwatches and smart weighing scales.

[0188] A "mobile device" is an electronic device that is carried by a user and provides various functions. Examples of such devices include smartphones and tablets.

[0189] A "cloud database" is a database that is accessible over the internet and provides data protection and scalability.

[0190] A "visual information device" is an electronic device that visually displays information to a user. Examples include smart glasses and digital signage.

[0191] A "generative artificial intelligence model" is an artificial intelligence system that uses machine learning algorithms to generate information in a format that is easy for users to understand.

[0192] "Preprocessing" is the process of preparing collected data in an analyzable format, including, for example, filling in missing values ​​and removing outliers.

[0193] "Health improvement advice" refers to specific instructions or recommendations for improving the user's health, such as advice on exercise or dietary changes.

[0194] "Health status" refers to the physical condition and physiological status of the user in their daily life, including, for example, weight, heart rate, and number of steps.

[0195] This invention is a system that uses wearable devices and mobile terminals to monitor a user's health status in real time in a physical store and provide health improvement advice. A specific implementation method of this system is described below.

[0196] Hardware and Software Configuration

[0197] It consists of a server, wearable devices, mobile terminals, a cloud database, and visual information devices.

[0198] Specific examples of hardware and software used

[0199] Wearable devices: smart watches, smart scales

[0200] Mobile device: Smartphone

[0201] Cloud Database: Internet-accessible database system

[0202] Visual information devices: smart glasses, digital signage

[0203] Software: Python 3.x, Requests library, REST API

[0204] Data collection and transmission

[0205] Users wear wearable devices (e.g., smart watches, smart scales) and install dedicated applications on their mobile devices (e.g., smartphones). These devices automatically collect data on the user's daily activities (weight, heart rate, number of steps, nutrition, etc.) and send it to the mobile device. The mobile device then sends the collected data to a cloud database for storage.

[0206] Data accumulation and preprocessing

[0207] The server stores data sent from mobile devices in a cloud database in real time, where it undergoes preprocessing (data cleansing, normalization, alignment of time-series data, etc.) and prepares the data in a format suitable for analysis.

[0208] Data analysis and evaluation

[0209] The server analyzes the preprocessed data using machine learning models and statistical methods to assess the user's health status, specifically weight changes, diet, walking patterns, etc. This allows it to detect sudden weight gain, nutritional imbalances, and abnormal walking patterns.

[0210] Generate health improvement advice

[0211] Based on the analysis results, the server uses a generative artificial intelligence model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0212] Advice Notification

[0213] The server then sends the generated health improvement advice and warnings to mobile devices and visual information devices, where users can check the advice in real time through a dedicated application on their mobile devices and visual information devices in physical stores.

[0214] Examples and prompts

[0215] For example, consider the case where a visitor with user ID 12345 visits a physical store and the following data is collected:

[0216] Weight: 85 kg

[0217] Heart rate: 105 bpm

[0218] Number of steps: 4000

[0219] Nutrients: "fast_food"

[0220] Based on this data, specific examples of prompt sentences for the generative artificial intelligence model are as follows:

[0221] "Evaluate the health risks of user ID: 12345 and generate advice. Weight: 85kg, Heart Rate: 105bpm, Steps: 4000, Nutrition: 'fast_food'"

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] The user sets up the wearable device and mobile device. The user installs a dedicated application on the mobile device and enters personal information to link the devices.

[0225] Input: User's personal information, initial settings of wearable device

[0226] Output: Devices are linked and working properly

[0227] Step 2:

[0228] Wearable devices and mobile devices collect various data about our daily lives: for example, smartwatches measure heart rate and steps taken, smart scales weigh us, and mobile devices collect GPS data.

[0229] Input: User's daily activity data (heart rate, number of steps, weight, distance traveled, etc.)

[0230] Output: Raw data collected

[0231] Step 3:

[0232] The mobile device sends the collected data to a cloud database, which is done automatically by a dedicated application.

[0233] Input: Raw data collected

[0234] Output: Data stored in a cloud database

[0235] Step 4:

[0236] The server preprocesses the data stored in the cloud database, for example, by performing data cleansing (filling in missing values, removing outliers) and data normalization (scaling values).

[0237] Input: Data stored in the cloud database

[0238] Output: Preprocessed data

[0239] Step 5:

[0240] The server analyzes the pre-processed data and assesses the user's health status, such as analyzing long-term weight trends, dietary habits, and walking patterns.

[0241] Input: Preprocessed data

[0242] Output: Analysis results (weight change trends, nutritional balance evaluation, abnormal walking pattern detection, etc.)

[0243] Step 6:

[0244] The server uses the generative AI model to generate health improvement advice, such as "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[0245] Input: Analysis results

[0246] Output: Generated health improvement advice

[0247] Step 7:

[0248] The server notifies the mobile terminal and visual information device of the generated advice, and the user can check it on the visual information device (e.g., smart glasses, digital signage) in the physical store and on the mobile terminal.

[0249] Input: Generated health improvement advice

[0250] Output: Advice displayed on mobile devices and visual information devices

[0251] Step 8:

[0252] The user can check the generated advice and work to improve their lifestyle. Feedback can be sent to the server if necessary.

[0253] Input: Health improvement advice, user feedback

[0254] Output: Improved lifestyle data (if any feedback is provided, it will be sent to the server)

[0255] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0256] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and combines it with an emotion engine to support health improvement. This system constantly monitors the user's health and emotional state, and provides appropriate advice to reduce health risks.

[0257] Device settings and integration

[0258] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[0259] Data collection and transmission

[0260] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0261] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[0262] The food logging app records the user's meal contents and sends the information to a mobile device.

[0263] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[0264] Your smartphone uses GPS data to record your distance traveled and walking speed.

[0265] The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[0266] Data accumulation and preprocessing

[0267] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[0268] Data Preprocessing and Sentiment Analysis

[0269] The server preprocesses the accumulated data, specifically by performing data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[0270] For emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment.

[0271] Data analysis

[0272] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[0273] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[0274] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0275] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0276] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[0277] Health assessment

[0278] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[0279] Generate health improvement advice

[0280] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0281] Advice Notification

[0282] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[0283] User feedback and continuous improvement

[0284] Users improve their lifestyle habits based on the advice they receive. Feedback and questions are also entered within the app, and the server uses a generative AI model to provide appropriate answers and additional advice. This enables continuous health management through interaction with the user.

[0285] As described above, this system combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage the user's health and emotional state in real time and provide specific and appropriate advice, allowing users to detect health risks early and take preventive measures.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] The user sets up the smart device (e.g., smart watch, smart scale) and mobile device initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.). They also configure the app to enable voice recognition and face recognition functions.

[0289] Step 2:

[0290] Terminals (smart devices and smartphones) collect data from daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed. Furthermore, the smartphone uses its built-in microphone and camera to collect voice data and facial expression data from the user.

[0291] Step 3:

[0292] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[0293] Step 4:

[0294] The server preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. The same preprocessing is performed on emotion data.

[0295] Step 5:

[0296] The emotion engine analyzes voice data and facial expression data to determine the user's emotional state. Specifically, it analyzes the tone and speed of the voice from the voice data and performs micro-expression analysis on the facial expression data to detect emotions such as joy, anger, and sadness. This emotional state data is integrated with other health data and used for a comprehensive assessment.

[0297] Step 6:

[0298] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[0299] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[0300] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0301] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0302] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[0303] Step 7:

[0304] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[0305] Step 8:

[0306] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0307] Step 9:

[0308] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[0309] Step 10:

[0310] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[0311] Step 11:

[0312] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] Conventional health management systems simply collect and analyze physical data (weight, number of steps, heart rate, etc.) and do not provide comprehensive health assessments that take into account the user's emotional state. As a result, the user's stress and mental health status may be overlooked, making it difficult to provide comprehensive health support. In addition, the health improvement advice provided is often general and lacking in specificity, making it difficult for users to understand. This presents a challenge, making it difficult to achieve effective health improvement.

[0316] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing voice data and facial expression data to determine the user's emotional state, means for analyzing the preprocessed data and emotional data to determine and evaluate the user's health state, means for generating health improvement advice based on the determination and evaluation, and means for notifying the mobile terminal of the generated advice. This makes it possible to comprehensively analyze the user's physical data and emotional data and provide comprehensive and specific health improvement advice.

[0317] A "wearable device" is an electronic device that is attached to the user's body or clothing and can collect various data from daily life.

[0318] A "mobile terminal" is a portable information processing device, such as a smartphone or tablet, that can receive and process data from a wearable device.

[0319] A "cloud database" is a database that can be accessed via the Internet and is a system that can store and manage various collected data.

[0320] "Preprocessing" refers to a series of processes carried out to prepare collected raw data in an analyzable format, and specifically includes filling in missing values, removing outliers, and normalizing the data.

[0321] An "emotion engine" is a software module or algorithm that analyzes voice data and facial expression data to determine the user's emotional state.

[0322] "Health status assessment" refers to the process of comprehensively assessing the user's physical and mental health status by analyzing the pre-processed data and emotional data.

[0323] "Health improvement advice" refers to advice that includes specific suggestions and instructions for improving the user's lifestyle and behavior based on the results of the health status assessment and evaluation.

[0324] A "generative artificial intelligence model" is an artificial intelligence technology that generates new information and sentences based on input data and prompts, and is used to provide health improvement advice to users in a format that is easy to understand.

[0325] The present invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from a user's daily life, and combines it with an emotion engine to support health improvement. In this embodiment, a specific method for reducing health risks by constantly monitoring the user's health and emotional state and providing appropriate advice will be described.

[0326] The system can be implemented using the following hardware and software combinations:

[0327] Wearable devices (e.g., smart watches, smart scales)

[0328] Mobile devices (e.g. smartphones)

[0329] Cloud Database

[0330] Emotion Engine

[0331] Machine learning models

[0332] Generative Artificial Intelligence Model

[0333] Device settings and integration

[0334] First, the user connects a wearable device such as a smartwatch or smart scale to their smartphone. After installing a dedicated app on their smartphone, they launch the app and enter their personal information, such as their name, age, gender, height, and weight. The data measured by each wearable device is then automatically sent to the mobile terminal.

[0335] Data collection and transmission

[0336] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0337] The smart scale allows users to weigh themselves every morning and sends the data to a mobile device via Bluetooth.

[0338] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[0339] The smartwatch measures the user's steps, heart rate, walking patterns, etc. and transmits the data to a mobile device in real time.

[0340] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[0341] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[0342] Data accumulation and preprocessing

[0343] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each piece of data is stored in a corresponding table; for example, weight data is stored in the "weight" table, and walking data is stored in the "walking" table.

[0344] Data Preprocessing and Sentiment Analysis

[0345] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. For emotion data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This result is integrated with other health data.

[0346] Data analysis

[0347] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[0348] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[0349] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[0350] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[0351] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[0352] Health status assessment and generation and notification of health improvement advice

[0353] The server makes a comprehensive assessment of the user's health condition based on the analysis results and uses a generative AI model to generate health improvement advice. An example of a prompt is, "The user's weight has increased dramatically recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation." The generative AI model then generates specific advice, such as, "Recent weight gain and increased stress have been confirmed. Try to get some moderate exercise and take time to relax." This advice is sent to the mobile device, and the user can check it through a dedicated app.

[0354] User feedback and continuous improvement

[0355] Users can improve their lifestyle habits based on the advice they receive. They can provide feedback and follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. Continuous health management is possible through interaction with the user.

[0356] As described above, the present invention combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage a user's health and emotional state in real time and provide specific and appropriate advice, thereby enabling users to detect health risks early and take preventive measures.

[0357] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0358] Step 1: Device Setup and Integration

[0359] The user sets up a wearable device such as a smart watch or smart scale and links it to a mobile device. The user installs and launches a dedicated app and enters personal information (name, age, gender, height, weight, etc.). The entered information is saved on the mobile device, and the wearable device is set up to automatically send measured data to the mobile device. The input here is the user's personal information and device measurement data, and the output is the completed setup state on the mobile device.

[0360] Step 2: Data collection and transmission

[0361] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0362] The smart scale allows users to weigh themselves every morning and transmits the data to a mobile device via Bluetooth.

[0363] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[0364] The smartwatch measures the user's steps, heart rate, and walking patterns and transmits the data in real time to a mobile device.

[0365] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[0366] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[0367] The input here is the various data measured by the device, and the output is all the data sent to the mobile terminal.

[0368] Step 3: Data accumulation and preprocessing

[0369] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each data item is stored in a corresponding table. The input is the raw data sent from the mobile device, and the output is the organized data accumulated in the cloud database.

[0370] Step 4: Data preprocessing and sentiment analysis

[0371] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values). The input here is the accumulated raw data, and the output is the preprocessed, clean data.

[0372] Regarding emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). The input is voice data and facial expression data, and the output is the determined emotional state.

[0373] Step 5: Data analysis

[0374] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[0375] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[0376] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[0377] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[0378] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[0379] The input here is the preprocessed data and emotion determination results, and the output is each analysis result.

[0380] Step 6: Assessing health status and generating health improvement advice

[0381] The server comprehensively evaluates the user's health condition based on the analysis results and generates health improvement advice using a generative AI model. For example, based on a prompt such as "The user's weight has increased rapidly recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation," the generative AI model generates specific advice (e.g., "Recent weight gain and increased stress have been confirmed. Try to get moderate exercise and spend time relaxing.").

[0382] The input here is the analysis results and a prompt for the generative AI model, and the output is the generated health improvement advice.

[0383] Step 7: Advice Notification

[0384] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app. The input here is the generated advice, and the output is a notification to the user.

[0385] Step 8: User feedback and continuous improvement

[0386] The user improves their lifestyle habits based on the advice they receive. The user can provide feedback or follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. The input here is feedback and questions from the user, and the output is answers and follow-up advice from the generative AI model. This enables continuous health management through interaction with the user.

[0387] The above is the flow of processing of the program of this system.

[0388] (Application example 2)

[0389] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] Current systems in health management and fitness gyms are not sufficient to monitor users' health and emotional states in real time and provide customized exercise menus and health advice suited to each individual user. Therefore, there is a need for a method to detect users' health risks early and efficiently provide appropriate improvement measures.

[0391] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0392] In this invention, the server includes means for collecting various data about the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation, means for notifying the mobile terminal of the generated advice, means for analyzing voice data and facial expression data to determine the user's emotional state, means for generating customized exercise menus and health advice based on the user's health data and emotional data, and means for collecting and analyzing member data in cooperation with smart fitness machines and sensors in the store and providing customized advice in real time, thereby enabling detailed analysis of the user's health status and emotional state and enabling truly personalized health management and fitness support.

[0393] A "wearable device" is a device that is worn on the user's body and is used to collect data from daily life.

[0394] A "mobile terminal" is an electronic device that can be carried by a user and is used to collect, process, and transmit data.

[0395] A "cloud database" is a remote data storage system for storing, managing, and accessing data over the Internet.

[0396] "Preprocessing" is the process of shaping and cleansing collected data to make it easier to analyze.

[0397] "Health status" refers to the overall state of a user's physical and mental health.

[0398] "Assessment and evaluation" is the process of analyzing the user's current situation based on collected and preprocessed data and identifying health risks, etc.

[0399] "Health improvement advice" refers to specific suggestions or instructions provided to improve the user's health.

[0400] "Voice data" is acoustic information generated by the user's voice.

[0401] "Facial expression data" is image data for capturing and analyzing the facial expressions of a user.

[0402] An "emotional state" is the emotional state a user is experiencing at a particular point in time.

[0403] A "customized exercise menu" is an exercise plan designed to suit an individual user's health condition and goals.

[0404] "Smart fitness machines" are devices used in fitness gyms that have advanced functions for collecting and analyzing exercise data.

[0405] A "sensor" is a device that senses a specific physical phenomenon and converts it into data.

[0406] "Real-time" refers to a state in which data is collected and processed immediately, without delay.

[0407] This invention provides a health support system customized for fitness gyms. The system uses wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and by combining this with an emotion engine, it monitors the user's health and emotional state in real time and provides appropriate advice.

[0408] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[0409] Mobile devices automatically collect the following data through daily activities: A smart scale measures the user's weight and sends the data to the mobile device. A food logging app records the user's diet and sends the information to the mobile device. A smartwatch measures the number of steps taken, heart rate, and walking patterns and sends the data to the mobile device. A smartphone uses GPS data to record distance traveled and walking speed. The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[0410] The server accumulates various data (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) sent from mobile devices in a cloud database in real time. The server then preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[0411] The emotion engine analyzes the voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment. The server then analyzes the pre-processed data and the emotion engine's determination result using machine learning models and statistical methods. The analysis includes trend analysis of weight change, evaluation of dietary habits, analysis of walking patterns, and evaluation of emotional state.

[0412] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high-risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state. The server uses a generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0413] The generated advice is provided through a prompt sentence as follows:

[0414] example:

[0415] "Recent data has confirmed that your weight is increasing, but your walking frequency has also increased, so it appears that you are exercising appropriately. However, your diet tends to be high in calories, so try to eat a more balanced diet. Your emotional data also suggests that your stress levels are a little high, so try incorporating relaxation techniques."

[0416] This system makes it easier for users to improve their daily lifestyle habits, providing comprehensive health management and fitness support.

[0417] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0418] Step 1:

[0419] The user sets up and links the wearable device and mobile device. The user installs a dedicated app on their smartphone and enters their personal information. This sets up the wearable device to send measured data to the mobile device.

[0420] Input: User's personal information, wearable device settings information

[0421] Output: Data integration setup is complete and data is ready to be sent

[0422] Step 2:

[0423] Mobile devices automatically collect health-related data through daily activities. A smart scale weighs you every morning and sends the data to your smartphone. A food logging app records what you eat. A smartwatch measures your steps, heart rate, and walking patterns. Your smartphone uses GPS data to record distance traveled and walking speed. Your mobile device uses a camera and microphone to collect audio and facial expression data.

[0424] Input: User's daily life activity data

[0425] Output: Various health-related data (weight, number of steps, dietary details, heart rate, distance traveled, voice data, facial expression data)

[0426] Step 3:

[0427] The mobile device sends the collected data to a cloud database, which stores health-related data in real time.

[0428] Input: Health-related data collected by mobile devices

[0429] Output: Data stored in a cloud database

[0430] Step 4:

[0431] The server preprocesses the data stored in the cloud database, performing data cleansing, imputing missing values, removing outliers, scaling values, and aligning time series data.

[0432] Input: Health-related data stored in a cloud database

[0433] Output: Preprocessed data (aligned time series data, cleansed data)

[0434] Step 5:

[0435] The server analyzes the voice data and facial expression data and uses an emotion engine to determine the user's emotional state, for example, analyzing emotions from the voice data and analyzing the facial expression data to determine the emotional state.

[0436] Input: Preprocessed speech and facial expression data

[0437] Output: Emotional state (e.g., happy, angry, sad, etc.)

[0438] Step 6:

[0439] The server analyzes the preprocessed health data and the emotional state calculated by the emotion engine using machine learning models and statistical methods to analyze weight trends, evaluate dietary habits, analyze walking patterns, and evaluate emotional states.

[0440] Input: Preprocessed health data, emotional state

[0441] Output: Health status analysis results (weight change, dietary assessment, walking patterns, emotional state assessment)

[0442] Step 7:

[0443] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score is generated. If the emotion engine detects a high stress state, it generates a warning.

[0444] Input: Health status analysis results

[0445] Output: Health score, health risk level

[0446] Step 8:

[0447] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in the form of prompt sentences.

[0448] Input: Health score, health risk level

[0449] Output: Health improvement advice (based on generative AI model)

[0450] Step 9:

[0451] The server sends the generated advice to the mobile device, where the user can check the advice through a dedicated app on the mobile device and obtain specific guidelines for improving their daily lifestyle habits.

[0452] Input: Health Improvement Advice

[0453] Output: Notification to mobile device, providing advice to user

[0454] This makes it easier for users to improve their daily lifestyle habits, enabling comprehensive health management and fitness support.

[0455] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0456] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0457] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0458] [Second embodiment]

[0459] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0460] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0461] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0462] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0463] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0464] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0465] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0466] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0467] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0468] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0469] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0470] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0471] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life to support health improvement. This system constantly monitors the user's health status and provides appropriate advice to reduce health risks.

[0472] Device settings and integration

[0473] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. Data measured by the wearable device is then sent to the mobile device and stored in a cloud database.

[0474] Data collection and transmission

[0475] Your devices (smart devices and smartphones) automatically collect the following data throughout your daily activities:

[0476] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[0477] The food logging app records the user's meal contents and sends the information to a mobile device.

[0478] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[0479] The smartphone measures distance traveled and walking speed using GPS and integrates the data.

[0480] Data accumulation and preprocessing

[0481] The server stores data sent from mobile devices in a cloud database in real time. Since the data stored in the database is difficult to analyze as is, it undergoes preprocessing. Specifically, this involves data cleansing (complementing missing values ​​and removing outliers), data normalization (scaling values), and sorting time-series data.

[0482] Data analysis

[0483] The server then analyzes the pre-processed data using machine learning models and statistical methods. This analysis evaluates the following:

[0484] Weight Trends: Analyze long-term weight patterns to detect abnormal weight changes.

[0485] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0486] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0487] Health assessment

[0488] The server evaluates the user's health condition based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if there is poor nutritional balance or abnormalities in walking patterns.

[0489] Generate health improvement advice

[0490] The server uses the generative AI model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0491] Advice Notification

[0492] The server notifies the mobile device of the generated advice and warnings, and the user can check these advices through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[0493] User feedback and continuous improvement

[0494] Users improve their lifestyle habits based on the advice they receive, and the server uses a generative AI model to provide appropriate answers to feedback and questions. This allows users to constantly monitor their health status and make continuous improvements.

[0495] As described above, this system utilizes wearable devices and mobile terminals to monitor and manage users' health status in real time and provide specific and appropriate advice, enabling users to detect health risks early and take preventive measures.

[0496] The processing flow will be explained below.

[0497] Step 1:

[0498] The user sets up the smart device (e.g., smart watch, smart scale) and mobile terminal initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.).

[0499] Step 2:

[0500] Terminals (smart devices and smartphones) collect data on daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed.

[0501] Step 3:

[0502] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, etc.) in a cloud database in real time. Specifically, it receives data through an API and stores each data item in the corresponding table.

[0503] Step 4:

[0504] The server preprocesses the accumulated data. Specifically, it performs data cleansing, imputes missing values, and removes outliers. It also performs data normalization and scales the data to a certain range. For time series data, it aligns the data based on timestamps.

[0505] Step 5:

[0506] The server then analyzes the preprocessed data using machine learning models and statistical methods, including trend analysis of weight change, nutritional evaluation of dietary data, and detection of abnormalities in walking patterns.

[0507] Step 6:

[0508] The server evaluates the user's health condition based on the analysis results, generating an abnormality score and calculating the health risk level in the event of weight gain, nutritional imbalance, or abnormal walking patterns.

[0509] Step 7:

[0510] The server uses the generative AI model to generate health improvement advice based on the analysis results. Specific advice might include, "Your calorie intake continues to be high. Include more vegetables and fruits in your diet," or "Your walking pattern is abnormal. We recommend that you see a doctor."

[0511] Step 8:

[0512] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[0513] Step 9:

[0514] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[0515] Step 10:

[0516] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[0517] Example 1

[0518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0519] In modern society, many people lead unhealthy lifestyles, resulting in an increase in lifestyle-related diseases, which has become a serious social problem. The present invention aims to solve these problems and improve the health status of users by providing a system that efficiently collects, analyzes, and evaluates health-related data from daily life and provides appropriate health improvement advice. In particular, highly accurate health management is possible by consistently performing the process from preprocessing, analysis, and evaluation of collected data to generating advice.

[0520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0521] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a portable information processing device, means for storing the collected data in a remote database, preprocessing means for performing data cleansing, data normalization, and time-series data alignment on the stored data, means for analyzing the preprocessed data using a machine learning model and statistical methods to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation using a generation AI model, and means for notifying the portable information processing device of the generated advice. This makes it possible to comprehensively monitor and evaluate the user's health status and provide individually optimized, specific health improvement advice.

[0522] A "wearable device" is an electronic device that can be worn to collect various data in daily life.

[0523] A "portable information processing device" is an electronic device that can be carried by a user and is used for performing information processing such as collecting, transmitting, and displaying data.

[0524] "Data cleansing" is the process of detecting missing or outliers in collected data and then completing or removing them.

[0525] "Data normalization" is the process of converting data in different units to a common scale to make it easier to analyze.

[0526] "Time series data sorting" is the process of sorting collected data into the correct order based on the time axis.

[0527] A "machine learning model" is an algorithm or mathematical model used to analyze collected data and make predictions or classifications.

[0528] "Statistical methods" are mathematical and statistical techniques used to analyze, evaluate, and interpret data.

[0529] "Assessment and evaluation" is the process of determining the user's health condition based on the analyzed data and making an evaluation of that condition.

[0530] A "generative AI model" is a model that uses artificial intelligence to generate new information and advice from data.

[0531] A "remote database" is a remote data storage system connected to a server for storing and managing collected data.

[0532] "Data preprocessing" is a series of processes performed to convert raw data into an analyzable form.

[0533] MODE FOR CARRYING OUT THE INVENTION

[0534] This invention is a system that collects, analyzes, and evaluates health-related data from a user's daily life and provides appropriate health improvement advice. This system supports the user's health management by collecting data using wearable devices and portable information processing devices and analyzing the data on a server.

[0535] Device settings and integration

[0536] First, the user sets up and links the wearable device and portable information processing device. Specific hardware examples include a smartwatch and a smart weighing scale. The user then installs a dedicated app on their smartphone and enters their personal information. The health-related data measured by the wearable device is then sent to the portable information processing device and stored in a cloud database.

[0537] Data collection and transmission

[0538] Wearable devices and portable information processing devices (e.g., smartwatches, smart scales, smartphones) collect daily activity data. Specifically, smart scales measure weight every morning and send the data to a smartphone, food log apps record meal contents and send the data to a smartphone, and smartwatches measure the number of steps taken, heart rate, and walking patterns and send the data to a smartphone. Smartphones also use GPS information to measure distance traveled and walking speed.

[0539] Data accumulation and preprocessing

[0540] The server accumulates data sent from mobile devices in a cloud database in real time, then performs preprocessing such as data cleansing (filling in missing values ​​and removing outliers), data normalization (scaling values), and time-series data alignment, ensuring the consistency and accuracy of the data and making it suitable for analysis.

[0541] Data analysis

[0542] The server then analyzes the preprocessed data using machine learning models and statistical methods. For example, it analyzes weight trends, dietary habits, and walking patterns. Specific models used include decision trees, random forests, and time series analysis models. The analysis results in an assessment of the user's health status.

[0543] Health assessment

[0544] The server evaluates the user's health status based on the analysis results and calculates an abnormality score and health risk level. For example, if sudden weight gain or nutritional imbalance is detected, a warning score will be generated. Abnormalities in the number of steps taken each day and walking patterns will also be detected.

[0545] Generate health improvement advice

[0546] The server uses the generative AI model to generate specific health improvement advice. This advice is presented in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0547] Advice Notification

[0548] The server notifies the mobile information processing device of the generated advice and warnings. The user can check these advice and warnings through a dedicated app. If necessary, the user can also send feedback and ask follow-up questions.

[0549] User feedback and continuous improvement

[0550] Users improve their lifestyle habits based on the advice they receive. The server uses a generative AI model based on feedback from users to provide appropriate answers. This allows users to constantly monitor their health status and make continuous improvements.

[0551] Examples of prompt statements

[0552] Example prompts to input to a generative AI model:

[0553] "I'm a 30-year-old man and I've recently gained a lot of weight. Can you give me some specific advice on how to manage my weight?"

[0554] The above is a specific embodiment for carrying out the present invention. This system allows users to monitor their own health condition in detail and take appropriate and specific measures to improve their health.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Program processing steps

[0557] Step 1: Device Setup and Integration

[0558] Input: Wearable device, smartphone, dedicated app

[0559] Output: Initial setup completed, device connectivity established

[0560] Specific operation:

[0561] 1. The user has a wearable device (e.g., smart watch, smart scale) at hand.

[0562] 2. The user installs the dedicated app on their smartphone and launches it.

[0563] 3. After launching the app, the user performs initial setup by entering personal information such as name, date of birth, gender, height, and weight.

[0564] 4. The user enables the Bluetooth function on their smartphone and pairs it with the wearable device, establishing a connection between the devices.

[0565] Step 2: Data collection and transmission

[0566] Input: User's daily activities (weight measurement, eating, walking, etc.), sensor data from wearable devices

[0567] Output: Measurement data, transmitted data

[0568] Specific operation:

[0569] 1. A smart scale measures a user's weight every morning and sends the data to a smartphone.

[0570] 2. The food logging app records the user's meal contents and sends the data to their smartphone.

[0571] 3. The smartwatch measures the number of steps, heart rate, and walking patterns and sends the data to your smartphone.

[0572] 4. Smartphones will use GPS to measure distance traveled and walking speed and centralize the data.

[0573] Step 3: Data accumulation and preprocessing

[0574] Input: Measurement data from a smartphone

[0575] Output: Accumulated data, preprocessed data

[0576] Specific operation:

[0577] 1. The server accumulates data sent from the smartphone in a cloud database in real time.

[0578] 2. The server cleanses the accumulated data, imputes missing values, and removes outliers.

[0579] 3. The server performs data normalization, converting data in different units to a common scale.

[0580] 4. The server sorts the time series data and puts it in the correct order.

[0581] Step 4: Data analysis

[0582] Input: Preprocessed data

[0583] Output: Analysis results, evaluation indicators

[0584] Specific operation:

[0585] 1. The server analyzes the preprocessed data using machine learning models and statistical methods.

[0586] 2. Examples of models used: decision trees, random forests, time series analysis models.

[0587] 3. The server analyzes weight trends and detects long-term patterns of gain or loss.

[0588] 4. The server analyzes the dietary data, evaluates nutritional balance, and extracts nutrients that are deficient or in excess.

[0589] 5. The server analyzes walking patterns and evaluates daily variations in step count, speed, and stride length.

[0590] Step 5: Health Assessment

[0591] Input: Analysis results, evaluation indicators

[0592] Output: Health score, risk level

[0593] Specific operation:

[0594] 1. The server comprehensively evaluates the user's health condition based on the analysis results.

[0595] 2. The server calculates a health score based on weight change, nutritional balance, and walking patterns.

[0596] 3. The server generates a high risk score if there is rapid weight gain or nutritional imbalance.

[0597] 4. The server classifies the health risk level based on the assessment results (e.g., low risk, medium risk, high risk).

[0598] Step 6: Generate health improvement advice

[0599] Input: Assessment results, risk level

[0600] Output: Specific health improvement advice

[0601] Specific operation:

[0602] 1. The server uses the generative AI model to generate specific health improvement advice based on the evaluation results.

[0603] 2. Input a prompt sentence into the generative AI model to generate specific advice. For example, "Your recent weight gain needs attention. In addition to limiting your calories, increase your intake of vegetables and fruits."

[0604] Step 7: Advice Notification

[0605] Input: Generated health improvement advice

[0606] Output: Notify user, gather feedback

[0607] Specific operation:

[0608] 1. The server transmits the generated advice or warning to the portable information processing device.

[0609] 2. The user checks the advice through a dedicated app and sends feedback or additional questions as needed.

[0610] Step 8: User feedback and continuous improvement

[0611] Input: User feedback, ongoing data collection

[0612] Output: Advice based on feedback, continuously improving the user's health

[0613] Specific operation:

[0614] 1. Improve your lifestyle based on the advice you receive.

[0615] 2. The server receives feedback from the user, analyzes it, and provides an appropriate answer using a generative AI model.

[0616] 3. The user asks additional questions based on the feedback, and the server answers them, helping to keep track of the user's health status and making continuous improvements.

[0617] (Application example 1)

[0618] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] In modern society, many people's busy daily lives make it difficult to effectively manage their health. Providing real-time health management and advice during visits to physical stores is particularly challenging. While there are technologies available to collect health data using wearable devices and mobile terminals, there is a lack of methods to instantly provide this data to users in physical stores. As a result, early detection of health risks and the provision of appropriate health improvement advice may be delayed. Therefore, there is a need for a system that can instantly assess a user's health status in a physical store and provide appropriate advice.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0621] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data and determining and evaluating the user's health condition, means for generating health improvement advice based on the determination and evaluation, and means for notifying the generated advice to the mobile terminal and a visual information device that can be viewed in the physical store. This makes it possible to instantly evaluate the user's health condition in the physical store and provide appropriate health improvement advice in real time.

[0622] A "wearable device" is an electronic device worn by a user that collects various data about daily life. Examples include smartwatches and smart weighing scales.

[0623] A "mobile device" is an electronic device that is carried by a user and provides various functions. Examples of such devices include smartphones and tablets.

[0624] A "cloud database" is a database that is accessible over the internet and provides data protection and scalability.

[0625] A "visual information device" is an electronic device that visually displays information to a user. Examples include smart glasses and digital signage.

[0626] A "generative artificial intelligence model" is an artificial intelligence system that uses machine learning algorithms to generate information in a format that is easy for users to understand.

[0627] "Preprocessing" is the process of preparing collected data in an analyzable format, including, for example, filling in missing values ​​and removing outliers.

[0628] "Health improvement advice" refers to specific instructions or recommendations for improving the user's health, such as advice on exercise or dietary changes.

[0629] "Health status" refers to the physical condition and physiological status of the user in their daily life, including, for example, weight, heart rate, and number of steps.

[0630] This invention is a system that uses wearable devices and mobile terminals to monitor a user's health status in real time in a physical store and provide health improvement advice. A specific implementation method of this system is described below.

[0631] Hardware and Software Configuration

[0632] It consists of a server, wearable devices, mobile terminals, a cloud database, and visual information devices.

[0633] Specific examples of hardware and software used

[0634] Wearable devices: smart watches, smart scales

[0635] Mobile device: Smartphone

[0636] Cloud Database: Internet-accessible database system

[0637] Visual information devices: smart glasses, digital signage

[0638] Software: Python 3.x, Requests library, REST API

[0639] Data collection and transmission

[0640] Users wear wearable devices (e.g., smart watches, smart scales) and install dedicated applications on their mobile devices (e.g., smartphones). These devices automatically collect data on the user's daily activities (weight, heart rate, number of steps, nutrition, etc.) and send it to the mobile device. The mobile device then sends the collected data to a cloud database for storage.

[0641] Data accumulation and preprocessing

[0642] The server stores data sent from mobile devices in a cloud database in real time, where it undergoes preprocessing (data cleansing, normalization, alignment of time-series data, etc.) and prepares the data in a format suitable for analysis.

[0643] Data analysis and evaluation

[0644] The server analyzes the preprocessed data using machine learning models and statistical methods to assess the user's health status, specifically weight changes, diet, walking patterns, etc. This allows it to detect sudden weight gain, nutritional imbalances, and abnormal walking patterns.

[0645] Generate health improvement advice

[0646] Based on the analysis results, the server uses a generative artificial intelligence model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0647] Advice Notification

[0648] The server then sends the generated health improvement advice and warnings to mobile devices and visual information devices, where users can check the advice in real time through a dedicated application on their mobile devices and visual information devices in physical stores.

[0649] Examples and prompts

[0650] For example, consider the case where a visitor with user ID 12345 visits a physical store and the following data is collected:

[0651] Weight: 85 kg

[0652] Heart rate: 105 bpm

[0653] Number of steps: 4000

[0654] Nutrients: "fast_food"

[0655] Based on this data, specific examples of prompt sentences for the generative artificial intelligence model are as follows:

[0656] "Evaluate the health risks of user ID: 12345 and generate advice. Weight: 85kg, Heart Rate: 105bpm, Steps: 4000, Nutrition: 'fast_food'"

[0657] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0658] Step 1:

[0659] The user sets up the wearable device and mobile device. The user installs a dedicated application on the mobile device and enters personal information to link the devices.

[0660] Input: User's personal information, initial settings of wearable device

[0661] Output: Devices are linked and working properly

[0662] Step 2:

[0663] Wearable devices and mobile devices collect various data about our daily lives: for example, smartwatches measure heart rate and steps taken, smart scales weigh us, and mobile devices collect GPS data.

[0664] Input: User's daily activity data (heart rate, number of steps, weight, distance traveled, etc.)

[0665] Output: Raw data collected

[0666] Step 3:

[0667] The mobile device sends the collected data to a cloud database, which is done automatically by a dedicated application.

[0668] Input: Raw data collected

[0669] Output: Data stored in a cloud database

[0670] Step 4:

[0671] The server preprocesses the data stored in the cloud database, for example, by performing data cleansing (filling in missing values, removing outliers) and data normalization (scaling values).

[0672] Input: Data stored in the cloud database

[0673] Output: Preprocessed data

[0674] Step 5:

[0675] The server analyzes the pre-processed data and assesses the user's health status, such as analyzing long-term weight trends, dietary habits, and walking patterns.

[0676] Input: Preprocessed data

[0677] Output: Analysis results (weight change trends, nutritional balance evaluation, abnormal walking pattern detection, etc.)

[0678] Step 6:

[0679] The server uses the generative AI model to generate health improvement advice, such as "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[0680] Input: Analysis results

[0681] Output: Generated health improvement advice

[0682] Step 7:

[0683] The server notifies the mobile terminal and visual information device of the generated advice, and the user can check it on the visual information device (e.g., smart glasses, digital signage) in the physical store and on the mobile terminal.

[0684] Input: Generated health improvement advice

[0685] Output: Advice displayed on mobile devices and visual information devices

[0686] Step 8:

[0687] The user can check the generated advice and work to improve their lifestyle. Feedback can be sent to the server if necessary.

[0688] Input: Health improvement advice, user feedback

[0689] Output: Improved lifestyle data (if any feedback is provided, it will be sent to the server)

[0690] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0691] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and combines it with an emotion engine to support health improvement. This system constantly monitors the user's health and emotional state, and provides appropriate advice to reduce health risks.

[0692] Device settings and integration

[0693] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[0694] Data collection and transmission

[0695] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0696] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[0697] The food logging app records the user's meal contents and sends the information to a mobile device.

[0698] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[0699] Your smartphone uses GPS data to record your distance traveled and walking speed.

[0700] The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[0701] Data accumulation and preprocessing

[0702] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[0703] Data Preprocessing and Sentiment Analysis

[0704] The server preprocesses the accumulated data, specifically by performing data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[0705] For emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment.

[0706] Data analysis

[0707] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[0708] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[0709] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0710] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0711] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[0712] Health assessment

[0713] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[0714] Generate health improvement advice

[0715] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0716] Advice Notification

[0717] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[0718] User feedback and continuous improvement

[0719] Users improve their lifestyle habits based on the advice they receive. Feedback and questions are also entered within the app, and the server uses a generative AI model to provide appropriate answers and additional advice. This enables continuous health management through interaction with the user.

[0720] As described above, this system combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage the user's health and emotional state in real time and provide specific and appropriate advice, allowing users to detect health risks early and take preventive measures.

[0721] The processing flow will be explained below.

[0722] Step 1:

[0723] The user sets up the smart device (e.g., smart watch, smart scale) and mobile device initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.). They also configure the app to enable voice recognition and face recognition functions.

[0724] Step 2:

[0725] Terminals (smart devices and smartphones) collect data from daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed. Furthermore, the smartphone uses its built-in microphone and camera to collect voice data and facial expression data from the user.

[0726] Step 3:

[0727] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[0728] Step 4:

[0729] The server preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. The same preprocessing is performed on emotion data.

[0730] Step 5:

[0731] The emotion engine analyzes voice data and facial expression data to determine the user's emotional state. Specifically, it analyzes the tone and speed of the voice from the voice data and performs micro-expression analysis on the facial expression data to detect emotions such as joy, anger, and sadness. This emotional state data is integrated with other health data and used for a comprehensive assessment.

[0732] Step 6:

[0733] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[0734] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[0735] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0736] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0737] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[0738] Step 7:

[0739] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[0740] Step 8:

[0741] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0742] Step 9:

[0743] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[0744] Step 10:

[0745] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[0746] Step 11:

[0747] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[0748] Example 2

[0749] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0750] Conventional health management systems simply collect and analyze physical data (weight, number of steps, heart rate, etc.) and do not provide comprehensive health assessments that take into account the user's emotional state. As a result, the user's stress and mental health status may be overlooked, making it difficult to provide comprehensive health support. In addition, the health improvement advice provided is often general and lacking in specificity, making it difficult for users to understand. This presents a challenge, making it difficult to achieve effective health improvement.

[0751] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing voice data and facial expression data to determine the user's emotional state, means for analyzing the preprocessed data and emotional data to determine and evaluate the user's health state, means for generating health improvement advice based on the determination and evaluation, and means for notifying the mobile terminal of the generated advice. This makes it possible to comprehensively analyze the user's physical data and emotional data and provide comprehensive and specific health improvement advice.

[0752] A "wearable device" is an electronic device that is attached to the user's body or clothing and can collect various data from daily life.

[0753] A "mobile terminal" is a portable information processing device, such as a smartphone or tablet, that can receive and process data from a wearable device.

[0754] A "cloud database" is a database that can be accessed via the Internet and is a system that can store and manage various collected data.

[0755] "Preprocessing" refers to a series of processes carried out to prepare collected raw data in an analyzable format, and specifically includes filling in missing values, removing outliers, and normalizing the data.

[0756] An "emotion engine" is a software module or algorithm that analyzes voice data and facial expression data to determine the user's emotional state.

[0757] "Health status assessment" refers to the process of comprehensively assessing the user's physical and mental health status by analyzing the pre-processed data and emotional data.

[0758] "Health improvement advice" refers to advice that includes specific suggestions and instructions for improving the user's lifestyle and behavior based on the results of the health status assessment and evaluation.

[0759] A "generative artificial intelligence model" is an artificial intelligence technology that generates new information and sentences based on input data and prompts, and is used to provide health improvement advice to users in a format that is easy to understand.

[0760] The present invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from a user's daily life, and combines it with an emotion engine to support health improvement. In this embodiment, a specific method for reducing health risks by constantly monitoring the user's health and emotional state and providing appropriate advice will be described.

[0761] The system can be implemented using the following hardware and software combinations:

[0762] Wearable devices (e.g., smart watches, smart scales)

[0763] Mobile devices (e.g. smartphones)

[0764] Cloud Database

[0765] Emotion Engine

[0766] Machine learning models

[0767] Generative Artificial Intelligence Model

[0768] Device settings and integration

[0769] First, the user connects a wearable device such as a smartwatch or smart scale to their smartphone. After installing a dedicated app on their smartphone, they launch the app and enter their personal information, such as their name, age, gender, height, and weight. The data measured by each wearable device is then automatically sent to the mobile terminal.

[0770] Data collection and transmission

[0771] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0772] The smart scale allows users to weigh themselves every morning and sends the data to a mobile device via Bluetooth.

[0773] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[0774] The smartwatch measures the user's steps, heart rate, walking patterns, etc. and transmits the data to a mobile device in real time.

[0775] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[0776] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[0777] Data accumulation and preprocessing

[0778] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each piece of data is stored in a corresponding table; for example, weight data is stored in the "weight" table, and walking data is stored in the "walking" table.

[0779] Data Preprocessing and Sentiment Analysis

[0780] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. For emotion data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This result is integrated with other health data.

[0781] Data analysis

[0782] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[0783] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[0784] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[0785] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[0786] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[0787] Health status assessment and generation and notification of health improvement advice

[0788] The server makes a comprehensive assessment of the user's health condition based on the analysis results and uses a generative AI model to generate health improvement advice. An example of a prompt is, "The user's weight has increased dramatically recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation." The generative AI model then generates specific advice, such as, "Recent weight gain and increased stress have been confirmed. Try to get some moderate exercise and take time to relax." This advice is sent to the mobile device, and the user can check it through a dedicated app.

[0789] User feedback and continuous improvement

[0790] Users can improve their lifestyle habits based on the advice they receive. They can provide feedback and follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. Continuous health management is possible through interaction with the user.

[0791] As described above, the present invention combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage a user's health and emotional state in real time and provide specific and appropriate advice, thereby enabling users to detect health risks early and take preventive measures.

[0792] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0793] Step 1: Device Setup and Integration

[0794] The user sets up a wearable device such as a smart watch or smart scale and links it to a mobile device. The user installs and launches a dedicated app and enters personal information (name, age, gender, height, weight, etc.). The entered information is saved on the mobile device, and the wearable device is set up to automatically send measured data to the mobile device. The input here is the user's personal information and device measurement data, and the output is the completed setup state on the mobile device.

[0795] Step 2: Data collection and transmission

[0796] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0797] The smart scale allows users to weigh themselves every morning and transmits the data to a mobile device via Bluetooth.

[0798] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[0799] The smartwatch measures the user's steps, heart rate, and walking patterns and transmits the data in real time to a mobile device.

[0800] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[0801] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[0802] The input here is the various data measured by the device, and the output is all the data sent to the mobile terminal.

[0803] Step 3: Data accumulation and preprocessing

[0804] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each data item is stored in a corresponding table. The input is the raw data sent from the mobile device, and the output is the organized data accumulated in the cloud database.

[0805] Step 4: Data preprocessing and sentiment analysis

[0806] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values). The input here is the accumulated raw data, and the output is the preprocessed, clean data.

[0807] Regarding emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). The input is voice data and facial expression data, and the output is the determined emotional state.

[0808] Step 5: Data analysis

[0809] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[0810] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[0811] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[0812] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[0813] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[0814] The input here is the preprocessed data and emotion determination results, and the output is each analysis result.

[0815] Step 6: Assessing health status and generating health improvement advice

[0816] The server comprehensively evaluates the user's health condition based on the analysis results and generates health improvement advice using a generative AI model. For example, based on a prompt such as "The user's weight has increased rapidly recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation," the generative AI model generates specific advice (e.g., "Recent weight gain and increased stress have been confirmed. Try to get moderate exercise and spend time relaxing.").

[0817] The input here is the analysis results and a prompt for the generative AI model, and the output is the generated health improvement advice.

[0818] Step 7: Advice Notification

[0819] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app. The input here is the generated advice, and the output is a notification to the user.

[0820] Step 8: User feedback and continuous improvement

[0821] The user improves their lifestyle habits based on the advice they receive. The user can provide feedback or follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. The input here is feedback and questions from the user, and the output is answers and follow-up advice from the generative AI model. This enables continuous health management through interaction with the user.

[0822] The above is the flow of processing of the program of this system.

[0823] (Application example 2)

[0824] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0825] Current systems in health management and fitness gyms are not sufficient to monitor users' health and emotional states in real time and provide customized exercise menus and health advice suited to each individual user. Therefore, there is a need for a method to detect users' health risks early and efficiently provide appropriate improvement measures.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0827] In this invention, the server includes means for collecting various data about the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation, means for notifying the mobile terminal of the generated advice, means for analyzing voice data and facial expression data to determine the user's emotional state, means for generating customized exercise menus and health advice based on the user's health data and emotional data, and means for collecting and analyzing member data in cooperation with smart fitness machines and sensors in the store and providing customized advice in real time, thereby enabling detailed analysis of the user's health status and emotional state and enabling truly personalized health management and fitness support.

[0828] A "wearable device" is a device that is worn on the user's body and is used to collect data from daily life.

[0829] A "mobile terminal" is an electronic device that can be carried by a user and is used to collect, process, and transmit data.

[0830] A "cloud database" is a remote data storage system for storing, managing, and accessing data over the Internet.

[0831] "Preprocessing" is the process of shaping and cleansing collected data to make it easier to analyze.

[0832] "Health status" refers to the overall state of a user's physical and mental health.

[0833] "Assessment and evaluation" is the process of analyzing the user's current situation based on collected and preprocessed data and identifying health risks, etc.

[0834] "Health improvement advice" refers to specific suggestions or instructions provided to improve the user's health.

[0835] "Voice data" is acoustic information generated by the user's voice.

[0836] "Facial expression data" is image data for capturing and analyzing the facial expressions of a user.

[0837] An "emotional state" is the emotional state a user is experiencing at a particular point in time.

[0838] A "customized exercise menu" is an exercise plan designed to suit an individual user's health condition and goals.

[0839] "Smart fitness machines" are devices used in fitness gyms that have advanced functions for collecting and analyzing exercise data.

[0840] A "sensor" is a device that senses a specific physical phenomenon and converts it into data.

[0841] "Real-time" refers to a state in which data is collected and processed immediately, without delay.

[0842] This invention provides a health support system customized for fitness gyms. The system uses wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and by combining this with an emotion engine, it monitors the user's health and emotional state in real time and provides appropriate advice.

[0843] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[0844] Mobile devices automatically collect the following data through daily activities: A smart scale measures the user's weight and sends the data to the mobile device. A food logging app records the user's diet and sends the information to the mobile device. A smartwatch measures the number of steps taken, heart rate, and walking patterns and sends the data to the mobile device. A smartphone uses GPS data to record distance traveled and walking speed. The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[0845] The server accumulates various data (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) sent from mobile devices in a cloud database in real time. The server then preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[0846] The emotion engine analyzes the voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment. The server then analyzes the pre-processed data and the emotion engine's determination result using machine learning models and statistical methods. The analysis includes trend analysis of weight change, evaluation of dietary habits, analysis of walking patterns, and evaluation of emotional state.

[0847] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high-risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state. The server uses a generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[0848] The generated advice is provided through a prompt sentence as follows:

[0849] example:

[0850] "Recent data has confirmed that your weight is increasing, but your walking frequency has also increased, so it appears that you are exercising appropriately. However, your diet tends to be high in calories, so try to eat a more balanced diet. Your emotional data also suggests that your stress levels are a little high, so try incorporating relaxation techniques."

[0851] This system makes it easier for users to improve their daily lifestyle habits, providing comprehensive health management and fitness support.

[0852] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0853] Step 1:

[0854] The user sets up and links the wearable device and mobile device. The user installs a dedicated app on their smartphone and enters their personal information. This sets up the wearable device to send measured data to the mobile device.

[0855] Input: User's personal information, wearable device settings information

[0856] Output: Data integration setup is complete and data is ready to be sent

[0857] Step 2:

[0858] Mobile devices automatically collect health-related data through daily activities. A smart scale weighs you every morning and sends the data to your smartphone. A food logging app records what you eat. A smartwatch measures your steps, heart rate, and walking patterns. Your smartphone uses GPS data to record distance traveled and walking speed. Your mobile device uses a camera and microphone to collect audio and facial expression data.

[0859] Input: User's daily life activity data

[0860] Output: Various health-related data (weight, number of steps, dietary details, heart rate, distance traveled, voice data, facial expression data)

[0861] Step 3:

[0862] The mobile device sends the collected data to a cloud database, which stores health-related data in real time.

[0863] Input: Health-related data collected by mobile devices

[0864] Output: Data stored in a cloud database

[0865] Step 4:

[0866] The server preprocesses the data stored in the cloud database, performing data cleansing, imputing missing values, removing outliers, scaling values, and aligning time series data.

[0867] Input: Health-related data stored in a cloud database

[0868] Output: Preprocessed data (aligned time series data, cleansed data)

[0869] Step 5:

[0870] The server analyzes the voice data and facial expression data and uses an emotion engine to determine the user's emotional state, for example, analyzing emotions from the voice data and analyzing the facial expression data to determine the emotional state.

[0871] Input: Preprocessed speech and facial expression data

[0872] Output: Emotional state (e.g., happy, angry, sad, etc.)

[0873] Step 6:

[0874] The server analyzes the preprocessed health data and the emotional state calculated by the emotion engine using machine learning models and statistical methods to analyze weight trends, evaluate dietary habits, analyze walking patterns, and evaluate emotional states.

[0875] Input: Preprocessed health data, emotional state

[0876] Output: Health status analysis results (weight change, dietary assessment, walking patterns, emotional state assessment)

[0877] Step 7:

[0878] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score is generated. If the emotion engine detects a high stress state, it generates a warning.

[0879] Input: Health status analysis results

[0880] Output: Health score, health risk level

[0881] Step 8:

[0882] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in the form of prompt sentences.

[0883] Input: Health score, health risk level

[0884] Output: Health improvement advice (based on generative AI model)

[0885] Step 9:

[0886] The server sends the generated advice to the mobile device, where the user can check the advice through a dedicated app on the mobile device and obtain specific guidelines for improving their daily lifestyle habits.

[0887] Input: Health Improvement Advice

[0888] Output: Notification to mobile device, providing advice to user

[0889] This makes it easier for users to improve their daily lifestyle habits, enabling comprehensive health management and fitness support.

[0890] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0891] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0892] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0893] [Third embodiment]

[0894] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0895] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0896] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0897] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0898] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0899] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0900] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0901] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0902] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0903] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0904] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0905] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0906] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life to support health improvement. This system constantly monitors the user's health status and provides appropriate advice to reduce health risks.

[0907] Device settings and integration

[0908] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. Data measured by the wearable device is then sent to the mobile device and stored in a cloud database.

[0909] Data collection and transmission

[0910] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[0911] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[0912] The food logging app records the user's meal contents and sends the information to a mobile device.

[0913] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[0914] The smartphone measures distance traveled and walking speed using GPS and integrates the data.

[0915] Data accumulation and preprocessing

[0916] The server stores data sent from mobile devices in a cloud database in real time. Since the data stored in the database is difficult to analyze as is, it undergoes preprocessing. Specifically, this involves data cleansing (complementing missing values ​​and removing outliers), data normalization (scaling values), and sorting time-series data.

[0917] Data analysis

[0918] The server then analyzes the pre-processed data using machine learning models and statistical methods. This analysis evaluates the following:

[0919] Weight Trends: Analyze long-term weight patterns to detect abnormal weight changes.

[0920] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[0921] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[0922] Health assessment

[0923] The server evaluates the user's health condition based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if there is poor nutritional balance or abnormalities in walking patterns.

[0924] Generate health improvement advice

[0925] The server uses the generative AI model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[0926] Advice Notice

[0927] The server notifies the mobile device of the generated advice and warnings, and the user can check these advices through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[0928] User feedback and continuous improvement

[0929] Users improve their lifestyle habits based on the advice they receive, and the server uses a generative AI model to provide appropriate answers to feedback and questions. This allows users to constantly monitor their health status and make continuous improvements.

[0930] As described above, this system utilizes wearable devices and mobile terminals to monitor and manage users' health status in real time and provide specific and appropriate advice, enabling users to detect health risks early and take preventive measures.

[0931] The processing flow will be explained below.

[0932] Step 1:

[0933] The user sets up the smart device (e.g., smart watch, smart scale) and mobile terminal initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.).

[0934] Step 2:

[0935] Terminals (smart devices and smartphones) collect data on daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed.

[0936] Step 3:

[0937] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, etc.) in a cloud database in real time. Specifically, it receives data through an API and stores each data item in the corresponding table.

[0938] Step 4:

[0939] The server preprocesses the accumulated data. Specifically, it performs data cleansing, fills in missing values, and removes outliers. It also performs data normalization and scales the data to a certain range. For time series data, it aligns the data based on timestamps.

[0940] Step 5:

[0941] The server then analyzes the preprocessed data using machine learning models and statistical methods, including trend analysis of weight change, nutritional evaluation of dietary data, and detection of abnormalities in walking patterns.

[0942] Step 6:

[0943] The server evaluates the user's health condition based on the analysis results, generating an abnormality score and calculating the health risk level in the event of weight gain, nutritional imbalance, or abnormal walking patterns.

[0944] Step 7:

[0945] The server uses the generative AI model to generate health improvement advice based on the analysis results. Specific advice might include, "Your calorie intake continues to be high. Include more vegetables and fruits in your diet," or "Your walking pattern is abnormal. We recommend that you see a doctor."

[0946] Step 8:

[0947] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[0948] Step 9:

[0949] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[0950] Step 10:

[0951] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[0952] Example 1

[0953] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0954] In modern society, many people lead unhealthy lifestyles, resulting in an increase in lifestyle-related diseases, which has become a serious social problem. The present invention aims to solve these problems and improve the health status of users by providing a system that efficiently collects, analyzes, and evaluates health-related data from daily life and provides appropriate health improvement advice. In particular, highly accurate health management is possible by consistently performing the process from preprocessing, analysis, and evaluation of collected data to generating advice.

[0955] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0956] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a portable information processing device, means for storing the collected data in a remote database, preprocessing means for performing data cleansing, data normalization, and time-series data alignment on the stored data, means for analyzing the preprocessed data using a machine learning model and statistical methods to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation using a generation AI model, and means for notifying the portable information processing device of the generated advice. This makes it possible to comprehensively monitor and evaluate the user's health status and provide individually optimized, specific health improvement advice.

[0957] A "wearable device" is an electronic device that can be worn to collect various data in daily life.

[0958] A "portable information processing device" is an electronic device that can be carried by a user and is used for performing information processing such as collecting, transmitting, and displaying data.

[0959] "Data cleansing" is the process of detecting missing or outliers in collected data and then completing or removing them.

[0960] "Data normalization" is the process of converting data in different units to a common scale to make it easier to analyze.

[0961] "Time series data sorting" is the process of sorting collected data into the correct order based on the time axis.

[0962] A "machine learning model" is an algorithm or mathematical model used to analyze collected data and make predictions or classifications.

[0963] "Statistical methods" are mathematical and statistical techniques used to analyze, evaluate, and interpret data.

[0964] "Assessment and evaluation" is the process of determining the user's health condition based on the analyzed data and making an evaluation of that condition.

[0965] A "generative AI model" is a model that uses artificial intelligence to generate new information and advice from data.

[0966] A "remote database" is a remote data storage system connected to a server for storing and managing collected data.

[0967] "Data preprocessing" is a series of processes performed to convert raw data into an analyzable form.

[0968] MODE FOR CARRYING OUT THE INVENTION

[0969] This invention is a system that collects, analyzes, and evaluates health-related data from a user's daily life and provides appropriate health improvement advice. This system supports the user's health management by collecting data using wearable devices and portable information processing devices and analyzing the data on a server.

[0970] Device settings and integration

[0971] First, the user sets up and links the wearable device and portable information processing device. Specific hardware examples include a smartwatch and a smart weighing scale. The user then installs a dedicated app on their smartphone and enters their personal information. The health-related data measured by the wearable device is then sent to the portable information processing device and stored in a cloud database.

[0972] Data collection and transmission

[0973] Wearable devices and portable information processing devices (e.g., smartwatches, smart scales, smartphones) collect daily activity data. Specifically, smart scales measure weight every morning and send the data to a smartphone, food log apps record meal contents and send the data to a smartphone, and smartwatches measure the number of steps taken, heart rate, and walking patterns and send the data to a smartphone. Smartphones also use GPS information to measure distance traveled and walking speed.

[0974] Data accumulation and preprocessing

[0975] The server accumulates data sent from mobile devices in a cloud database in real time, then performs preprocessing such as data cleansing (filling in missing values ​​and removing outliers), data normalization (scaling values), and time-series data alignment, ensuring the consistency and accuracy of the data and making it suitable for analysis.

[0976] Data analysis

[0977] The server then analyzes the preprocessed data using machine learning models and statistical methods. For example, it analyzes weight trends, dietary habits, and walking patterns. Specific models used include decision trees, random forests, and time series analysis models. The analysis results in an assessment of the user's health status.

[0978] Health assessment

[0979] The server evaluates the user's health status based on the analysis results and calculates an abnormality score and health risk level. For example, if sudden weight gain or nutritional imbalance is detected, a warning score will be generated. Abnormalities in daily walking count and walking patterns will also be detected.

[0980] Generate health improvement advice

[0981] The server uses the generative AI model to generate specific health improvement advice. This advice is presented in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[0982] Advice Notice

[0983] The server notifies the mobile information processing device of the generated advice and warnings. The user can check these advice and warnings through a dedicated app. If necessary, the user can also send feedback and ask follow-up questions.

[0984] User feedback and continuous improvement

[0985] Users improve their lifestyle habits based on the advice they receive. The server uses a generative AI model based on feedback from users to provide appropriate answers. This allows users to constantly monitor their health status and make continuous improvements.

[0986] Examples of prompt statements

[0987] Example prompts to input to the generative AI model:

[0988] "I'm a 30-year-old man and I've recently gained a lot of weight. Can you give me some specific advice on how to manage my weight?"

[0989] The above is a specific embodiment for carrying out the present invention. This system allows users to monitor their own health condition in detail and take appropriate and specific measures to improve their health.

[0990] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0991] Program processing steps

[0992] Step 1: Device Setup and Integration

[0993] Input: Wearable device, smartphone, dedicated app

[0994] Output: Initial setup completed, device connectivity established

[0995] Specific operation:

[0996] 1. The user has a wearable device (e.g., smart watch, smart scale) at hand.

[0997] 2. The user installs the dedicated app on their smartphone and launches it.

[0998] 3. After launching the app, the user performs initial setup by entering personal information such as name, date of birth, gender, height, and weight.

[0999] 4. The user enables the Bluetooth function on their smartphone and pairs it with the wearable device, establishing a connection between the devices.

[1000] Step 2: Data collection and transmission

[1001] Input: User's daily activities (weight measurement, eating, walking, etc.), sensor data from wearable devices

[1002] Output: Measurement data, transmitted data

[1003] Specific operation:

[1004] 1. A smart scale measures a user's weight every morning and sends the data to a smartphone.

[1005] 2. The food logging app records the user's meal contents and sends the data to their smartphone.

[1006] 3. The smartwatch measures the number of steps, heart rate, and walking patterns and sends the data to your smartphone.

[1007] 4. Smartphones will use GPS to measure distance traveled and walking speed and centralize the data.

[1008] Step 3: Data accumulation and preprocessing

[1009] Input: Measurement data from a smartphone

[1010] Output: Accumulated data, preprocessed data

[1011] Specific operation:

[1012] 1. The server accumulates data sent from the smartphone in a cloud database in real time.

[1013] 2. The server cleanses the accumulated data, imputes missing values, and removes outliers.

[1014] 3. The server performs data normalization, converting data in different units to a common scale.

[1015] 4. The server sorts the time series data and puts it in the correct order.

[1016] Step 4: Data analysis

[1017] Input: Preprocessed data

[1018] Output: Analysis results, evaluation indicators

[1019] Specific operation:

[1020] 1. The server analyzes the preprocessed data using machine learning models and statistical methods.

[1021] 2. Examples of models used: decision trees, random forests, time series analysis models.

[1022] 3. The server analyzes weight trends and detects long-term patterns of gain or loss.

[1023] 4. The server analyzes the dietary data, evaluates nutritional balance, and extracts nutrients that are deficient or in excess.

[1024] 5. The server analyzes walking patterns and evaluates daily variations in step count, speed, and stride length.

[1025] Step 5: Health Assessment

[1026] Input: Analysis results, evaluation indicators

[1027] Output: Health score, risk level

[1028] Specific operation:

[1029] 1. The server comprehensively evaluates the user's health condition based on the analysis results.

[1030] 2. The server calculates a health score based on weight change, nutritional balance, and walking patterns.

[1031] 3. The server generates a high risk score if there is rapid weight gain or nutritional imbalance.

[1032] 4. The server classifies the health risk level based on the assessment results (e.g., low risk, medium risk, high risk).

[1033] Step 6: Generate health improvement advice

[1034] Input: Assessment results, risk level

[1035] Output: Specific health improvement advice

[1036] Specific operation:

[1037] 1. The server uses the generative AI model to generate specific health improvement advice based on the evaluation results.

[1038] 2. Input a prompt sentence into the generative AI model to generate specific advice. For example, "Your recent weight gain needs attention. In addition to limiting your calories, increase your intake of vegetables and fruits."

[1039] Step 7: Advice Notification

[1040] Input: Generated health improvement advice

[1041] Output: Notify user, gather feedback

[1042] Specific operation:

[1043] 1. The server transmits the generated advice or warning to the portable information processing device.

[1044] 2. The user checks the advice through a dedicated app and sends feedback or additional questions as needed.

[1045] Step 8: User feedback and continuous improvement

[1046] Input: User feedback, ongoing data collection

[1047] Output: Advice based on feedback, continuously improving the user's health

[1048] Specific operation:

[1049] 1. Improve your lifestyle based on the advice you receive.

[1050] 2. The server receives feedback from the user, analyzes it, and provides an appropriate answer using a generative AI model.

[1051] 3. The user asks additional questions based on the feedback, and the server answers them, helping to keep track of the user's health status and making continuous improvements.

[1052] (Application example 1)

[1053] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1054] In modern society, many people's busy daily lives make it difficult to effectively manage their health. Providing real-time health management and advice during visits to physical stores is particularly challenging. While there are technologies available to collect health data using wearable devices and mobile terminals, there is a lack of methods to instantly provide this data to users in physical stores. As a result, early detection of health risks and the provision of appropriate health improvement advice may be delayed. Therefore, there is a need for a system that can instantly assess a user's health status in a physical store and provide appropriate advice.

[1055] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1056] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data and determining and evaluating the user's health condition, means for generating health improvement advice based on the determination and evaluation, and means for notifying the generated advice to the mobile terminal and a visual information device that can be viewed in the physical store. This makes it possible to instantly evaluate the user's health condition in the physical store and provide appropriate health improvement advice in real time.

[1057] A "wearable device" is an electronic device worn by a user that collects various data about daily life. Examples include smartwatches and smart weighing scales.

[1058] A "mobile device" is an electronic device that is carried by a user and provides various functions. Examples of such devices include smartphones and tablets.

[1059] A "cloud database" is a database that is accessible over the internet and provides data protection and scalability.

[1060] A "visual information device" is an electronic device that visually displays information to a user. Examples include smart glasses and digital signage.

[1061] A "generative artificial intelligence model" is an artificial intelligence system that uses machine learning algorithms to generate information in a format that is easy for users to understand.

[1062] "Preprocessing" is the process of preparing collected data in an analyzable format, including, for example, filling in missing values ​​and removing outliers.

[1063] "Health improvement advice" refers to specific instructions or recommendations for improving the user's health, such as advice on exercise or dietary changes.

[1064] "Health status" refers to the physical condition and physiological status of the user in their daily life, including, for example, weight, heart rate, and number of steps.

[1065] This invention is a system that uses wearable devices and mobile terminals to monitor a user's health status in real time in a physical store and provide health improvement advice. A specific implementation method of this system is described below.

[1066] Hardware and Software Configuration

[1067] It consists of a server, wearable devices, mobile terminals, a cloud database, and visual information devices.

[1068] Specific examples of hardware and software used

[1069] Wearable devices: smart watches, smart scales

[1070] Mobile device: Smartphone

[1071] Cloud Database: Internet-accessible database system

[1072] Visual information devices: smart glasses, digital signage

[1073] Software: Python 3.x, Requests library, REST API

[1074] Data collection and transmission

[1075] Users wear wearable devices (e.g., smart watches, smart scales) and install dedicated applications on their mobile devices (e.g., smartphones). These devices automatically collect data on the user's daily activities (weight, heart rate, number of steps, nutrition, etc.) and send it to the mobile device. The mobile device then sends the collected data to a cloud database for storage.

[1076] Data accumulation and preprocessing

[1077] The server stores data sent from mobile devices in a cloud database in real time, where it undergoes preprocessing (data cleansing, normalization, alignment of time-series data, etc.) and prepares the data in a format suitable for analysis.

[1078] Data analysis and evaluation

[1079] The server analyzes the preprocessed data using machine learning models and statistical methods to assess the user's health status, specifically weight changes, diet, walking patterns, etc. This allows it to detect sudden weight gain, nutritional imbalances, and abnormal walking patterns.

[1080] Generate health improvement advice

[1081] Based on the analysis results, the server uses a generative artificial intelligence model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[1082] Advice Notice

[1083] The server then sends the generated health improvement advice and warnings to mobile devices and visual information devices, where users can check the advice in real time through a dedicated application on their mobile devices and visual information devices in physical stores.

[1084] Examples and prompts

[1085] For example, consider the case where a visitor with user ID 12345 visits a physical store and the following data is collected:

[1086] Weight: 85 kg

[1087] Heart rate: 105 bpm

[1088] Number of steps: 4000

[1089] Nutrients: "fast_food"

[1090] Based on this data, specific examples of prompt sentences for the generative artificial intelligence model are as follows:

[1091] "Evaluate the health risks of user ID: 12345 and generate advice. Weight: 85kg, Heart Rate: 105bpm, Steps: 4000, Nutrition: 'fast_food'"

[1092] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1093] Step 1:

[1094] The user sets up the wearable device and mobile device. The user installs a dedicated application on the mobile device and enters personal information to link the devices.

[1095] Input: User's personal information, initial settings of wearable device

[1096] Output: Devices are linked and working properly

[1097] Step 2:

[1098] Wearable devices and mobile devices collect various data about our daily lives: smartwatches measure heart rate and steps, smart scales weigh us, and mobile devices collect GPS data.

[1099] Input: User's daily activity data (heart rate, number of steps, weight, distance traveled, etc.)

[1100] Output: Raw data collected

[1101] Step 3:

[1102] The mobile device sends the collected data to a cloud database, which is done automatically by a dedicated application.

[1103] Input: Raw data collected

[1104] Output: Data stored in a cloud database

[1105] Step 4:

[1106] The server preprocesses the data stored in the cloud database, for example, by performing data cleansing (filling in missing values, removing outliers) and data normalization (scaling values).

[1107] Input: Data stored in the cloud database

[1108] Output: Preprocessed data

[1109] Step 5:

[1110] The server analyzes the pre-processed data and assesses the user's health status, such as analyzing long-term weight trends, dietary habits, and walking patterns.

[1111] Input: Preprocessed data

[1112] Output: Analysis results (weight change trends, nutritional balance evaluation, abnormal walking pattern detection, etc.)

[1113] Step 6:

[1114] The server uses the generative AI model to generate health improvement advice, such as "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[1115] Input: Analysis results

[1116] Output: Generated health improvement advice

[1117] Step 7:

[1118] The server notifies the mobile terminal and visual information device of the generated advice, which the user can then check on the visual information device (e.g., smart glasses, digital signage) in the physical store and on the mobile terminal.

[1119] Input: Generated health improvement advice

[1120] Output: Advice displayed on mobile devices and visual information devices

[1121] Step 8:

[1122] The user can check the generated advice and work to improve their lifestyle. Feedback can be sent to the server if necessary.

[1123] Input: Health improvement advice, user feedback

[1124] Output: Improved lifestyle data (if any feedback is provided, it will be sent to the server)

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and combines it with an emotion engine to support health improvement. This system constantly monitors the user's health and emotional state, and provides appropriate advice to reduce health risks.

[1127] Device settings and integration

[1128] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[1129] Data collection and transmission

[1130] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1131] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[1132] The food logging app records the user's meal contents and sends the information to a mobile device.

[1133] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[1134] Your smartphone uses GPS data to record your distance traveled and walking speed.

[1135] The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[1136] Data accumulation and preprocessing

[1137] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[1138] Data Preprocessing and Sentiment Analysis

[1139] The server preprocesses the accumulated data, specifically by performing data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[1140] For emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment.

[1141] Data analysis

[1142] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[1143] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[1144] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[1145] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[1146] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[1147] Health assessment

[1148] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[1149] Generate health improvement advice

[1150] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1151] Advice Notice

[1152] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[1153] User feedback and continuous improvement

[1154] Users improve their lifestyle habits based on the advice they receive. Feedback and questions are also entered within the app, and the server uses a generative AI model to provide appropriate answers and additional advice. This enables continuous health management through interaction with the user.

[1155] As described above, this system combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage the user's health and emotional state in real time and provide specific and appropriate advice, allowing users to detect health risks early and take preventive measures.

[1156] The processing flow will be explained below.

[1157] Step 1:

[1158] The user initially configures the smart device (e.g., smart watch, smart scale) and mobile device. The user installs a dedicated app on the smartphone and enters personal information (age, gender, height, weight, etc.). The user also configures the app to enable voice recognition and face recognition functions.

[1159] Step 2:

[1160] Terminals (smart devices and smartphones) collect data from daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed. Furthermore, the smartphone uses its built-in microphone and camera to collect voice data and facial expression data from the user.

[1161] Step 3:

[1162] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[1163] Step 4:

[1164] The server preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. The same preprocessing is performed on emotion data.

[1165] Step 5:

[1166] The emotion engine analyzes voice data and facial expression data to determine the user's emotional state. Specifically, it analyzes the tone and speed of the voice from the voice data and performs micro-expression analysis on the facial expression data to detect emotions such as joy, anger, and sadness. This emotional state data is integrated with other health data and used for a comprehensive assessment.

[1167] Step 6:

[1168] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[1169] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[1170] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[1171] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[1172] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[1173] Step 7:

[1174] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[1175] Step 8:

[1176] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1177] Step 9:

[1178] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[1179] Step 10:

[1180] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[1181] Step 11:

[1182] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[1183] Example 2

[1184] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1185] Conventional health management systems simply collect and analyze physical data (weight, number of steps, heart rate, etc.) and do not provide comprehensive health assessments that take into account the user's emotional state. As a result, the user's stress and mental health status may be overlooked, making it difficult to provide comprehensive health support. In addition, the health improvement advice provided is often general and lacking in specificity, making it difficult for users to understand. This presents a challenge, making it difficult to achieve effective health improvement.

[1186] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing voice data and facial expression data to determine the user's emotional state, means for analyzing the preprocessed data and emotional data to determine and evaluate the user's health state, means for generating health improvement advice based on the determination and evaluation, and means for notifying the mobile terminal of the generated advice. This makes it possible to comprehensively analyze the user's physical data and emotional data and provide comprehensive and specific health improvement advice.

[1187] A "wearable device" is an electronic device that is attached to the user's body or clothing and can collect various data from daily life.

[1188] A "mobile terminal" is a portable information processing device, such as a smartphone or tablet, that can receive and process data from a wearable device.

[1189] A "cloud database" is a database that can be accessed via the Internet and is a system that can store and manage various collected data.

[1190] "Preprocessing" refers to a series of processes carried out to prepare collected raw data in an analyzable format, and specifically includes filling in missing values, removing outliers, and normalizing the data.

[1191] An "emotion engine" is a software module or algorithm that analyzes voice data and facial expression data to determine the user's emotional state.

[1192] "Health status assessment" refers to the process of comprehensively assessing the user's physical and mental health status by analyzing the pre-processed data and emotional data.

[1193] "Health improvement advice" refers to advice that includes specific suggestions and instructions for improving the user's lifestyle and behavior based on the results of the health status assessment and evaluation.

[1194] A "generative artificial intelligence model" is an artificial intelligence technology that generates new information and sentences based on input data and prompts, and is used to provide health improvement advice to users in a format that is easy to understand.

[1195] The present invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from a user's daily life, and combines it with an emotion engine to support health improvement. In this embodiment, a specific method for reducing health risks by constantly monitoring the user's health and emotional state and providing appropriate advice will be described.

[1196] The system can be implemented using the following hardware and software combinations:

[1197] Wearable devices (e.g., smart watches, smart scales)

[1198] Mobile devices (e.g. smartphones)

[1199] Cloud Database

[1200] Emotion Engine

[1201] Machine learning models

[1202] Generative Artificial Intelligence Model

[1203] Device settings and integration

[1204] First, the user connects a wearable device such as a smartwatch or smart scale to their smartphone. After installing a dedicated app on their smartphone, they launch the app and enter their personal information, such as their name, age, gender, height, and weight. The data measured by each wearable device is then automatically sent to the mobile terminal.

[1205] Data collection and transmission

[1206] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1207] The smart scale allows users to weigh themselves every morning and sends the data to a mobile device via Bluetooth.

[1208] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[1209] The smartwatch measures the user's steps, heart rate, walking patterns, etc. and transmits the data to a mobile device in real time.

[1210] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[1211] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[1212] Data accumulation and preprocessing

[1213] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each piece of data is stored in a corresponding table; for example, weight data is stored in the "weight" table, and walking data is stored in the "walking" table.

[1214] Data Preprocessing and Sentiment Analysis

[1215] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. For emotion data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This result is integrated with other health data.

[1216] Data analysis

[1217] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[1218] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[1219] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[1220] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[1221] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[1222] Health status assessment and generation and notification of health improvement advice

[1223] The server makes a comprehensive assessment of the user's health condition based on the analysis results and uses a generative AI model to generate health improvement advice. An example of a prompt is, "The user's weight has increased dramatically recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation." The generative AI model then generates specific advice, such as, "Recent weight gain and increased stress have been confirmed. Try to get some moderate exercise and take time to relax." This advice is sent to the mobile device, and the user can check it through a dedicated app.

[1224] User feedback and continuous improvement

[1225] Users can improve their lifestyle habits based on the advice they receive. Feedback and follow-up questions can be provided through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. Continuous health management is possible through interaction with the user.

[1226] As described above, the present invention combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage a user's health and emotional state in real time and provide specific and appropriate advice, thereby enabling users to detect health risks early and take preventive measures.

[1227] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1228] Step 1: Device Setup and Integration

[1229] The user sets up a wearable device such as a smart watch or smart scale and links it to a mobile device. The user installs and launches a dedicated app and enters personal information (name, age, gender, height, weight, etc.). The entered information is saved on the mobile device, and the wearable device is set up to automatically send measured data to the mobile device. The input here is the user's personal information and device measurement data, and the output is the completed setup state on the mobile device.

[1230] Step 2: Data collection and transmission

[1231] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1232] The smart scale allows users to weigh themselves every morning and transmits the data to a mobile device via Bluetooth.

[1233] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[1234] The smartwatch measures the user's steps, heart rate, and walking patterns and transmits the data in real time to a mobile device.

[1235] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[1236] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[1237] The input here is the various data measured by the device, and the output is all the data sent to the mobile terminal.

[1238] Step 3: Data accumulation and preprocessing

[1239] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each data item is stored in a corresponding table. The input is the raw data sent from the mobile device, and the output is the organized data accumulated in the cloud database.

[1240] Step 4: Data preprocessing and sentiment analysis

[1241] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values). The input here is the accumulated raw data, and the output is the preprocessed, clean data.

[1242] Regarding emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). The input is voice data and facial expression data, and the output is the determined emotional state.

[1243] Step 5: Data analysis

[1244] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[1245] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[1246] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[1247] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[1248] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[1249] The input here is the preprocessed data and emotion determination results, and the output is each analysis result.

[1250] Step 6: Assessing health status and generating health improvement advice

[1251] The server comprehensively evaluates the user's health condition based on the analysis results and generates health improvement advice using a generative AI model. For example, based on a prompt such as "The user's weight has increased rapidly recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation," the generative AI model generates specific advice (e.g., "Recent weight gain and increased stress have been confirmed. Try to get moderate exercise and spend time relaxing.").

[1252] The input here is the analysis results and a prompt for the generative AI model, and the output is the generated health improvement advice.

[1253] Step 7: Advice Notification

[1254] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app. The input here is the generated advice, and the output is a notification to the user.

[1255] Step 8: User feedback and continuous improvement

[1256] The user improves their lifestyle habits based on the advice they receive. The user can provide feedback or follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. The input here is feedback and questions from the user, and the output is answers and follow-up advice from the generative AI model. This enables continuous health management through interaction with the user.

[1257] The above is the flow of processing of the program of this system.

[1258] (Application example 2)

[1259] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] Current systems in health management and fitness gyms are not sufficient to monitor users' health and emotional states in real time and provide customized exercise menus and health advice suited to each individual user. Therefore, there is a need for a method to detect users' health risks early and efficiently provide appropriate improvement measures.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1262] In this invention, the server includes means for collecting various data about the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation, means for notifying the mobile terminal of the generated advice, means for analyzing voice data and facial expression data to determine the user's emotional state, means for generating customized exercise menus and health advice based on the user's health data and emotional data, and means for collecting and analyzing member data in cooperation with smart fitness machines and sensors in the store and providing customized advice in real time, thereby enabling detailed analysis of the user's health status and emotional state and enabling truly personalized health management and fitness support.

[1263] A "wearable device" is a device that is worn on the user's body and is used to collect data from daily life.

[1264] A "mobile terminal" is an electronic device that can be carried by a user and is used to collect, process, and transmit data.

[1265] A "cloud database" is a remote data storage system for storing, managing, and accessing data over the Internet.

[1266] "Preprocessing" is the process of shaping and cleansing collected data to make it easier to analyze.

[1267] "Health status" refers to the overall state of a user's physical and mental health.

[1268] "Assessment and evaluation" is the process of analyzing the user's current situation based on collected and preprocessed data and identifying health risks, etc.

[1269] "Health improvement advice" refers to specific suggestions or instructions provided to improve the user's health.

[1270] "Voice data" is acoustic information generated by the user's voice.

[1271] "Facial expression data" is image data for capturing and analyzing the facial expressions of a user.

[1272] An "emotional state" is the emotional state a user is experiencing at a particular point in time.

[1273] A "customized exercise menu" is an exercise plan designed to suit an individual user's health condition and goals.

[1274] "Smart fitness machines" are devices used in fitness gyms that have advanced functions for collecting and analyzing exercise data.

[1275] A "sensor" is a device that senses a specific physical phenomenon and converts it into data.

[1276] "Real-time" refers to a state in which data is collected and processed immediately, without delay.

[1277] This invention provides a health support system customized for fitness gyms. The system uses wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and by combining this with an emotion engine, it monitors the user's health and emotional state in real time and provides appropriate advice.

[1278] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[1279] Mobile devices automatically collect the following data through daily activities: A smart scale measures the user's weight and sends the data to the mobile device. A food logging app records the user's diet and sends the information to the mobile device. A smartwatch measures the number of steps taken, heart rate, and walking patterns and sends the data to the mobile device. A smartphone uses GPS data to record distance traveled and walking speed. The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[1280] The server accumulates various data (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) sent from mobile devices in a cloud database in real time. The server then preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[1281] The emotion engine analyzes the voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment. The server then analyzes the pre-processed data and the emotion engine's determination result using machine learning models and statistical methods. The analysis includes trend analysis of weight change, evaluation of dietary habits, analysis of walking patterns, and evaluation of emotional state.

[1282] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high-risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state. The server uses a generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1283] The generated advice is provided through a prompt sentence as follows:

[1284] example:

[1285] "Recent data has confirmed that your weight is increasing, but your walking frequency has also increased, so it appears that you are exercising appropriately. However, your diet tends to be high in calories, so try to eat a more balanced diet. Your emotional data also suggests that your stress levels are a little high, so try incorporating relaxation techniques."

[1286] This system makes it easier for users to improve their daily lifestyle habits, providing comprehensive health management and fitness support.

[1287] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1288] Step 1:

[1289] The user sets up and links the wearable device and mobile device. The user installs a dedicated app on their smartphone and enters their personal information. This sets up the wearable device to send measured data to the mobile device.

[1290] Input: User's personal information, wearable device settings information

[1291] Output: Data integration setup is complete and data is ready to be sent

[1292] Step 2:

[1293] Mobile devices automatically collect health-related data through daily activities. A smart scale weighs you every morning and sends the data to your smartphone. A food logging app records what you eat. A smartwatch measures your steps, heart rate, and walking patterns. Your smartphone uses GPS data to record distance traveled and walking speed. Your mobile device uses a camera and microphone to collect audio and facial expression data.

[1294] Input: User's daily life activity data

[1295] Output: Various health-related data (weight, number of steps, dietary details, heart rate, distance traveled, voice data, facial expression data)

[1296] Step 3:

[1297] The mobile device sends the collected data to a cloud database, which stores health-related data in real time.

[1298] Input: Health-related data collected by mobile devices

[1299] Output: Data stored in a cloud database

[1300] Step 4:

[1301] The server preprocesses the data stored in the cloud database, performing data cleansing, imputing missing values, removing outliers, scaling values, and aligning time series data.

[1302] Input: Health-related data stored in a cloud database

[1303] Output: Preprocessed data (aligned time series data, cleansed data)

[1304] Step 5:

[1305] The server analyzes the voice data and facial expression data and uses an emotion engine to determine the user's emotional state, for example, analyzing emotions from the voice data and analyzing the facial expression data to determine the emotional state.

[1306] Input: Preprocessed speech and facial expression data

[1307] Output: Emotional state (e.g., happy, angry, sad, etc.)

[1308] Step 6:

[1309] The server analyzes the preprocessed health data and the emotional state calculated by the emotion engine using machine learning models and statistical methods to analyze weight trends, evaluate dietary habits, analyze walking patterns, and evaluate emotional states.

[1310] Input: Preprocessed health data, emotional state

[1311] Output: Health status analysis results (weight change, dietary assessment, walking patterns, emotional state assessment)

[1312] Step 7:

[1313] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score is generated. If the emotion engine detects a high stress state, it generates a warning.

[1314] Input: Health status analysis results

[1315] Output: Health score, health risk level

[1316] Step 8:

[1317] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in the form of prompt sentences.

[1318] Input: Health score, health risk level

[1319] Output: Health improvement advice (based on generative AI model)

[1320] Step 9:

[1321] The server sends the generated advice to the mobile device, where the user can check the advice through a dedicated app on the mobile device and obtain specific guidelines for improving their daily lifestyle habits.

[1322] Input: Health Improvement Advice

[1323] Output: Notification to mobile device, providing advice to user

[1324] This makes it easier for users to improve their daily lifestyle habits, enabling comprehensive health management and fitness support.

[1325] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1326] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1327] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1328] [Fourth embodiment]

[1329] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1330] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1331] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1332] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1333] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1334] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1335] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1336] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1337] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1338] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1339] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1340] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1341] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1342] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life to support health improvement. This system constantly monitors the user's health status and provides appropriate advice to reduce health risks.

[1343] Device settings and integration

[1344] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. Data measured by the wearable device is then sent to the mobile device and stored in a cloud database.

[1345] Data collection and transmission

[1346] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1347] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[1348] The food logging app records the user's meal contents and sends the information to a mobile device.

[1349] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[1350] The smartphone measures distance traveled and walking speed using GPS and integrates the data.

[1351] Data accumulation and preprocessing

[1352] The server stores data sent from mobile devices in a cloud database in real time. Since the data stored in the database is difficult to analyze as is, it undergoes preprocessing. Specifically, this involves data cleansing (complementing missing values ​​and removing outliers), data normalization (scaling values), and sorting time-series data.

[1353] Data analysis

[1354] The server then analyzes the pre-processed data using machine learning models and statistical methods. This analysis evaluates the following:

[1355] Weight Trends: Analyze long-term weight patterns to detect abnormal weight changes.

[1356] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[1357] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[1358] Health assessment

[1359] The server evaluates the user's health condition based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if there is poor nutritional balance or abnormalities in walking patterns.

[1360] Generate health improvement advice

[1361] The server uses the generative AI model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[1362] Advice Notice

[1363] The server notifies the mobile device of the generated advice and warnings, and the user can check these advices through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[1364] User feedback and continuous improvement

[1365] Users improve their lifestyle habits based on the advice they receive, and the server uses a generative AI model to provide appropriate answers to feedback and questions. This allows users to constantly monitor their health status and make continuous improvements.

[1366] As described above, this system utilizes wearable devices and mobile terminals to monitor and manage users' health status in real time and provide specific and appropriate advice, enabling users to detect health risks early and take preventive measures.

[1367] The processing flow will be explained below.

[1368] Step 1:

[1369] The user sets up the smart device (e.g., smart watch, smart scale) and mobile terminal initially. The user installs a dedicated app on their smartphone and enters their personal information (age, gender, height, weight, etc.).

[1370] Step 2:

[1371] Terminals (smart devices and smartphones) collect data on daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed.

[1372] Step 3:

[1373] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, etc.) in a cloud database in real time. Specifically, it receives data through an API and stores each data item in the corresponding table.

[1374] Step 4:

[1375] The server preprocesses the accumulated data. Specifically, it performs data cleansing, fills in missing values, and removes outliers. It also performs data normalization and scales the data to a certain range. For time series data, it aligns the data based on timestamps.

[1376] Step 5:

[1377] The server then analyzes the preprocessed data using machine learning models and statistical methods, including trend analysis of weight change, nutritional evaluation of dietary data, and detection of abnormalities in walking patterns.

[1378] Step 6:

[1379] The server evaluates the user's health condition based on the analysis results, generating an abnormality score and calculating the health risk level in the event of weight gain, nutritional imbalance, or abnormal walking patterns.

[1380] Step 7:

[1381] The server uses the generative AI model to generate health improvement advice based on the analysis results. Specific advice might include, "Your calorie intake continues to be high. Include more vegetables and fruits in your diet," or "Your walking pattern is abnormal. We recommend that you see a doctor."

[1382] Step 8:

[1383] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[1384] Step 9:

[1385] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[1386] Step 10:

[1387] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[1388] Example 1

[1389] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1390] In modern society, many people lead unhealthy lifestyles, resulting in an increase in lifestyle-related diseases, which has become a serious social problem. The present invention aims to solve these problems and improve the health status of users by providing a system that efficiently collects, analyzes, and evaluates health-related data from daily life and provides appropriate health improvement advice. In particular, highly accurate health management is possible by consistently performing the process from preprocessing, analysis, and evaluation of collected data to generating advice.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1392] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a portable information processing device, means for storing the collected data in a remote database, preprocessing means for performing data cleansing, data normalization, and time-series data alignment on the stored data, means for analyzing the preprocessed data using a machine learning model and statistical methods to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation using a generation AI model, and means for notifying the portable information processing device of the generated advice. This makes it possible to comprehensively monitor and evaluate the user's health status and provide individually optimized, specific health improvement advice.

[1393] A "wearable device" is an electronic device that can be worn to collect various data in daily life.

[1394] A "portable information processing device" is an electronic device that can be carried by a user and is used for performing information processing such as collecting, transmitting, and displaying data.

[1395] "Data cleansing" is the process of detecting missing or outliers in collected data and then completing or removing them.

[1396] "Data normalization" is the process of converting data in different units to a common scale to make it easier to analyze.

[1397] "Time series data sorting" is the process of sorting collected data into the correct order based on the time axis.

[1398] A "machine learning model" is an algorithm or mathematical model used to analyze collected data and make predictions or classifications.

[1399] "Statistical methods" are mathematical and statistical techniques used to analyze, evaluate, and interpret data.

[1400] "Assessment and evaluation" is the process of determining the user's health condition based on the analyzed data and making an evaluation of that condition.

[1401] A "generative AI model" is a model that uses artificial intelligence to generate new information and advice from data.

[1402] A "remote database" is a remote data storage system connected to a server for storing and managing collected data.

[1403] "Data preprocessing" is a series of processes performed to convert raw data into an analyzable form.

[1404] MODE FOR CARRYING OUT THE INVENTION

[1405] This invention is a system that collects, analyzes, and evaluates health-related data from a user's daily life and provides appropriate health improvement advice. This system supports the user's health management by collecting data using wearable devices and portable information processing devices and analyzing the data on a server.

[1406] Device settings and integration

[1407] First, the user sets up and links the wearable device and portable information processing device. Specific hardware examples include a smartwatch and a smart weighing scale. The user then installs a dedicated app on their smartphone and enters their personal information. The health-related data measured by the wearable device is then sent to the portable information processing device and stored in a cloud database.

[1408] Data collection and transmission

[1409] Wearable devices and portable information processing devices (e.g., smartwatches, smart scales, smartphones) collect daily activity data. Specifically, smart scales measure weight every morning and send the data to a smartphone, food log apps record meal contents and send the data to a smartphone, and smartwatches measure the number of steps taken, heart rate, and walking patterns and send the data to a smartphone. Smartphones also use GPS information to measure distance traveled and walking speed.

[1410] Data accumulation and preprocessing

[1411] The server accumulates data sent from mobile devices in a cloud database in real time, then performs preprocessing such as data cleansing (filling in missing values ​​and removing outliers), data normalization (scaling values), and time-series data alignment, ensuring the consistency and accuracy of the data and making it suitable for analysis.

[1412] Data analysis

[1413] The server then analyzes the preprocessed data using machine learning models and statistical methods. For example, it analyzes weight trends, dietary habits, and walking patterns. Specific models used include decision trees, random forests, and time series analysis models. The analysis results in an assessment of the user's health status.

[1414] Health assessment

[1415] The server evaluates the user's health status based on the analysis results and calculates an abnormality score and health risk level. For example, if sudden weight gain or nutritional imbalance is detected, a warning score will be generated. Abnormalities in daily walking count and walking patterns will also be detected.

[1416] Generate health improvement advice

[1417] The server uses the generative AI model to generate specific health improvement advice. This advice is presented in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[1418] Advice Notice

[1419] The server notifies the mobile information processing device of the generated advice and warnings. The user can check these advice and warnings through a dedicated app. If necessary, the user can also send feedback and ask follow-up questions.

[1420] User feedback and continuous improvement

[1421] Users improve their lifestyle habits based on the advice they receive. The server uses a generative AI model based on feedback from users to provide appropriate answers. This allows users to constantly monitor their health status and make continuous improvements.

[1422] Examples of prompt statements

[1423] Example prompts to input to the generative AI model:

[1424] "I'm a 30-year-old man and I've recently gained a lot of weight. Can you give me some specific advice on how to manage my weight?"

[1425] The above is a specific embodiment for carrying out the present invention. This system allows users to monitor their own health condition in detail and take appropriate and specific measures to improve their health.

[1426] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1427] Program processing steps

[1428] Step 1: Device Setup and Integration

[1429] Input: Wearable device, smartphone, dedicated app

[1430] Output: Initial setup completed, device connectivity established

[1431] Specific operation:

[1432] 1. The user has a wearable device (e.g., smart watch, smart scale) at hand.

[1433] 2. The user installs the dedicated app on their smartphone and launches it.

[1434] 3. After launching the app, the user performs initial setup by entering personal information such as name, date of birth, gender, height, and weight.

[1435] 4. The user enables the Bluetooth function on their smartphone and pairs it with the wearable device, establishing a connection between the devices.

[1436] Step 2: Data collection and transmission

[1437] Input: User's daily activities (weight measurement, eating, walking, etc.), sensor data from wearable devices

[1438] Output: Measurement data, transmitted data

[1439] Specific operation:

[1440] 1. A smart scale measures a user's weight every morning and sends the data to a smartphone.

[1441] 2. The food logging app records the user's meal contents and sends the data to their smartphone.

[1442] 3. The smartwatch measures the number of steps, heart rate, and walking patterns and sends the data to your smartphone.

[1443] 4. Smartphones will use GPS to measure distance traveled and walking speed and centralize the data.

[1444] Step 3: Data accumulation and preprocessing

[1445] Input: Measurement data from a smartphone

[1446] Output: Accumulated data, preprocessed data

[1447] Specific operation:

[1448] 1. The server accumulates data sent from the smartphone in a cloud database in real time.

[1449] 2. The server cleanses the accumulated data, imputes missing values, and removes outliers.

[1450] 3. The server performs data normalization, converting data in different units to a common scale.

[1451] 4. The server sorts the time series data and puts it in the correct order.

[1452] Step 4: Data analysis

[1453] Input: Preprocessed data

[1454] Output: Analysis results, evaluation indicators

[1455] Specific operation:

[1456] 1. The server analyzes the preprocessed data using machine learning models and statistical methods.

[1457] 2. Examples of models used: decision trees, random forests, time series analysis models.

[1458] 3. The server analyzes weight trends and detects long-term patterns of gain or loss.

[1459] 4. The server analyzes the dietary data, evaluates nutritional balance, and extracts nutrients that are deficient or in excess.

[1460] 5. The server analyzes walking patterns and evaluates daily variations in step count, speed, and stride length.

[1461] Step 5: Health Assessment

[1462] Input: Analysis results, evaluation indicators

[1463] Output: Health score, risk level

[1464] Specific operation:

[1465] 1. The server comprehensively evaluates the user's health condition based on the analysis results.

[1466] 2. The server calculates a health score based on weight change, nutritional balance, and walking patterns.

[1467] 3. The server generates a high risk score if there is rapid weight gain or nutritional imbalance.

[1468] 4. The server classifies the health risk level based on the assessment results (e.g., low risk, medium risk, high risk).

[1469] Step 6: Generate health improvement advice

[1470] Input: Assessment results, risk level

[1471] Output: Specific health improvement advice

[1472] Specific operation:

[1473] 1. The server uses the generative AI model to generate specific health improvement advice based on the evaluation results.

[1474] 2. Input a prompt sentence into the generative AI model to generate specific advice. For example, "Your recent weight gain needs attention. In addition to limiting your calories, increase your intake of vegetables and fruits."

[1475] Step 7: Advice Notification

[1476] Input: Generated health improvement advice

[1477] Output: Notify user, gather feedback

[1478] Specific operation:

[1479] 1. The server transmits the generated advice or warning to the portable information processing device.

[1480] 2. The user checks the advice through a dedicated app and sends feedback or additional questions as needed.

[1481] Step 8: User feedback and continuous improvement

[1482] Input: User feedback, ongoing data collection

[1483] Output: Advice based on feedback, continuously improving the user's health

[1484] Specific operation:

[1485] 1. Improve your lifestyle based on the advice you receive.

[1486] 2. The server receives feedback from the user, analyzes it, and provides an appropriate answer using a generative AI model.

[1487] 3. The user asks additional questions based on the feedback, and the server answers them, helping to keep track of the user's health status and making continuous improvements.

[1488] (Application example 1)

[1489] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] In modern society, many people's busy daily lives make it difficult to effectively manage their health. Providing real-time health management and advice during visits to physical stores is particularly challenging. While there are technologies available to collect health data using wearable devices and mobile terminals, there is a lack of methods to instantly provide this data to users in physical stores. As a result, early detection of health risks and the provision of appropriate health improvement advice may be delayed. Therefore, there is a need for a system that can instantly assess a user's health status in a physical store and provide appropriate advice.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1492] In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data and determining and evaluating the user's health condition, means for generating health improvement advice based on the determination and evaluation, and means for notifying the generated advice to the mobile terminal and a visual information device that can be viewed in the physical store. This makes it possible to instantly evaluate the user's health condition in the physical store and provide appropriate health improvement advice in real time.

[1493] A "wearable device" is an electronic device worn by a user that collects various data about daily life. Examples include smartwatches and smart weighing scales.

[1494] A "mobile device" is an electronic device that is carried by a user and provides various functions. Examples of such devices include smartphones and tablets.

[1495] A "cloud database" is a database that is accessible over the internet and provides data protection and scalability.

[1496] A "visual information device" is an electronic device that visually displays information to a user. Examples include smart glasses and digital signage.

[1497] A "generative artificial intelligence model" is an artificial intelligence system that uses machine learning algorithms to generate information in a format that is easy for users to understand.

[1498] "Preprocessing" is the process of preparing collected data in an analyzable format, including, for example, filling in missing values ​​and removing outliers.

[1499] "Health improvement advice" refers to specific instructions or recommendations for improving the user's health, such as advice on exercise or dietary changes.

[1500] "Health status" refers to the physical condition and physiological status of the user in their daily life, including, for example, weight, heart rate, and number of steps.

[1501] This invention is a system that uses wearable devices and mobile terminals to monitor a user's health status in real time in a physical store and provide health improvement advice. A specific implementation method of this system is described below.

[1502] Hardware and Software Configuration

[1503] It consists of a server, wearable devices, mobile terminals, a cloud database, and visual information devices.

[1504] Specific examples of hardware and software used

[1505] Wearable devices: smart watches, smart scales

[1506] Mobile device: Smartphone

[1507] Cloud Database: Internet-accessible database system

[1508] Visual information devices: smart glasses, digital signage

[1509] Software: Python 3.x, Requests library, REST API

[1510] Data collection and transmission

[1511] Users wear wearable devices (e.g., smart watches, smart scales) and install dedicated applications on their mobile devices (e.g., smartphones). These devices automatically collect data on the user's daily activities (weight, heart rate, number of steps, nutrition, etc.) and send it to the mobile device. The mobile device then sends the collected data to a cloud database for storage.

[1512] Data accumulation and preprocessing

[1513] The server stores data sent from mobile devices in a cloud database in real time, where it undergoes preprocessing (data cleansing, normalization, alignment of time-series data, etc.) and prepares the data in a format suitable for analysis.

[1514] Data analysis and evaluation

[1515] The server analyzes the preprocessed data using machine learning models and statistical methods to assess the user's health status, specifically weight changes, diet, walking patterns, etc. This allows it to detect sudden weight gain, nutritional imbalances, and abnormal walking patterns.

[1516] Generate health improvement advice

[1517] Based on the analysis results, the server uses a generative artificial intelligence model to generate specific health improvement advice. This advice is provided in a format that is easy for the user to understand. For example, "You need to be careful about your recent weight gain. In addition to limiting your calorie intake, try increasing your intake of vegetables and fruits."

[1518] Advice Notice

[1519] The server then sends the generated health improvement advice and warnings to mobile devices and visual information devices, where users can check the advice in real time through a dedicated application on their mobile devices and visual information devices in physical stores.

[1520] Examples and prompts

[1521] For example, consider the case where a visitor with user ID 12345 visits a physical store and the following data is collected:

[1522] Weight: 85 kg

[1523] Heart rate: 105 bpm

[1524] Number of steps: 4000

[1525] Nutrients: "fast_food"

[1526] Based on this data, specific examples of prompt sentences for the generative artificial intelligence model are as follows:

[1527] "Evaluate the health risks of user ID: 12345 and generate advice. Weight: 85kg, Heart Rate: 105bpm, Steps: 4000, Nutrition: 'fast_food'"

[1528] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1529] Step 1:

[1530] The user sets up the wearable device and mobile device. The user installs a dedicated application on the mobile device and enters personal information to link the devices.

[1531] Input: User's personal information, initial settings of wearable device

[1532] Output: Devices are linked and working properly

[1533] Step 2:

[1534] Wearable devices and mobile devices collect various data about our daily lives: smartwatches measure heart rate and steps, smart scales weigh us, and mobile devices collect GPS data.

[1535] Input: User's daily activity data (heart rate, number of steps, weight, distance traveled, etc.)

[1536] Output: Raw data collected

[1537] Step 3:

[1538] The mobile device sends the collected data to a cloud database, which is done automatically by a dedicated application.

[1539] Input: Raw data collected

[1540] Output: Data stored in a cloud database

[1541] Step 4:

[1542] The server preprocesses the data stored in the cloud database, for example, by performing data cleansing (filling in missing values, removing outliers) and data normalization (scaling values).

[1543] Input: Data stored in the cloud database

[1544] Output: Preprocessed data

[1545] Step 5:

[1546] The server analyzes the pre-processed data and assesses the user's health status, such as analyzing long-term weight trends, dietary habits, and walking patterns.

[1547] Input: Preprocessed data

[1548] Output: Analysis results (weight change trends, nutritional balance evaluation, abnormal walking pattern detection, etc.)

[1549] Step 6:

[1550] The server uses the generative AI model to generate health improvement advice, such as "You need to be careful about your recent weight gain. In addition to limiting your calories, try increasing your intake of vegetables and fruits."

[1551] Input: Analysis results

[1552] Output: Generated health improvement advice

[1553] Step 7:

[1554] The server notifies the mobile terminal and visual information device of the generated advice, which the user can then check on the visual information device (e.g., smart glasses, digital signage) in the physical store and on the mobile terminal.

[1555] Input: Generated health improvement advice

[1556] Output: Advice displayed on mobile devices and visual information devices

[1557] Step 8:

[1558] The user can check the generated advice and work to improve their lifestyle. Feedback can be sent to the server if necessary.

[1559] Input: Health improvement advice, user feedback

[1560] Output: Improved lifestyle data (if any feedback is provided, it will be sent to the server)

[1561] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1562] This invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and combines it with an emotion engine to support health improvement. This system constantly monitors the user's health and emotional state, and provides appropriate advice to reduce health risks.

[1563] Device settings and integration

[1564] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[1565] Data collection and transmission

[1566] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1567] A smart scale measures the user's weight every morning and sends the data to a mobile device.

[1568] The food logging app records the user's meal contents and sends the information to a mobile device.

[1569] The smartwatch measures steps, heart rate, and walking patterns and transmits the data to a mobile device.

[1570] Your smartphone uses GPS data to record your distance traveled and walking speed.

[1571] The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[1572] Data accumulation and preprocessing

[1573] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[1574] Data Preprocessing and Sentiment Analysis

[1575] The server preprocesses the accumulated data, specifically by performing data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[1576] For emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment.

[1577] Data analysis

[1578] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[1579] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[1580] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[1581] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[1582] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[1583] Health assessment

[1584] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[1585] Generate health improvement advice

[1586] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1587] Advice Notice

[1588] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app on the mobile device. If necessary, the user can also send feedback and ask follow-up questions.

[1589] User feedback and continuous improvement

[1590] Users improve their lifestyle habits based on the advice they receive. Feedback and questions are also entered within the app, and the server uses a generative AI model to provide appropriate answers and additional advice. This enables continuous health management through interaction with the user.

[1591] As described above, this system combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage the user's health and emotional state in real time and provide specific and appropriate advice, allowing users to detect health risks early and take preventive measures.

[1592] The processing flow will be explained below.

[1593] Step 1:

[1594] The user initially configures the smart device (e.g., smart watch, smart scale) and mobile device. The user installs a dedicated app on the smartphone and enters personal information (age, gender, height, weight, etc.). The user also configures the app to enable voice recognition and face recognition functions.

[1595] Step 2:

[1596] Terminals (smart devices and smartphones) collect data from daily life. Specifically, a smart scale measures weight and sends the data to a smartphone. A smart watch measures steps taken and heart rate and sends the data to a smartphone. A food log app records the user's diet and sends the information to a smartphone. The smartphone also uses GPS data to record distance traveled and walking speed. Furthermore, the smartphone uses its built-in microphone and camera to collect voice data and facial expression data from the user.

[1597] Step 3:

[1598] The server accumulates various data sent from the mobile device (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Specifically, each data item is saved in the corresponding table.

[1599] Step 4:

[1600] The server preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. The same preprocessing is performed on emotion data.

[1601] Step 5:

[1602] The emotion engine analyzes voice data and facial expression data to determine the user's emotional state. Specifically, it analyzes the tone and speed of the voice from the voice data and performs micro-expression analysis on the facial expression data to detect emotions such as joy, anger, and sadness. This emotional state data is integrated with other health data and used for a comprehensive assessment.

[1603] Step 6:

[1604] The server analyzes the preprocessed data and the emotion engine's judgment results using machine learning models and statistical methods. The analysis results are as follows:

[1605] Weight change trend analysis: Analyze long-term weight gain and loss patterns to detect abnormal weight changes.

[1606] Dietary assessment: Dietary data is assessed using a nutrient balance model to identify nutrient deficiencies or excesses.

[1607] Walking pattern analysis: Analyzes daily variations in step count, speed, and stride length to detect abnormal patterns.

[1608] Emotional state assessment: Evaluate the emotional state determined by the emotion engine comprehensively to detect stress and mental instability.

[1609] Step 7:

[1610] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state.

[1611] Step 8:

[1612] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1613] Step 9:

[1614] The server notifies the mobile device of the generated advice, specifically by sending a message to the user using the in-app notification function.

[1615] Step 10:

[1616] Users can check the advice provided in the app and improve their lifestyle habits as needed. They can also enter questions and feedback in the app and send them to the server.

[1617] Step 11:

[1618] The server analyzes the user's feedback using a generative AI model, generates appropriate answers and additional advice, and sends them to the mobile device, enabling continuous health management through interaction with the user.

[1619] Example 2

[1620] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1621] Conventional health management systems simply collect and analyze physical data (weight, number of steps, heart rate, etc.) and do not provide comprehensive health assessments that take into account the user's emotional state. As a result, the user's stress and mental health status may be overlooked, making it difficult to provide comprehensive health support. In addition, the health improvement advice provided is often general and lacking in specificity, making it difficult for users to understand. This presents a challenge, making it difficult to achieve effective health improvement.

[1622] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting various data on the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing voice data and facial expression data to determine the user's emotional state, means for analyzing the preprocessed data and emotional data to determine and evaluate the user's health state, means for generating health improvement advice based on the determination and evaluation, and means for notifying the mobile terminal of the generated advice. This makes it possible to comprehensively analyze the user's physical data and emotional data and provide comprehensive and specific health improvement advice.

[1623] A "wearable device" is an electronic device that is attached to the user's body or clothing and can collect various data from daily life.

[1624] A "mobile terminal" is a portable information processing device, such as a smartphone or tablet, that can receive and process data from a wearable device.

[1625] A "cloud database" is a database that can be accessed via the Internet and is a system that can store and manage various collected data.

[1626] "Preprocessing" refers to a series of processes carried out to prepare collected raw data in an analyzable format, and specifically includes filling in missing values, removing outliers, and normalizing the data.

[1627] An "emotion engine" is a software module or algorithm that analyzes voice data and facial expression data to determine the user's emotional state.

[1628] "Health status assessment" refers to the process of comprehensively assessing the user's physical and mental health status by analyzing the pre-processed data and emotional data.

[1629] "Health improvement advice" refers to advice that includes specific suggestions and instructions for improving the user's lifestyle and behavior based on the results of the health status assessment and evaluation.

[1630] A "generative artificial intelligence model" is an artificial intelligence technology that generates new information and sentences based on input data and prompts, and is used to provide health improvement advice to users in a format that is easy to understand.

[1631] The present invention relates to a system that utilizes wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from a user's daily life, and combines it with an emotion engine to support health improvement. In this embodiment, a specific method for reducing health risks by constantly monitoring the user's health and emotional state and providing appropriate advice will be described.

[1632] The system can be implemented using the following hardware and software combinations:

[1633] Wearable devices (e.g., smart watches, smart scales)

[1634] Mobile devices (e.g. smartphones)

[1635] Cloud Database

[1636] Emotion Engine

[1637] Machine learning models

[1638] Generative Artificial Intelligence Model

[1639] Device settings and integration

[1640] First, the user connects a wearable device such as a smartwatch or smart scale to their smartphone. After installing a dedicated app on their smartphone, they launch the app and enter their personal information, such as their name, age, gender, height, and weight. The data measured by each wearable device is then automatically sent to the mobile terminal.

[1641] Data collection and transmission

[1642] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1643] The smart scale allows users to weigh themselves every morning and sends the data to a mobile device via Bluetooth.

[1644] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[1645] The smartwatch measures the user's steps, heart rate, walking patterns, etc. and transmits the data to a mobile device in real time.

[1646] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[1647] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[1648] Data accumulation and preprocessing

[1649] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each piece of data is stored in a corresponding table; for example, weight data is stored in the "weight" table, and walking data is stored in the "walking" table.

[1650] Data Preprocessing and Sentiment Analysis

[1651] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data. For emotion data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This result is integrated with other health data.

[1652] Data analysis

[1653] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[1654] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[1655] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[1656] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[1657] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[1658] Health status assessment and generation and notification of health improvement advice

[1659] The server makes a comprehensive assessment of the user's health condition based on the analysis results and uses a generative AI model to generate health improvement advice. An example of a prompt is, "The user's weight has increased dramatically recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation." The generative AI model then generates specific advice, such as, "Recent weight gain and increased stress have been confirmed. Try to get some moderate exercise and take time to relax." This advice is sent to the mobile device, and the user can check it through a dedicated app.

[1660] User feedback and continuous improvement

[1661] Users can improve their lifestyle habits based on the advice they receive. Feedback and follow-up questions can be provided through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. Continuous health management is possible through interaction with the user.

[1662] As described above, the present invention combines a wearable device, a mobile terminal, and an emotion engine to monitor and manage a user's health and emotional state in real time and provide specific and appropriate advice, thereby enabling users to detect health risks early and take preventive measures.

[1663] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1664] Step 1: Device Setup and Integration

[1665] The user sets up a wearable device such as a smart watch or smart scale and links it to a mobile device. The user installs and launches a dedicated app and enters personal information (name, age, gender, height, weight, etc.). The entered information is saved on the mobile device, and the wearable device is set up to automatically send measured data to the mobile device. The input here is the user's personal information and device measurement data, and the output is the completed setup state on the mobile device.

[1666] Step 2: Data collection and transmission

[1667] Your devices (smart devices and smartphones) automatically collect the following data through your daily activities:

[1668] The smart scale allows users to weigh themselves every morning and transmits the data to a mobile device via Bluetooth.

[1669] The food log app records the meal details (breakfast, lunch, dinner, snacks, etc.) manually entered by the user and sends the information to a mobile device.

[1670] The smartwatch measures the user's steps, heart rate, and walking patterns and transmits the data in real time to a mobile device.

[1671] The smartphone automatically records the distance traveled and walking speed using the built-in GPS.

[1672] The mobile device uses a camera and microphone to collect voice data and facial expressions of the user and stores this data.

[1673] The input here is the various data measured by the device, and the output is all the data sent to the mobile terminal.

[1674] Step 3: Data accumulation and preprocessing

[1675] The server accumulates various data sent from mobile devices (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) in a cloud database in real time. Each data item is stored in a corresponding table. The input is the raw data sent from the mobile device, and the output is the organized data accumulated in the cloud database.

[1676] Step 4: Data preprocessing and sentiment analysis

[1677] The server preprocesses the accumulated data. Specifically, it performs data cleansing (filling in missing values ​​and removing outliers) and data normalization (scaling values). The input here is the accumulated raw data, and the output is the preprocessed, clean data.

[1678] Regarding emotional data, the emotion engine analyzes voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). The input is voice data and facial expression data, and the output is the determined emotional state.

[1679] Step 5: Data analysis

[1680] The server uses the pre-processed data and the results of the emotion engine to perform the following analysis using machine learning models and statistical methods:

[1681] Weight trend analysis: Analyze long-term weight data to check for sudden increases or decreases.

[1682] Dietary assessment: Dietary data is input into a nutritional balance model to identify nutrient deficiencies and excesses.

[1683] Walking pattern analysis: Analyzes fluctuations in step count, stride length, and speed to detect unusual patterns.

[1684] Emotional state assessment: The emotion engine's results are combined with other data to assess stress levels and mental instability.

[1685] The input here is the preprocessed data and emotion determination results, and the output is each analysis result.

[1686] Step 6: Assessing health status and generating health improvement advice

[1687] The server comprehensively evaluates the user's health condition based on the analysis results and generates health improvement advice using a generative AI model. For example, based on a prompt such as "The user's weight has increased rapidly recently, and the emotion engine has detected a high stress state. Please provide appropriate health improvement advice for this situation," the generative AI model generates specific advice (e.g., "Recent weight gain and increased stress have been confirmed. Try to get moderate exercise and spend time relaxing.").

[1688] The input here is the analysis results and a prompt for the generative AI model, and the output is the generated health improvement advice.

[1689] Step 7: Advice Notification

[1690] The server notifies the mobile device of the generated advice, and the user can check the advice through a dedicated app. The input here is the generated advice, and the output is a notification to the user.

[1691] Step 8: User feedback and continuous improvement

[1692] The user improves their lifestyle habits based on the advice they receive. The user can provide feedback or follow-up questions through the app, and the server uses a generative AI model to provide appropriate answers and follow-up advice. The input here is feedback and questions from the user, and the output is answers and follow-up advice from the generative AI model. This enables continuous health management through interaction with the user.

[1693] The above is the flow of processing of the program of this system.

[1694] (Application example 2)

[1695] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1696] Current systems in health management and fitness gyms are not sufficient to monitor users' health and emotional states in real time and provide customized exercise menus and health advice suited to each individual user. Therefore, there is a need for a method to detect users' health risks early and efficiently provide appropriate improvement measures.

[1697] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1698] In this invention, the server includes means for collecting various data about the user's daily life using a wearable device and a mobile terminal, means for storing the collected data in a cloud database, means for preprocessing the stored data, means for analyzing the preprocessed data to determine and evaluate the user's health status, means for generating health improvement advice based on the determination and evaluation, means for notifying the mobile terminal of the generated advice, means for analyzing voice data and facial expression data to determine the user's emotional state, means for generating customized exercise menus and health advice based on the user's health data and emotional data, and means for collecting and analyzing member data in cooperation with smart fitness machines and sensors in the store and providing customized advice in real time, thereby enabling detailed analysis of the user's health status and emotional state and enabling truly personalized health management and fitness support.

[1699] A "wearable device" is a device that is worn on the user's body and is used to collect data from daily life.

[1700] A "mobile terminal" is an electronic device that can be carried by a user and is used to collect, process, and transmit data.

[1701] A "cloud database" is a remote data storage system for storing, managing, and accessing data over the Internet.

[1702] "Preprocessing" is the process of shaping and cleansing collected data to make it easier to analyze.

[1703] "Health status" refers to the overall state of a user's physical and mental health.

[1704] "Assessment and evaluation" is the process of analyzing the user's current situation based on collected and preprocessed data and identifying health risks, etc.

[1705] "Health improvement advice" refers to specific suggestions or instructions provided to improve the user's health.

[1706] "Voice data" is acoustic information generated by the user's voice.

[1707] "Facial expression data" is image data for capturing and analyzing the facial expressions of a user.

[1708] An "emotional state" is the emotional state a user is experiencing at a particular point in time.

[1709] A "customized exercise menu" is an exercise plan designed to suit an individual user's health condition and goals.

[1710] "Smart fitness machines" are devices used in fitness gyms that have advanced functions for collecting and analyzing exercise data.

[1711] A "sensor" is a device that senses a specific physical phenomenon and converts it into data.

[1712] "Real-time" refers to a state in which data is collected and processed immediately, without delay.

[1713] This invention provides a health support system customized for fitness gyms. The system uses wearable devices and mobile terminals to collect, analyze, and evaluate health-related data from daily life, and by combining this with an emotion engine, it monitors the user's health and emotional state in real time and provides appropriate advice.

[1714] First, the user sets up and links a wearable device (e.g., smartwatch, smart scale) with a mobile device (e.g., smartphone). The user installs a dedicated app on the smartphone and enters personal information as the initial setup. This allows data measured by the wearable device to be sent to the mobile device and stored in a cloud database.

[1715] Mobile devices automatically collect the following data through daily activities: A smart scale measures the user's weight and sends the data to the mobile device. A food logging app records the user's diet and sends the information to the mobile device. A smartwatch measures the number of steps taken, heart rate, and walking patterns and sends the data to the mobile device. A smartphone uses GPS data to record distance traveled and walking speed. The mobile device collects the user's voice data and facial expressions using a camera and microphone.

[1716] The server accumulates various data (weight data, walking data, dietary data, GPS data, voice data, facial expression data, etc.) sent from mobile devices in a cloud database in real time. The server then preprocesses the accumulated data. Specifically, it performs data cleansing (complementing missing values ​​and removing outliers) and data normalization (scaling values) to align the time series data.

[1717] The emotion engine analyzes the voice data and facial expression data to determine the user's emotional state (e.g., joy, anger, sadness, etc.). This determination result is integrated with other health data and used for a comprehensive assessment. The server then analyzes the pre-processed data and the emotion engine's determination result using machine learning models and statistical methods. The analysis includes trend analysis of weight change, evaluation of dietary habits, analysis of walking patterns, and evaluation of emotional state.

[1718] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high-risk score will be generated. A warning will also be generated if the emotion engine detects a high stress state. The server uses a generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in a format that is easy for the user to understand, such as, "You have recently gained weight and experienced increased stress. Try to get some moderate exercise and take time to relax."

[1719] The generated advice is provided through a prompt sentence as follows:

[1720] example:

[1721] "Recent data has confirmed that your weight is increasing, but your walking frequency has also increased, so it appears that you are exercising appropriately. However, your diet tends to be high in calories, so try to eat a more balanced diet. Your emotional data also suggests that your stress levels are a little high, so try incorporating relaxation techniques."

[1722] This system makes it easier for users to improve their daily lifestyle habits, providing comprehensive health management and fitness support.

[1723] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1724] Step 1:

[1725] The user sets up and links the wearable device and mobile device. The user installs a dedicated app on their smartphone and enters their personal information. This sets up the wearable device to send measured data to the mobile device.

[1726] Input: User's personal information, wearable device settings information

[1727] Output: Data integration setup is complete and data is ready to be sent

[1728] Step 2:

[1729] Mobile devices automatically collect health-related data through daily activities. A smart scale weighs you every morning and sends the data to your smartphone. A food logging app records what you eat. A smartwatch measures your steps, heart rate, and walking patterns. Your smartphone uses GPS data to record distance traveled and walking speed. Your mobile device uses a camera and microphone to collect audio and facial expression data.

[1730] Input: User's daily life activity data

[1731] Output: Various health-related data (weight, number of steps, dietary details, heart rate, distance traveled, voice data, facial expression data)

[1732] Step 3:

[1733] The mobile device sends the collected data to a cloud database, which stores health-related data in real time.

[1734] Input: Health-related data collected by mobile devices

[1735] Output: Data stored in a cloud database

[1736] Step 4:

[1737] The server preprocesses the data stored in the cloud database, performs data cleansing, fills in missing values, removes outliers, scales values, and aligns time series data.

[1738] Input: Health-related data stored in a cloud database

[1739] Output: Preprocessed data (aligned time series data, cleansed data)

[1740] Step 5:

[1741] The server analyzes the voice data and facial expression data and uses an emotion engine to determine the user's emotional state, for example, analyzing emotions from the voice data and analyzing the facial expression data to determine the emotional state.

[1742] Input: Preprocessed speech and facial expression data

[1743] Output: Emotional state (e.g., happy, angry, sad, etc.)

[1744] Step 6:

[1745] The server analyzes the preprocessed health data and the emotional state calculated by the emotion engine using machine learning models and statistical methods to analyze weight trends, evaluate dietary habits, analyze walking patterns, and evaluate emotional states.

[1746] Input: Preprocessed health data, emotional state

[1747] Output: Health status analysis results (weight change, dietary assessment, walking patterns, emotional state assessment)

[1748] Step 7:

[1749] The server evaluates the user's health and emotional state based on the analysis results and calculates an abnormality score and health risk level. For example, if there is a sudden weight gain, a high risk score is generated. If the emotion engine detects a high stress state, it generates a warning.

[1750] Input: Health status analysis results

[1751] Output: Health score, health risk level

[1752] Step 8:

[1753] The server uses the generative AI model to generate health improvement advice based on the analysis results. The advice takes into account the user's health and emotional state and is provided in the form of prompt sentences.

[1754] Input: Health score, health risk level

[1755] Output: Health improvement advice (based on generative AI model)

[1756] Step 9:

[1757] The server sends the generated advice to the mobile device, where the user can check the advice through a dedicated app on the mobile device and obtain specific guidelines for improving their daily lifestyle habits.

[1758] Input: Health Improvement Advice

[1759] Output: Notification to mobile device, providing advice to user

[1760] This makes it easier for users to improve their daily lifestyle habits, enabling comprehensive health management and fitness support.

[1761] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1763] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1764] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1765] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1766] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1767] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1768] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1769] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1770] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1771] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1772] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1773] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1774] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1775] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1776] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1777] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1778] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1779] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1780] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1781] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1782] The following is further disclosed regarding the above embodiment.

[1783] (Claim 1)

[1784] A means for collecting various data of a user's daily life using a wearable device and a mobile terminal;

[1785] means for storing the collected data in a cloud database;

[1786] means for preprocessing the accumulated data;

[1787] A means for analyzing the...

Claims

1. A means for collecting various data of a user's daily life using a wearable device and a mobile terminal; means for storing the collected data in a cloud database; means for preprocessing the accumulated data; A means for analyzing the preprocessed data to determine and evaluate the user's health condition; means for generating health improvement advice based on said determination and evaluation; The system further comprises means for notifying a mobile terminal of the generated advice.

2. The system of claim 1 , wherein the health improvement advice is generated based on assessing weight change, diet, step count, and walking patterns.

3. The system of claim 1 , wherein the health improvement advice is provided in a user-friendly format using a generative artificial intelligence model.

Images