System

Abstract

A system is provided.SOLUTION: The system includes a sensor means for collecting personal health information, a communication means for transmitting the collected health information to a server, a generation and AI means for analyzing the transmitted health information and generating personal advice, a communication means for transmitting the generated advice to user terminals, and a display means for notifying the advice to be displayed on the user terminals.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Health management is becoming increasingly important in modern society, especially in this era of the 100-year lifespan, where all generations are expected to maintain their health over the long term. However, currently popular activity trackers and health management devices are limited to collecting and displaying data and lack the functionality to provide appropriate advice to each user. Furthermore, there is a lack of easy ways to manage dietary calorie intake, making comprehensive health management difficult. Therefore, there is a need for a system that can provide practical, specific advice and comprehensively manage health data such as diet, exercise, and sleep. [Means for solving the problem]

[0005] This invention is a system that includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data to a server, an AI generation means for analyzing the transmitted health data and generating personalized advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. It also includes an image recognition means for analyzing dietary data and calculating calorie intake, and a data storage means for providing long-term health management advice based on the collected health data and calorie intake data, thereby achieving comprehensive health management. This system allows users to understand their health condition on a daily basis and adjust their behavior based on specific advice, thereby supporting long-term health maintenance.

[0006] The term "sensor means" includes various sensor devices for collecting personal health data.

[0007] The term "communication means" includes devices and protocols for network communication to transmit collected data to a server.

[0008] A "generative AI means" is a system that includes artificial intelligence that analyzes the received data and generates personalized health advice.

[0009] The "display means" is a device including a display and an application for notifying the user of the generated advice.

[0010] "Image recognition means" includes an image analysis system for analyzing photos of meals and calculating calorie intake.

[0011] "Data storage means" includes a database and storage system for storing and managing collected health data and calorie intake data, and providing advice for long-term health management based on the analysis results. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal.

[0034] Overall system overview

[0035] This system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice. The advice is then sent to the terminal and notified to the user.

[0036] Data collection

[0037] The device is equipped with a step sensor, heart rate sensor, sleep sensor, etc., and collects the user's step count, heart rate, sleep time, etc. in real time. It also has a built-in camera, so it can collect meal data by allowing the user to take photos of their meals.

[0038] Data transmission

[0039] The device periodically sends the collected health and dietary data to a server, and the transmission is encrypted to ensure security.

[0040] Data analysis and generative AI

[0041] The server analyzes the received data and generates personalized health advice using AI. Specifically, if the step count data is low, it will point out a lack of exercise and advise, "You should walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake.

[0042] Sending and viewing advice

[0043] The advice generated by the server is delivered to the device. The device notifies the user of the advice and displays it on a display or app. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0044] Specific examples

[0045] Scenario 1: Step data collection and advice

[0046] Device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, such as 3,000.

[0047] Device: Sends this step count data to the server.

[0048] Server: Analyzes the received step count data and determines that the amount of exercise is insufficient.

[0049] Server: Using generative AI, generate advice such as "You should walk a little more today."

[0050] Server: Sends the generated advice to the terminal.

[0051] Terminal: Notify the user.

[0052] Scenario 2: Food image analysis and calorie management

[0053] Device: The user takes a photo of the food they had for lunch.

[0054] Device: Sends this food image to the server.

[0055] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0056] Server: Determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0057] Server: Sends the generated advice to the terminal.

[0058] Terminal: Notify the user.

[0059] In this way, this system provides comprehensive support for the user's health through a series of processes, from data collection by sensors to data analysis, and the generation and notification of individualized advice. Users can easily understand their health status in their daily lives and improve their lifestyle habits based on the advice.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, meal times, etc.) in real time. This includes a step sensor, heart rate sensor, sleep sensor, and camera. During this time, the device periodically reads the sensor values ​​and records them in its internal memory.

[0063] Step 2:

[0064] User: When eating, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. At this time, a confirmation message for the photo-taking procedure and saving is displayed, allowing the user to check the contents of the photo.

[0065] Step 3:

[0066] Device: The collected health data and meal images are sent to the server at regular intervals. All information is encrypted before transmission to ensure communication security.

[0067] Step 4:

[0068] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. The analysis includes the process of evaluating data such as the number of steps, heart rate, and sleep time over time.

[0069] Step 5:

[0070] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. In this process, an image analysis model recognizes the contents of the meal and compares it with a general calorie database to estimate the calories.

[0071] Step 6:

[0072] Server: Using generation AI, it generates personalized health advice based on the analysis results. For example, if the number of steps is low, it generates advice such as "You should walk a little more today," and if the calorie intake is high, it generates advice such as "You should eat more vegetables at your next meal."

[0073] Step 7:

[0074] Server: The server sends the generated advice to the device. At this time, the advice is customized for each user and organized in an appropriate format. The content of the advice is concise and organized in a format that is easy for the user to understand.

[0075] Step 8:

[0076] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0077] Step 9:

[0078] User: Improves lifestyle habits based on the advice received, for example, taking an extra walk after receiving a notification or adding more vegetables to their next meal.

[0079] Through these steps, the system provides comprehensive support for users in maintaining their health and offers specific, practical advice.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional health management systems have had difficulty effectively analyzing collected health data and providing personalized health advice. They also lacked the functionality to analyze users' dietary data in detail and accurately calculate calorie intake. Furthermore, they lacked mechanisms for notifying users of generated advice in real time. For these reasons, there was a need for the development of a system that could comprehensively support users' health management.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data to the server, a generation AI means for analyzing the transmitted health data and generating personalized advice, a generation AI model means for inputting a prompt sentence into the generation AI model and generating health advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. This makes it possible to analyze the user's health data and dietary data in detail and quickly and effectively generate and notify personalized health advice.

[0085] "Sensor means" refers to a device used to collect personal health data, and includes a step sensor, a heart rate sensor, a sleep sensor, etc.

[0086] "Communication means" refers to the devices and technologies used to transmit collected data to the server, including wireless communication modules and internet connections.

[0087] "Generative AI Means" are the artificial intelligence algorithms and systems used to analyze the submitted data and generate personalized advice.

[0088] A "generative AI model means" is an artificial intelligence model used to generate health advice in response to a prompt statement.

[0089] "Display means" refers to the devices or techniques used to notify advice that is displayed on a user terminal, including displays and notification systems.

[0090] "Image recognition means" is a technology used to analyze collected dietary data and calculate calorie intake.

[0091] "Data storage means" refers to a system used to store and manage collected health and dietary data over the long term and provide ongoing health management advice.

[0092] This invention is a system that provides comprehensive support for the health of users in their daily lives. The system consists of a wearable device (hereinafter referred to as a "terminal") worn by the user and a server that works in conjunction with the device.

[0093] Data collection

[0094] Device: The device is equipped with various sensors, such as a step sensor, heart rate sensor, and sleep sensor, and collects health data such as the user's number of steps, heart rate, and sleep time in real time.The device also has a camera, which collects dietary data by allowing the user to take photos of their meals.

[0095] Data transmission

[0096] Device: All collected health and dietary data is encrypted and sent to a server at regular intervals using encryption technology such as AES-256. Transmission is via Wi-Fi or mobile data.

[0097] Data analysis and generative AI

[0098] Server: The server stores the received data in a database, and uses an analysis module to detect patterns and outliers in the data. It then inputs prompts into a generative AI model to generate health advice.

[0099] An example of a specific prompt: "Generate advice for the user on days when their step count data is low."

[0100] For example, the generative AI model might generate advice such as, "You should walk a little more today. Why not try a walk in a nearby park?"

[0101] Generative AI model: The generative AI model used utilizes the latest natural language processing techniques to generate detailed and relevant advice based on the prompt.

[0102] Sending and viewing advice

[0103] Server: The generated advice is encrypted and sent to the terminal.

[0104] Device: Interprets the received advice and notifies the user. Notification methods include push notifications and banner displays. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed on the screen.

[0105] Specific examples

[0106] Step 1: Collecting step data and giving advice

[0107] The device records the user's steps throughout the day, and if the user takes only 3,000 steps in a day, the data is stored on the device.

[0108] The terminal transmits this step count data to the server.

[0109] The server analyzes the received step count data and determines that the amount of exercise is insufficient.

[0110] The server uses a generative AI to generate advice such as, "You should walk a little more today."

[0111] The server transmits the generated advice to the terminal.

[0112] The terminal notifies the user and displays advice on the display.

[0113] Step 2: Food image analysis and calorie management

[0114] The terminal takes a photo of the food the user had for lunch.

[0115] The device sends this food image to the server.

[0116] The server analyzes the food image and calculates the calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0117] The server determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0118] The server transmits the generated advice to the terminal.

[0119] The terminal notifies the user and displays advice on the display.

[0120] As described above, the present invention combines various technologies such as sensor means, communication means, and generation AI means to grasp a user's health condition in real time and provide personalized health advice, allowing users to easily manage their health condition in their daily lives and continuously improve their lifestyle habits.

[0121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0122] Step 1: Data collection

[0123] Device: Using an activity tracker, heart rate sensor, and sleep sensor, the device collects real-time health data (number of steps, heart rate, sleep time, etc.) from the user's daily life. For example, a smartwatch measures the number of steps a user takes in a day and stores the data on the device when they reach 10,000 steps.

[0124] Device: The user takes a photo of their lunch using their smartphone camera to collect meal data. The device stores the captured image in its internal storage.

[0125] Input: Raw data from sensors and food photos.

[0126] Output: Health and dietary data stored on the device.

[0127] Step 2: Send data

[0128] Terminal: All collected data is sent to the server at regular intervals. Before being sent, the data is securely protected using AES-256 encryption technology.

[0129] On your device: Using Wi-Fi or mobile data, the encrypted data is sent to a server.

[0130] Input: Health and dietary data stored on the device.

[0131] Output: The encrypted data sent to the server.

[0132] Step 3: Receiving and storing data

[0133] Server: Decrypts the received encrypted data and stores it in a database. The database uses a storage system that allows for quick search and analysis.

[0134] Input: Encrypted data.

[0135] Output: Decrypted data stored in a database.

[0136] Step 4: Data analysis

[0137] Server: Passes the data stored in the database to the analysis module, which detects patterns in the step count, heart rate, and sleep data and identifies outliers, such as a low current step count compared to the average step count over the past week.

[0138] Input: Health data from the database.

[0139] Output: Analysis results (e.g., information indicating low step count).

[0140] Step 5: Generative AI model generates advice

[0141] Server: Based on the analysis results, input a prompt to the generative AI model. For example, "Please generate advice for days when the user's step count data is low."

[0142] Generative AI model: Generates detailed and relevant health advice based on prompts. Generative AI takes into account context and past data to provide optimal advice.

[0143] Input: prompt statement, analysis result.

[0144] Output: Health advice (e.g., "You should walk a little more today").

[0145] Step 6: Send advice

[0146] Server: Encrypts the generated advice and sends it to the terminal.

[0147] Input: The generated advice.

[0148] Output: Encrypted advice data to the terminal.

[0149] Step 7: Display Advice

[0150] Device: Interprets the received advice and notifies the user. This can be done using push notifications or in-app banners. For example, the device might display advice such as "Your heart rate tends to be high. Take some time to relax."

[0151] Input: Encrypted advice data sent by the server.

[0152] Output: Displaying information and advice messages to the user.

[0153] (Application example 1)

[0154] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0155] In modern society, personal health management is an important issue. In particular, lack of exercise and improper diet often have negative effects on health, so there is a need for effective means of managing these. Furthermore, when users use food delivery services, there is a lack of functionality that provides optimal meal options based on their health status. For this reason, a system is needed that can collect and analyze individual health data in real time and provide appropriate advice to comprehensively support users' health management.

[0156] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0157] In this invention, the server includes sensor means for collecting personal health data, communication means for transmitting the collected health data to the server, generation AI means for analyzing the transmitted health data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, display means for notifying the user of the advice to be displayed on the user terminal, and suggestion means for suggesting optimal meal options based on the user's health data. This allows users to understand their own health status in real time and enables them to make health-conscious meal choices even when using food delivery services.

[0158] The "sensor means" is a device for collecting personal health data, and has the function of detecting the number of steps taken, heart rate, sleep time, etc. in real time.

[0159] "Communication means" refers to a device or system that has the function of transmitting collected health data to a server, and is capable of encrypting data communication.

[0160] "Generative AI means" refers to a device or system that uses artificial intelligence technology to analyze health data sent to the server and generate personalized health advice.

[0161] The "display means" refers to a display or application that notifies and displays advice sent from the server on the user terminal.

[0162] "Dietary data" is information about the meals consumed by the user, and includes photos of the meals and manually entered lists of ingredients.

[0163] The "image recognition means" is a device or system that has the function of analyzing collected dietary data and calculating calorie intake.

[0164] "Data storage means" refers to a device or system that has the function of storing and accumulating collected health data and calorie intake data over the long term.

[0165] The "suggestion means" is a device or system for suggesting optimal meal options based on the user's health data, and has the function of assisting in meal selection based on the generated advice.

[0166] Overall system overview

[0167] This invention is a system that comprehensively supports personal health management. The system includes a sensor means for collecting health data, a communication means for transmitting the data to a server, an AI generation means for analyzing the data and generating personalized advice, a means for transmitting the advice to a terminal and displaying it, and a suggestion means for suggesting optimal meal options based on the user's health data. The system works in conjunction with a wearable device to collect the user's health data in real time. This data is analyzed by the server and provided to the user as advice.

[0168] Hardware and software used

[0169] Wearable device: Equipped with step sensors, heart rate sensors, and sleep sensors.

[0170] Server: Uses data analysis and generative AI models (e.g., GPT-4).

[0171] User device: Advice is displayed and notified using a display or application.

[0172] Data processing and calculation

[0173] 1. Health Data Collection:

[0174] Wearable devices collect real-time health data such as the user's steps, heart rate, and sleep duration.

[0175] The device sends the collected data to a server at regular intervals, and the data is encrypted for security purposes.

[0176] 2. Data transmission:

[0177] The collected health data is transmitted to a server via a communication means and analyzed by the server.

[0178] 3. Data analysis and advice generation:

[0179] The server analyzes the received health data and generates personalized health advice using a generative AI model (e.g., GPT-4).

[0180] For example, if the number of steps taken is low, the system will generate advice such as, "You should walk a little more today." It will also analyze photos of meals, calculate calorie intake, and provide advice on calorie management.

[0181] 4. Send and display advice to user device:

[0182] The advice generated by the server is transmitted to the user terminal via a communication means.

[0183] The user terminal notifies the user of the advice and displays it in an application or on a display.

[0184] 5. Suggestions based on health advice:

[0185] The suggestion tool provides optimal meal options based on the user's health data, for example, healthy meal options if the user is not getting enough exercise.

[0186] Specific examples

[0187] Scenario 1: Step data collection and advice

[0188] On the device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, say 3,000.

[0189] Server: Analyzes the received step count data, determines that the amount of exercise is insufficient, and generates advice such as, "You should walk a little more today."

[0190] Terminal: The generated advice is notified to the user.

[0191] Scenario 2: Food image analysis and calorie management

[0192] Device: The user takes a photo of the food they had for lunch.

[0193] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal, it generates specific advice such as "Increase the amount of vegetables at your next meal."

[0194] Terminal: The generated advice is notified to the user.

[0195] Prompt Sentence Examples

[0196] User health data: 3000 steps (daily), heart rate 80 (average), sleep time 6 hours (average)

[0197] User's recent meals: pizza, hamburger

[0198] Prompt to spawn AI:

[0199] "The user is not very active, so please suggest a lower calorie option for their next meal."

[0200] This allows users to understand their health status in real time and choose healthy meals when using food delivery services.In addition, by using the suggestion method, optimal advice based on the user's health data is provided, allowing for efficient health management.

[0201] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0202] Step 1:

[0203] A user wears a wearable device while going about their daily life. This device is equipped with a step sensor, a heart rate sensor, and a sleep sensor, and has the function of collecting health data such as the user's steps, heart rate, and sleep time in real time. The input of the data collection is the user's physical activity, and the output is the collected health data.

[0204] Step 2:

[0205] The device sends the collected health data to the server at regular intervals. The data is encrypted before transmission to ensure security. The input is the collected health data, and the output is the transmission of the encrypted health data to the server. This ensures that the user's health data reaches the server safely.

[0206] Step 3:

[0207] The server analyzes the received health data. The input is the received health data, which is analyzed using a generation AI method. Specific operations of data analysis include pattern recognition, such as low step counts and high heart rate. Based on the results of this analysis, optimal health advice is generated for each user. The output is the generated health advice.

[0208] Step 4:

[0209] The server sends the generated health advice to the user terminal. The input is the generated health advice, and the output is the transmission of the advice to the user terminal. The data is encrypted and transferred securely by the communication means.

[0210] Step 5:

[0211] The device receives the advice sent from the server and notifies the user through a display or application. The input is the health advice sent from the server, and the output is the content to be notified to the user. Specific actions can include a pop-up notification in the application or an email notification.

[0212] Step 6:

[0213] A user launches a food delivery app. The app references the health data and advice previously received from the device. The input is the user's health data and generated advice, and the output is the health-conscious meal options displayed on the food delivery menu screen.

[0214] Step 7:

[0215] The suggestion mechanism suggests optimal meal options based on the user's health data. For example, if the user is not exercising enough, it suggests "salads" or "low-calorie dishes." The generative AI model then uses a prompt such as "The user is not exercising much, so please suggest low-calorie options" to make optimal suggestions. The input is the user's health data and the prompt, and the output is a suggestion of a healthy meal option.

[0216] This will enable the entire system to consistently handle everything from data collection to advice suggestions and meal selection, providing comprehensive support for users' health management.

[0217] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0218] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. Furthermore, an emotion engine that recognizes the user's emotions is also incorporated.

[0219] Overall system overview

[0220] This system consists of a wearable device (hereafter referred to as the terminal) and a server linked to it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates individualized advice that takes the emotional data into account. The generated advice is then sent to the terminal and notified to the user.

[0221] Data collection

[0222] The device is equipped with a step sensor, heart rate sensor, sleep sensor, camera, and emotion engine. These sensors collect the user's health data (step count, heart rate, sleep time, meal time) and emotion data in real time. The emotion engine analyzes the user's facial expressions and voice to determine their mood and emotional state for the day.

[0223] Data transmission

[0224] The device periodically transmits the collected health data, food images, and emotion data to a server, and the transmission is encrypted to ensure security.

[0225] Data analysis and generative AI

[0226] The server analyzes the received data and uses AI generation to generate personalized health advice. Specifically, if the step count data is low, it will point out a lack of exercise and advise the user to "walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake. Furthermore, it adjusts the advice based on emotional data. For example, if the emotion engine detects the user's stress level, it will add the advice "Take time to relax."

[0227] Sending and viewing advice

[0228] The advice generated by the server is sent to the device. The device notifies the user of the advice and displays it on a display or in an app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" may be displayed.

[0229] Specific examples

[0230] Scenario 1: Collecting step count data and emotion data, giving advice

[0231] Device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps today, the emotion engine detects that the user's stress level is high.

[0232] Terminal: Sends collected step count data and emotion data to the server.

[0233] Server: Analyzes data and detects lack of exercise and stress.

[0234] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[0235] Server: Sends the generated advice to the terminal.

[0236] Terminal: Notify the user.

[0237] Scenario 2: Calorie management based on food image analysis and emotional state

[0238] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[0239] Device: Sends food images and emotion data to the server.

[0240] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0241] Server: Considers emotional data, determines that the calorie intake is high, and generates specific advice such as, "Increase the amount of vegetables at your next meal. Eat healthily to maintain your great mood today."

[0242] Server: Sends the generated advice to the terminal.

[0243] Terminal: Notify the user.

[0244] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[0245] The processing flow will be explained below.

[0246] Step 1:

[0247] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, etc.) in real time. This includes a step sensor, heart rate sensor, and sleep sensor. The device periodically records the data obtained from these sensors in its internal memory.

[0248] Step 2:

[0249] User: When eating a meal, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. The user is prompted to confirm that they want to save the photo.

[0250] Step 3:

[0251] On the device: The emotion engine collects emotion data from the user's facial expressions and voice. For example, it uses a facial recognition algorithm to analyze the user's facial expressions and determine their emotional state, such as joy, sadness, or anger.

[0252] Step 4:

[0253] Device: The device periodically transmits collected health data, food images, and emotion data to the server. The transmitted data is encrypted to ensure communication security.

[0254] Step 5:

[0255] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. Analysis includes time-series evaluation of health data such as number of steps, heart rate, and sleep time, calorie calculation of meals, and analysis of emotional data.

[0256] Step 6:

[0257] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. The food photos are analyzed to determine the type and amount of food, and the calorie intake is estimated.

[0258] Step 7:

[0259] Server: Using the generative AI, it generates personalized health advice based on the analysis results. For example, if the analysis results indicate a lack of exercise, it generates the advice "You should walk a little more today." If the emotion engine determines that the user is feeling stressed, it adds the advice "Take time to relax."

[0260] Step 8:

[0261] Server: Sends the generated advice to the device. The advice is customized for each user and organized in an appropriate format.

[0262] Step 9:

[0263] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app, and the user can check the content. For example, a notification may be displayed saying, "Your heart rate tends to be high. Take time to relax."

[0264] Step 10:

[0265] User: Improves lifestyle habits based on the advice received. For example, taking an extra walk or adding more vegetables to their next meal. At this stage, users can understand their own health and emotional state and optimize their behavior based on high-quality advice.

[0266] Through this series of processing steps, users receive specific advice tailored to their individual health and emotional state, enabling them to effectively manage their own health in their daily lives.

[0267] Example 2

[0268] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0269] In modern society, busy lifestyles and irregular eating habits make it difficult to manage personal health. In addition to simple health data, emotional states are also important factors in comprehensive health management. However, current systems struggle to effectively collect and analyze this data and provide users with personalized advice. Furthermore, there is a lack of systems that can manage and analyze health and emotional data together and provide users with specific behavioral guidance.

[0270] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to the server, and generation AI means for analyzing the transmitted health data and emotional data and generating individual advice. This makes it possible to provide comprehensive and personalized health management based on the user's health condition and emotional state.

[0271] "Sensor means" refers to devices or mechanisms for collecting health and emotional data of an individual.

[0272] "Communication means" refers to the network connection capability for transmitting collected health and emotion data to a server.

[0273] "Generative AI means" refers to the artificial intelligence models and algorithms used to analyze submitted data and generate personalized advice.

[0274] "Display means" refers to a display or application function for notifying advice displayed on a user terminal.

[0275] "Image recognition means" refers to image recognition technology and software used to analyze collected dietary data and calculate calorie intake.

[0276] "Data storage means" refers to a database or storage system for long-term storage and management of collected health data, emotional data, and calorie intake data.

[0277] This invention provides a system for supporting a user's health management. The system includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to a server, AI generation means for analyzing the transmitted health data and emotional data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, and display means for displaying the advice on the user terminal.

[0278] System Configuration

[0279] The entire system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice that takes the emotional data into account. The advice is then sent to the terminal and notified to the user.

[0280] Data collection

[0281] The device is equipped with the following sensors:

[0282] Pedometer: Records the number of steps taken by the user. Example: Records 5,000 steps in a day.

[0283] Heart Rate Sensor: Measures the user's heart rate. Example: Heart rate is 70 bpm on average.

[0284] Sleep sensor: Records the user's sleep time. Example: The previous night's sleep time was 7 hours.

[0285] Camera: Takes a photo of the user's meal. Example: Takes a photo of lunch.

[0286] Emotion engine: Analyzes the user's facial expressions and voice to determine their emotional state. Example: Detects that the user is smiling and determines that they are feeling happy.

[0287] The emotion engine is equipped with advanced analytical technology and can identify the user's mood and stress level for the day through facial expression and voice analysis.

[0288] Data transmission

[0289] The device sends the collected data to the server at regular intervals. This transmission is performed using encryption technology (e.g., TLS / SSL) to ensure security. The data is sent every 15 minutes, for example.

[0290] Data analysis and generative AI

[0291] The server uses advanced analysis algorithms and generative AI models (e.g., GPT-3) to analyze the received data.

[0292] Analysis of step count data: Analyze daily step count data and evaluate the amount of exercise. For example, if the number of steps taken in a day is less than 3,000, it is determined that the person is not exercising enough.

[0293] Food image analysis: Using image recognition technology (e.g., Google Cloud Vision API), calculate calorie intake from food photos. Example: Estimate 800 kcal from a photo of lunch.

[0294] Emotion data analysis: The emotion engine considers the analyzed emotion data. For example, the stress level is determined to be high.

[0295] Example prompt sentence:

[0296] "Please send the collected health and emotional data to the server." "Please analyze the received data and evaluate the health and emotional state." "Please generate appropriate health advice for the user based on the analysis results." "Please send the generated advice to the device and notify the user."

[0297] Advice Generation

[0298] Based on the analysis results, the server generates personalized health advice using a generative AI model:

[0299] Example 1: If step count data is low and stress levels are high, generate advice such as "Walk a little more today. Take time to relax."

[0300] Example 2: If your calorie intake is high, generate advice like "Increase your vegetables at your next meal. Eat healthily to keep feeling great today."

[0301] Sending and viewing advice

[0302] The advice generated by the server is sent back to the device, which then notifies the user of the received advice and displays it on the display or app:

[0303] Example 1: The notification "Your heart rate tends to be high. Take time to relax" appears on the display.

[0304] Example 2: The app displays a notification saying, "You've consumed a lot of calories, so try eating a lighter meal next time."

[0305] The system allows users to receive specific advice based on their health and emotional data to improve their lifestyle habits.

[0306] Specific examples

[0307] As a concrete example, consider the following scenario:

[0308] Scenario 1: Collecting step count data and emotion data, giving advice

[0309] On the device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps, the emotion engine detects that the user's stress level is high.

[0310] Terminal: Sends collected data to the server.

[0311] Server: Analyzes data and detects lack of exercise and stress.

[0312] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[0313] Server: Sends the generated advice to the device.

[0314] Terminal: Notify the user.

[0315] Scenario 2: Calorie management based on food image analysis and emotional state

[0316] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[0317] Device: Sends food images and emotion data to the server.

[0318] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0319] Server: Considers emotional data, determines that your calorie intake is high, and generates specific advice such as, "Increase your vegetables at your next meal. Eat healthily to maintain your great mood today."

[0320] Server: Sends the generated advice to the device.

[0321] Terminal: Notify the user.

[0322] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] Step 1: Data collection

[0325] The device uses the following sensors to collect health and emotional data. Specifically, the device uses a pedometer to record the number of steps taken each day, a heart rate sensor to measure the heart rate, and a sleep sensor to record the amount of sleep. It also uses a camera to take photos of meals, and an emotion engine to analyze the user's facial expressions and voice to determine their emotional state for the day. For example, if a user records 5,000 steps, an average heart rate of 70 bpm, and seven hours of sleep, and then takes a photo of the meal, the device determines their emotional state as "high happiness."

[0326] Input: Raw data collected by sensors (number of steps, heart rate, sleep time, meal photos, facial expression and voice data)

[0327] Output: Collected health and emotional data (e.g., steps = 5000, heart rate = 70 bpm, sleep time = 7 hours, emotional state = high happiness)

[0328] Examples of using prompt statements

[0329] "Collect user health and emotional data."

[0330] Step 2: Send data

[0331] The device sends the collected health and emotional data to the server. Specifically, the device sends the collected data to the server every 15 minutes using encryption technology (e.g., TLS / SSL). For example, the number of steps taken (5,000), heart rate (70 bpm), sleep time (7 hours), emotional state ("high happiness"), and meal photos are sent to the server.

[0332] Input: Collected health and emotion data

[0333] Output: Data sent to the server

[0334] Examples of using prompt statements

[0335] "Please send the collected health and emotion data to the server."

[0336] Step 3: Data analysis

[0337] The server analyzes the received data. Specifically, it first analyzes the step count data and evaluates the amount of exercise. It also uses image recognition technology to calculate calorie intake from meal photos and takes into account the emotional data provided by the emotion engine. For example, if the number of steps taken in a day is less than 3,000, it determines that the person is not exercising enough, identifies the calorie intake of the meal as 800 kcal from a photo of lunch, and determines the stress level as high from the emotional data.

[0338] Input: Health and emotion data sent to the server

[0339] Output: Analysis results (lack of exercise, calorie intake 800 kcal, high stress level)

[0340] Examples of using prompt statements

[0341] "Analyze the data received and assess your health and emotional state."

[0342] Step 4: Advice Generation

[0343] The server uses the generative AI model to generate personalized health advice based on the analysis results. Specifically, it generates advice such as "You should walk a little more today" and "Take time to relax" based on the analysis results. If the calorie intake is high, it also generates specific advice such as "Increase the amount of vegetables at your next meal."

[0344] Input: Analysis results

[0345] Output: Personalized health advice (e.g., "You should walk a little more today" or "Take time to relax")

[0346] Examples of using prompt statements

[0347] "Based on the analysis results, generate appropriate health advice for the user."

[0348] Step 5: Submitting and viewing advice

[0349] The server sends the generated advice to the device. The device notifies the user of the received advice and displays it on a display or in an app. For example, a notification saying "Your heart rate tends to be high. Take time to relax" may appear on the display. Another notification saying "Your calorie intake is high, so have a lighter meal next time" may appear in the app.

[0350] Input: Generated advice

[0351] Output: Advice given to the user

[0352] Examples of using prompt statements

[0353] Send the generated advice to the device and notify the user.

[0354] (Application example 2)

[0355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Conventional health management systems collect health and emotional data and provide users with health advice, but they are unable to provide specific improvement suggestions based on users' spending patterns and lifestyle habits. Therefore, there is a need for a system that supports users in improving their lifestyle habits through healthy consumption behavior.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0358] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data and emotional data to the server, a generation AI means for analyzing the transmitted health data and emotional data and generating personalized advice, a communication means for transmitting the generated advice to a user terminal, a display means for notifying the user of the advice to be displayed on the user terminal, a data collection and analysis means for collecting and analyzing expenditure data, and a generation AI model means for providing personalized advice to promote healthy consumption based on the health data and expenditure data. This enables comprehensive health management and improvement of consumption behavior based on the user's health and emotional state and spending patterns.

[0359] "Sensor means" refers to devices for collecting personal health data, including step sensors, heart rate sensors, sleep sensors, cameras, etc.

[0360] "Communication means" refers to the technology used to transmit collected data to the server. This includes wireless communication technologies such as Wi-Fi and Bluetooth.

[0361] "Generative AI methods" are artificial intelligence technologies used to analyze submitted data and generate personalized advice, including natural language processing and machine learning algorithms.

[0362] The "display means" is a function for displaying the generated advice on the user terminal, such as the screen of a smartphone or the display of a smartwatch.

[0363] "Data collection and analysis means" refers to technology for collecting and analyzing user spending data, including electronic payment systems and purchase history management systems.

[0364] "Generative AI modeling tools" are artificial intelligence technologies, including machine learning models and generative adversarial networks (GANs), that provide personalized advice based on health and expenditure data.

[0365] This invention is a system that supports users in managing their health and improving their consumption behavior. This system collects personal health and emotional data and provides personalized health advice based on the analysis results. It also collects and analyzes the user's expenditure data and generates advice to promote healthy consumption behavior.

[0366] Hardware and software used

[0367] Sensor devices: Health data is collected through wearable devices such as the Apple Watch and Fitbit, which incorporate step, heart rate, and sleep sensors.

[0368] Communication technology: Data is transmitted using wireless communication technologies such as Wi-Fi and Bluetooth, which allows for smooth data transfer from the device to the server.

[0369] Server: AWS Lambda and AWS RDS are used to analyze and store data. AWS S3 is used to temporarily store and encrypt data.

[0370] Generative AI model: OpenAI's GPT-3 is used to generate personalized health advice, and Affectiva SDK is used to analyze emotion data.

[0371] System processing overview

[0372] The device collects personal health data (step count, heart rate, sleep data) and emotional data. Emotional data is extracted from the user's facial expressions and voice using the Affectiva SDK. This data is sent to a server at regular intervals. The sent data is stored in AWS S3 via AWS Lambda. The stored data is then integrated into a database using AWS RDS.

[0373] The server analyzes the health and emotional data and generates personalized health advice using a generative AI model (GPT-3). For example, if a user's step count is low and emotional data indicates a high stress level, the server generates advice such as, "You should walk a little more today. Take time to relax."

[0374] Spending data will also be collected and integrated with health data for analysis. Purchase history collected through electronic payment systems will be used to analyze consumer behavior. This will generate specific, healthy spending advice, such as, "Your food expenses have tended to be high this month. Consider making healthy choices the next time you buy food."

[0375] The generated advice is sent to the user's device in real time, allowing the user to improve their health and consumption behavior in their daily lives.

[0376] Specific examples

[0377] For example, if a user only walked 3,000 steps on a particular day and the emotion engine detects that the user's stress level is high, the following example prompts will be fed into the generative AI model:

[0378] Example prompt sentence:

[0379] Health data: Steps: 3000, Heart rate: 80 / min, Sleep time: 6 hours

[0380] Emotional data: Stress level: High, Mood: Unstable

[0381] Generation advice: With your step count low and stress levels high, take time to relax and take a short walk to refresh your mind.

[0382] In this way, the generative AI model takes into account the user's health and emotional state to provide appropriate advice. Furthermore, the user's spending data is also analyzed, and specific instructions are given to promote a healthy lifestyle. For example, advice such as "This month's food expenses are above average. Next time, choose a menu that includes lots of vegetables" is provided.

[0383] By using this system, users can improve their overall health management and consumption habits, leading to a healthier lifestyle.

[0384] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0385] Step 1:

[0386] The device collects the user's health data (step count, heart rate, sleep data) and emotional data. The sensor device (e.g., wearable device) used measures the user's biometric data in real time. The input is the user's biometric data, and the output is the collected health data and emotional data.

[0387] Step 2:

[0388] The device sends the collected health and emotion data to a server via Wi-Fi or Bluetooth. The input is the collected data, and the output is the encrypted data sent to the server.

[0389] Step 3:

[0390] The server uses AWS Lambda to temporarily store the submitted data in AWS S3, where the input is the data submitted to the server and the output is the data stored in AWS S3.

[0391] Step 4:

[0392] The server uses AWS RDS to integrate data stored in S3 into a database, where the input is data stored in AWS S3 and the output is the integrated data in the database.

[0393] Step 5:

[0394] The server uses AWS Lambda to analyze the health and emotional data. The analysis is performed using the Affectiva SDK. The input is the data integrated into the database, and the output is the analyzed results of the user's health and emotional state.

[0395] Step 6:

[0396] The server uses a generative AI model (GPT-3) to generate personalized health advice based on the analysis results, where the input is the health and emotional state, and the output is the generated personalized health advice.

[0397] Step 7:

[0398] The server stores the generated individual health advice in AWS RDS and simultaneously sends it to the user's device. The input here is the generated advice, and the output is a notification of the advice to the device.

[0399] Step 8:

[0400] The user terminal notifies the user of the generated advice and displays it on the display. The input here is the advice sent from the server, and the output is the advice displayed to the user.

[0401] Step 9:

[0402] The terminal collects the user's spending data from the electronic payment system and sends it to the server. At this stage, the input is the user's purchase history and the output is the collected spending data.

[0403] Step 10:

[0404] The server analyzes the expenditure data and integrates it with the health data for analysis, where the inputs are expenditure data and health data, and the output is the integrated analysis results.

[0405] Step 11:

[0406] Based on the integrated analysis results, the server generates advice to promote healthy spending behaviors using a generative AI model, where the input is the integrated analysis results and the output is the generated spending advice.

[0407] Step 12:

[0408] The server stores the generated spending advice in AWS RDS and sends it to the user's device. The input here is the generated spending advice, and the output is a notification of the advice to the device.

[0409] Step 13:

[0410] The user terminal notifies the generated spending advice to the user and displays it on a display, where the input is the spending advice sent from the server and the output is the advice displayed to the user.

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0412] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0414] [Second embodiment]

[0415] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0416] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0417] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0419] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0421] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0422] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0423] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0426] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0427] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal.

[0428] Overall system overview

[0429] This system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice. The advice is then sent to the terminal and notified to the user.

[0430] Data collection

[0431] The device is equipped with a step sensor, heart rate sensor, sleep sensor, etc., and collects the user's step count, heart rate, sleep time, etc. in real time. It also has a built-in camera, so it can collect meal data by allowing the user to take photos of their meals.

[0432] Data transmission

[0433] The device periodically sends the collected health and dietary data to a server, and the transmission is encrypted to ensure security.

[0434] Data analysis and generative AI

[0435] The server analyzes the received data and generates personalized health advice using AI. Specifically, if the step count data is low, it will point out a lack of exercise and advise, "You should walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake.

[0436] Sending and viewing advice

[0437] The advice generated by the server is delivered to the device. The device notifies the user of the advice and displays it on a display or app. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0438] Specific examples

[0439] Scenario 1: Step data collection and advice

[0440] Device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, such as 3,000.

[0441] Device: Sends this step count data to the server.

[0442] Server: Analyzes the received step count data and determines that the amount of exercise is insufficient.

[0443] Server: Using generative AI, generate advice such as "You should walk a little more today."

[0444] Server: Sends the generated advice to the terminal.

[0445] Terminal: Notify the user.

[0446] Scenario 2: Food image analysis and calorie management

[0447] Device: The user takes a photo of the food they had for lunch.

[0448] Device: Sends this food image to the server.

[0449] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0450] Server: Determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0451] Server: Sends the generated advice to the terminal.

[0452] Terminal: Notify the user.

[0453] In this way, this system provides comprehensive support for the user's health through a series of processes, from data collection by sensors to data analysis, and the generation and notification of individualized advice. Users can easily understand their health status in their daily lives and improve their lifestyle habits based on the advice.

[0454] The processing flow will be explained below.

[0455] Step 1:

[0456] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, meal times, etc.) in real time. This includes a step sensor, heart rate sensor, sleep sensor, and camera. During this time, the device periodically reads the sensor values ​​and records them in its internal memory.

[0457] Step 2:

[0458] User: When eating, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. At this time, a confirmation message for the photo-taking procedure and saving is displayed, allowing the user to check the contents of the photo.

[0459] Step 3:

[0460] Device: The collected health data and meal images are sent to the server at regular intervals. All information is encrypted before transmission to ensure communication security.

[0461] Step 4:

[0462] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. The analysis includes the process of evaluating data such as the number of steps, heart rate, and sleep time over time.

[0463] Step 5:

[0464] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. In this process, an image analysis model recognizes the contents of the meal and compares it with a general calorie database to estimate the calories.

[0465] Step 6:

[0466] Server: Using generation AI, it generates personalized health advice based on the analysis results. For example, if the number of steps is low, it generates advice such as "You should walk a little more today," and if the calorie intake is high, it generates advice such as "You should eat more vegetables at your next meal."

[0467] Step 7:

[0468] Server: The server sends the generated advice to the device. At this time, the advice is customized for each user and organized in an appropriate format. The content of the advice is concise and organized in a format that is easy for the user to understand.

[0469] Step 8:

[0470] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0471] Step 9:

[0472] User: Improves lifestyle habits based on the advice received, for example, taking an extra walk after receiving a notification or adding more vegetables to their next meal.

[0473] Through these steps, the system provides comprehensive support for users in maintaining their health and offers specific, practical advice.

[0474] Example 1

[0475] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0476] Conventional health management systems have had difficulty effectively analyzing collected health data and providing personalized health advice. They also lacked the functionality to analyze users' dietary data in detail and accurately calculate calorie intake. Furthermore, they lacked mechanisms for notifying users of generated advice in real time. For these reasons, there was a need for the development of a system that could comprehensively support users' health management.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0478] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data to the server, a generation AI means for analyzing the transmitted health data and generating personalized advice, a generation AI model means for inputting a prompt sentence into the generation AI model and generating health advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. This makes it possible to analyze the user's health data and dietary data in detail and quickly and effectively generate and notify personalized health advice.

[0479] "Sensor means" refers to a device used to collect personal health data, and includes a step sensor, a heart rate sensor, a sleep sensor, etc.

[0480] "Communication means" refers to the devices and technologies used to transmit collected data to the server, including wireless communication modules and internet connections.

[0481] "Generative AI Means" are the artificial intelligence algorithms and systems used to analyze the submitted data and generate personalized advice.

[0482] A "generative AI model means" is an artificial intelligence model used to generate health advice in response to a prompt statement.

[0483] "Display means" refers to the devices or techniques used to notify advice that is displayed on a user terminal, including displays and notification systems.

[0484] "Image recognition means" is a technology used to analyze collected dietary data and calculate calorie intake.

[0485] "Data storage means" refers to a system used to store and manage collected health and dietary data over the long term and provide ongoing health management advice.

[0486] This invention is a system that provides comprehensive support for the health of users in their daily lives. The system consists of a wearable device (hereinafter referred to as a "terminal") worn by the user and a server that works in conjunction with the device.

[0487] Data collection

[0488] Device: The device is equipped with various sensors, such as a step sensor, heart rate sensor, and sleep sensor, and collects health data such as the user's number of steps, heart rate, and sleep time in real time.The device also has a camera, which collects dietary data by allowing the user to take photos of their meals.

[0489] Data transmission

[0490] Device: All collected health and dietary data is encrypted and sent to a server at regular intervals using encryption technology such as AES-256. Transmission is via Wi-Fi or mobile data.

[0491] Data analysis and generative AI

[0492] Server: The server stores the received data in a database, and uses an analysis module to detect patterns and outliers in the data. It then inputs prompts into a generative AI model to generate health advice.

[0493] An example of a specific prompt: "Generate advice for the user on days when their step count data is low."

[0494] For example, the generative AI model might generate advice such as, "You should walk a little more today. Why not try a walk in a nearby park?"

[0495] Generative AI model: The generative AI model used utilizes the latest natural language processing techniques to generate detailed and relevant advice based on the prompt.

[0496] Sending and viewing advice

[0497] Server: The generated advice is encrypted and sent to the terminal.

[0498] Device: Interprets the received advice and notifies the user. Notification methods include push notifications and banner displays. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed on the screen.

[0499] Specific examples

[0500] Step 1: Collecting step data and giving advice

[0501] The device records the user's steps throughout the day, and if the user takes only 3,000 steps in a day, the data is stored on the device.

[0502] The terminal transmits this step count data to the server.

[0503] The server analyzes the received step count data and determines that the amount of exercise is insufficient.

[0504] The server uses a generative AI to generate advice such as, "You should walk a little more today."

[0505] The server transmits the generated advice to the terminal.

[0506] The terminal notifies the user and displays advice on the display.

[0507] Step 2: Food image analysis and calorie management

[0508] The terminal takes a photo of the food the user had for lunch.

[0509] The device sends this food image to the server.

[0510] The server analyzes the food image and calculates the calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0511] The server determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0512] The server transmits the generated advice to the terminal.

[0513] The terminal notifies the user and displays advice on the display.

[0514] As described above, the present invention combines various technologies such as sensor means, communication means, and generation AI means to grasp a user's health condition in real time and provide personalized health advice, allowing users to easily manage their health condition in their daily lives and continuously improve their lifestyle habits.

[0515] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0516] Step 1: Data collection

[0517] Device: Using an activity tracker, heart rate sensor, and sleep sensor, the device collects real-time health data (number of steps, heart rate, sleep time, etc.) from the user's daily life. For example, a smartwatch measures the number of steps a user takes in a day and stores the data on the device when they reach 10,000 steps.

[0518] Device: The user takes a photo of their lunch using their smartphone camera to collect meal data. The device stores the captured image in its internal storage.

[0519] Input: Raw data from sensors and food photos.

[0520] Output: Health and dietary data stored on the device.

[0521] Step 2: Send data

[0522] Terminal: All collected data is sent to the server at regular intervals. Before being sent, the data is securely protected using AES-256 encryption technology.

[0523] On your device: Using Wi-Fi or mobile data, the encrypted data is sent to a server.

[0524] Input: Health and dietary data stored on the device.

[0525] Output: The encrypted data sent to the server.

[0526] Step 3: Receiving and storing data

[0527] Server: Decrypts the received encrypted data and stores it in a database. The database uses a storage system that allows for quick search and analysis.

[0528] Input: Encrypted data.

[0529] Output: Decrypted data stored in a database.

[0530] Step 4: Data analysis

[0531] Server: Passes the data stored in the database to the analysis module, which detects patterns in the step count, heart rate, and sleep data and identifies outliers, such as a low current step count compared to the average step count over the past week.

[0532] Input: Health data from the database.

[0533] Output: Analysis results (e.g., information indicating low step count).

[0534] Step 5: Generative AI model generates advice

[0535] Server: Based on the analysis results, input a prompt to the generative AI model. For example, "Please generate advice for days when the user's step count data is low."

[0536] Generative AI model: Generates detailed and relevant health advice based on prompts. Generative AI takes into account context and past data to provide optimal advice.

[0537] Input: prompt statement, analysis result.

[0538] Output: Health advice (e.g., "You should walk a little more today").

[0539] Step 6: Send advice

[0540] Server: Encrypts the generated advice and sends it to the terminal.

[0541] Input: The generated advice.

[0542] Output: Encrypted advice data to the terminal.

[0543] Step 7: Display Advice

[0544] Device: Interprets the received advice and notifies the user. This can be done using push notifications or in-app banners. For example, the device might display advice such as "Your heart rate tends to be high. Take some time to relax."

[0545] Input: Encrypted advice data sent by the server.

[0546] Output: Displaying information and advice messages to the user.

[0547] (Application example 1)

[0548] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0549] In modern society, personal health management is an important issue. In particular, lack of exercise and improper diet often have negative effects on health, so there is a need for effective means of managing these. Furthermore, when users use food delivery services, there is a lack of functionality that provides optimal meal options based on their health status. For this reason, a system is needed that can collect and analyze individual health data in real time and provide appropriate advice to comprehensively support users' health management.

[0550] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0551] In this invention, the server includes sensor means for collecting personal health data, communication means for transmitting the collected health data to the server, generation AI means for analyzing the transmitted health data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, display means for notifying the user of the advice to be displayed on the user terminal, and suggestion means for suggesting optimal meal options based on the user's health data. This allows users to understand their own health status in real time and enables them to make health-conscious meal choices even when using food delivery services.

[0552] The "sensor means" is a device for collecting personal health data, and has the function of detecting the number of steps taken, heart rate, sleep time, etc. in real time.

[0553] "Communication means" refers to a device or system that has the function of transmitting collected health data to a server, and is capable of encrypting data communication.

[0554] "Generative AI means" refers to a device or system that uses artificial intelligence technology to analyze health data sent to the server and generate personalized health advice.

[0555] The "display means" refers to a display or application that notifies and displays advice sent from the server on the user terminal.

[0556] "Dietary data" is information about the meals consumed by the user, and includes photos of the meals and manually entered lists of ingredients.

[0557] The "image recognition means" is a device or system that has the function of analyzing collected dietary data and calculating calorie intake.

[0558] "Data storage means" refers to a device or system that has the function of storing and accumulating collected health data and calorie intake data over the long term.

[0559] The "suggestion means" is a device or system for suggesting optimal meal options based on the user's health data, and has the function of assisting in meal selection based on the generated advice.

[0560] Overall system overview

[0561] This invention is a system that comprehensively supports personal health management. The system includes a sensor means for collecting health data, a communication means for transmitting the data to a server, an AI generation means for analyzing the data and generating personalized advice, a means for transmitting the advice to a terminal and displaying it, and a suggestion means for suggesting optimal meal options based on the user's health data. The system works in conjunction with a wearable device to collect the user's health data in real time. This data is analyzed by the server and provided to the user as advice.

[0562] Hardware and software used

[0563] Wearable device: Equipped with step sensors, heart rate sensors, and sleep sensors.

[0564] Server: Uses data analysis and generative AI models (e.g., GPT-4).

[0565] User device: Advice is displayed and notified using a display or application.

[0566] Data processing and calculation

[0567] 1. Health Data Collection:

[0568] Wearable devices collect real-time health data such as the user's steps, heart rate, and sleep duration.

[0569] The device sends the collected data to a server at regular intervals, and the data is encrypted for security purposes.

[0570] 2. Data transmission:

[0571] The collected health data is transmitted to a server via a communication means and analyzed by the server.

[0572] 3. Data analysis and advice generation:

[0573] The server analyzes the received health data and generates personalized health advice using a generative AI model (e.g., GPT-4).

[0574] For example, if the number of steps taken is low, the system will generate advice such as, "You should walk a little more today." It will also analyze photos of meals, calculate calorie intake, and provide advice on calorie management.

[0575] 4. Send and display advice to user device:

[0576] The advice generated by the server is transmitted to the user terminal via a communication means.

[0577] The user terminal notifies the user of the advice and displays it in an application or on a display.

[0578] 5. Suggestions based on health advice:

[0579] The suggestion tool provides optimal meal options based on the user's health data, for example, healthy meal options if the user is not getting enough exercise.

[0580] Specific examples

[0581] Scenario 1: Step data collection and advice

[0582] On the device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, say 3,000.

[0583] Server: Analyzes the received step count data, determines that the amount of exercise is insufficient, and generates advice such as, "You should walk a little more today."

[0584] Terminal: The generated advice is notified to the user.

[0585] Scenario 2: Food image analysis and calorie management

[0586] Device: The user takes a photo of the food they had for lunch.

[0587] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal, it generates specific advice such as "Increase the amount of vegetables at your next meal."

[0588] Terminal: The generated advice is notified to the user.

[0589] Prompt Sentence Examples

[0590] User health data: 3000 steps (daily), heart rate 80 (average), sleep time 6 hours (average)

[0591] User's recent meals: pizza, hamburger

[0592] Prompt to spawn AI:

[0593] "The user is not very active, so please suggest a lower calorie option for their next meal."

[0594] This allows users to understand their health status in real time and choose healthy meals when using food delivery services.In addition, by using the suggestion method, optimal advice based on the user's health data is provided, allowing for efficient health management.

[0595] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0596] Step 1:

[0597] A user wears a wearable device while going about their daily life. This device is equipped with a step sensor, a heart rate sensor, and a sleep sensor, and has the function of collecting health data such as the user's steps, heart rate, and sleep time in real time. The input of the data collection is the user's physical activity, and the output is the collected health data.

[0598] Step 2:

[0599] The device sends the collected health data to the server at regular intervals. The data is encrypted before transmission to ensure security. The input is the collected health data, and the output is the transmission of the encrypted health data to the server. This ensures that the user's health data reaches the server safely.

[0600] Step 3:

[0601] The server analyzes the received health data. The input is the received health data, which is analyzed using a generation AI method. Specific operations of data analysis include pattern recognition, such as low step counts and high heart rate. Based on the results of this analysis, optimal health advice is generated for each user. The output is the generated health advice.

[0602] Step 4:

[0603] The server sends the generated health advice to the user terminal. The input is the generated health advice, and the output is the transmission of the advice to the user terminal. The data is encrypted and transferred securely by the communication means.

[0604] Step 5:

[0605] The device receives the advice sent from the server and notifies the user through a display or application. The input is the health advice sent from the server, and the output is the content to be notified to the user. Specific actions can include a pop-up notification in the application or an email notification.

[0606] Step 6:

[0607] A user launches a food delivery app. The app references the health data and advice previously received from the device. The input is the user's health data and generated advice, and the output is the health-conscious meal options displayed on the food delivery menu screen.

[0608] Step 7:

[0609] The suggestion mechanism suggests optimal meal options based on the user's health data. For example, if the user is not exercising enough, it suggests "salads" or "low-calorie dishes." The generative AI model then uses a prompt such as "The user is not exercising much, so please suggest low-calorie options" to make optimal suggestions. The input is the user's health data and the prompt, and the output is a suggestion of a healthy meal option.

[0610] This will enable the entire system to consistently handle everything from data collection to advice suggestions and meal selection, providing comprehensive support for users' health management.

[0611] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0612] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. Furthermore, an emotion engine that recognizes the user's emotions is also incorporated.

[0613] Overall system overview

[0614] This system consists of a wearable device (hereafter referred to as the terminal) and a server linked to it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates individualized advice that takes the emotional data into account. The generated advice is then sent to the terminal and notified to the user.

[0615] Data collection

[0616] The device is equipped with a step sensor, heart rate sensor, sleep sensor, camera, and emotion engine. These sensors collect the user's health data (step count, heart rate, sleep time, meal time) and emotion data in real time. The emotion engine analyzes the user's facial expressions and voice to determine their mood and emotional state for the day.

[0617] Data transmission

[0618] The device periodically transmits the collected health data, food images, and emotion data to a server, and the transmission is encrypted to ensure security.

[0619] Data analysis and generative AI

[0620] The server analyzes the received data and uses AI generation to generate personalized health advice. Specifically, if the step count data is low, it will point out a lack of exercise and advise the user to "walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake. Furthermore, it adjusts the advice based on emotional data. For example, if the emotion engine detects the user's stress level, it will add the advice "Take time to relax."

[0621] Sending and viewing advice

[0622] The advice generated by the server is sent to the device. The device notifies the user of the advice and displays it on a display or in an app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" may be displayed.

[0623] Specific examples

[0624] Scenario 1: Collecting step count data and emotion data, giving advice

[0625] Device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps today, the emotion engine detects that the user's stress level is high.

[0626] Terminal: Sends collected step count data and emotion data to the server.

[0627] Server: Analyzes data and detects lack of exercise and stress.

[0628] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[0629] Server: Sends the generated advice to the terminal.

[0630] Terminal: Notify the user.

[0631] Scenario 2: Calorie management based on food image analysis and emotional state

[0632] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[0633] Device: Sends food images and emotion data to the server.

[0634] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0635] Server: Considers emotional data, determines that the calorie intake is high, and generates specific advice such as, "Increase the amount of vegetables at your next meal. Eat healthily to maintain your great mood today."

[0636] Server: Sends the generated advice to the terminal.

[0637] Terminal: Notify the user.

[0638] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[0639] The processing flow will be explained below.

[0640] Step 1:

[0641] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, etc.) in real time. This includes a step sensor, heart rate sensor, and sleep sensor. The device periodically records the data obtained from these sensors in its internal memory.

[0642] Step 2:

[0643] User: When eating a meal, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. The user is prompted to confirm that they want to save the photo.

[0644] Step 3:

[0645] On the device: The emotion engine collects emotion data from the user's facial expressions and voice. For example, it uses a facial recognition algorithm to analyze the user's facial expressions and determine their emotional state, such as joy, sadness, or anger.

[0646] Step 4:

[0647] Device: The device periodically transmits collected health data, food images, and emotion data to the server. The transmitted data is encrypted to ensure communication security.

[0648] Step 5:

[0649] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. Analysis includes time-series evaluation of health data such as number of steps, heart rate, and sleep time, calorie calculation of meals, and analysis of emotional data.

[0650] Step 6:

[0651] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. The food photos are analyzed to determine the type and amount of food, and the calorie intake is estimated.

[0652] Step 7:

[0653] Server: Using the generative AI, it generates personalized health advice based on the analysis results. For example, if the analysis results indicate a lack of exercise, it generates the advice "You should walk a little more today." If the emotion engine determines that the user is feeling stressed, it adds the advice "Take time to relax."

[0654] Step 8:

[0655] Server: Sends the generated advice to the device. The advice is customized for each user and organized in an appropriate format.

[0656] Step 9:

[0657] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app, and the user can check the content. For example, a notification may be displayed saying, "Your heart rate tends to be high. Take time to relax."

[0658] Step 10:

[0659] User: Improves lifestyle habits based on the advice received. For example, taking an extra walk or adding more vegetables to their next meal. At this stage, users can understand their own health and emotional state and optimize their behavior based on high-quality advice.

[0660] Through this series of processing steps, users receive specific advice tailored to their individual health and emotional state, enabling them to effectively manage their own health in their daily lives.

[0661] Example 2

[0662] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0663] In modern society, busy lifestyles and irregular eating habits make it difficult to manage personal health. In addition to simple health data, emotional states are also important factors in comprehensive health management. However, current systems struggle to effectively collect and analyze this data and provide users with personalized advice. Furthermore, there is a lack of systems that can manage and analyze health and emotional data together and provide users with specific behavioral guidance.

[0664] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to the server, and generation AI means for analyzing the transmitted health data and emotional data and generating individual advice. This makes it possible to provide comprehensive and personalized health management based on the user's health condition and emotional state.

[0665] "Sensor means" refers to devices or mechanisms for collecting health and emotional data of an individual.

[0666] "Communication means" refers to the network connection capability for transmitting collected health and emotion data to a server.

[0667] "Generative AI means" refers to the artificial intelligence models and algorithms used to analyze submitted data and generate personalized advice.

[0668] "Display means" refers to a display or application function for notifying advice displayed on a user terminal.

[0669] "Image recognition means" refers to image recognition technology and software used to analyze collected dietary data and calculate calorie intake.

[0670] "Data storage means" refers to a database or storage system for long-term storage and management of collected health data, emotional data, and calorie intake data.

[0671] This invention provides a system for supporting a user's health management. The system includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to a server, AI generation means for analyzing the transmitted health data and emotional data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, and display means for displaying the advice on the user terminal.

[0672] System Configuration

[0673] The entire system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice that takes the emotional data into account. The advice is then sent to the terminal and notified to the user.

[0674] Data collection

[0675] The device is equipped with the following sensors:

[0676] Pedometer: Records the number of steps taken by the user. Example: Records 5,000 steps in a day.

[0677] Heart Rate Sensor: Measures the user's heart rate. Example: Heart rate is 70 bpm on average.

[0678] Sleep sensor: Records the user's sleep time. Example: The previous night's sleep time was 7 hours.

[0679] Camera: Takes a photo of the user's meal. Example: Takes a photo of lunch.

[0680] Emotion engine: Analyzes the user's facial expressions and voice to determine their emotional state. Example: Detects that the user is smiling and determines that they are feeling happy.

[0681] The emotion engine is equipped with advanced analytical technology and can identify the user's mood and stress level for the day through facial expression and voice analysis.

[0682] Data transmission

[0683] The device sends the collected data to the server at regular intervals. This transmission is performed using encryption technology (e.g., TLS / SSL) to ensure security. The data is sent every 15 minutes, for example.

[0684] Data analysis and generative AI

[0685] The server uses advanced analysis algorithms and generative AI models (e.g., GPT-3) to analyze the received data.

[0686] Analysis of step count data: Analyze daily step count data and evaluate the amount of exercise. For example, if the number of steps taken in a day is less than 3,000, it is determined that the person is not exercising enough.

[0687] Food image analysis: Using image recognition technology (e.g., Google Cloud Vision API), calculate calorie intake from food photos. Example: Estimate 800 kcal from a photo of lunch.

[0688] Emotion data analysis: The emotion engine considers the analyzed emotion data. For example, the stress level is determined to be high.

[0689] Example prompt sentence:

[0690] "Please send the collected health and emotional data to the server." "Please analyze the received data and evaluate the health and emotional state." "Please generate appropriate health advice for the user based on the analysis results." "Please send the generated advice to the device and notify the user."

[0691] Advice Generation

[0692] Based on the analysis results, the server generates personalized health advice using a generative AI model:

[0693] Example 1: If step count data is low and stress levels are high, generate advice such as "Walk a little more today. Take time to relax."

[0694] Example 2: If your calorie intake is high, generate advice like "Increase your vegetables at your next meal. Eat healthily to keep feeling great today."

[0695] Sending and viewing advice

[0696] The advice generated by the server is sent back to the device, which then notifies the user of the received advice and displays it on the display or app:

[0697] Example 1: The notification "Your heart rate tends to be high. Take time to relax" appears on the display.

[0698] Example 2: The app displays a notification saying, "You've consumed a lot of calories, so try eating a lighter meal next time."

[0699] The system allows users to receive specific advice based on their health and emotional data to improve their lifestyle habits.

[0700] Specific examples

[0701] As a concrete example, consider the following scenario:

[0702] Scenario 1: Collecting step count data and emotion data, giving advice

[0703] On the device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps, the emotion engine detects that the user's stress level is high.

[0704] Terminal: Sends collected data to the server.

[0705] Server: Analyzes data and detects lack of exercise and stress.

[0706] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[0707] Server: Sends the generated advice to the device.

[0708] Terminal: Notify the user.

[0709] Scenario 2: Calorie management based on food image analysis and emotional state

[0710] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[0711] Device: Sends food images and emotion data to the server.

[0712] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0713] Server: Considers emotional data, determines that your calorie intake is high, and generates specific advice such as, "Increase your vegetables at your next meal. Eat healthily to maintain your great mood today."

[0714] Server: Sends the generated advice to the device.

[0715] Terminal: Notify the user.

[0716] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[0717] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0718] Step 1: Data collection

[0719] The device uses the following sensors to collect health and emotional data. Specifically, the device uses a pedometer to record the number of steps taken each day, a heart rate sensor to measure the heart rate, and a sleep sensor to record the amount of sleep. It also uses a camera to take photos of meals, and an emotion engine to analyze the user's facial expressions and voice to determine their emotional state for the day. For example, if a user records 5,000 steps, an average heart rate of 70 bpm, and seven hours of sleep, and then takes a photo of the meal, the device determines their emotional state as "high happiness."

[0720] Input: Raw data collected by sensors (number of steps, heart rate, sleep time, meal photos, facial expression and voice data)

[0721] Output: Collected health and emotional data (e.g., steps = 5000, heart rate = 70 bpm, sleep time = 7 hours, emotional state = high happiness)

[0722] Examples of using prompt statements

[0723] "Collect user health and emotional data."

[0724] Step 2: Send data

[0725] The device sends the collected health and emotional data to the server. Specifically, the device sends the collected data to the server every 15 minutes using encryption technology (e.g., TLS / SSL). For example, the number of steps taken (5,000), heart rate (70 bpm), sleep time (7 hours), emotional state ("high happiness"), and meal photos are sent to the server.

[0726] Input: Collected health and emotion data

[0727] Output: Data sent to the server

[0728] Examples of using prompt statements

[0729] "Please send the collected health and emotion data to the server."

[0730] Step 3: Data analysis

[0731] The server analyzes the received data. Specifically, it first analyzes the step count data and evaluates the amount of exercise. It also uses image recognition technology to calculate calorie intake from meal photos and takes into account the emotional data provided by the emotion engine. For example, if the number of steps taken in a day is less than 3,000, it determines that the person is not exercising enough, identifies the calorie intake of the meal as 800 kcal from a photo of lunch, and determines the stress level as high from the emotional data.

[0732] Input: Health and emotion data sent to the server

[0733] Output: Analysis results (lack of exercise, calorie intake 800 kcal, high stress level)

[0734] Examples of using prompt statements

[0735] "Analyze the data received and assess your health and emotional state."

[0736] Step 4: Advice Generation

[0737] The server uses the generative AI model to generate personalized health advice based on the analysis results. Specifically, it generates advice such as "You should walk a little more today" and "Take time to relax" based on the analysis results. If the calorie intake is high, it also generates specific advice such as "Increase the amount of vegetables at your next meal."

[0738] Input: Analysis results

[0739] Output: Personalized health advice (e.g., "You should walk a little more today" or "Take time to relax")

[0740] Examples of using prompt statements

[0741] "Based on the analysis results, generate appropriate health advice for the user."

[0742] Step 5: Submitting and viewing advice

[0743] The server sends the generated advice to the device. The device notifies the user of the received advice and displays it on a display or in an app. For example, a notification saying "Your heart rate tends to be high. Take time to relax" may appear on the display. Another notification saying "Your calorie intake is high, so have a lighter meal next time" may appear in the app.

[0744] Input: Generated advice

[0745] Output: Advice given to the user

[0746] Examples of using prompt statements

[0747] Send the generated advice to the device and notify the user.

[0748] (Application example 2)

[0749] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0750] Conventional health management systems collect health and emotional data and provide users with health advice, but they are unable to provide specific improvement suggestions based on users' spending patterns and lifestyle habits. Therefore, there is a need for a system that supports users in improving their lifestyle habits through healthy consumption behavior.

[0751] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0752] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data and emotional data to the server, a generation AI means for analyzing the transmitted health data and emotional data and generating personalized advice, a communication means for transmitting the generated advice to a user terminal, a display means for notifying the user of the advice to be displayed on the user terminal, a data collection and analysis means for collecting and analyzing expenditure data, and a generation AI model means for providing personalized advice to promote healthy consumption based on the health data and expenditure data. This enables comprehensive health management and improvement of consumption behavior based on the user's health and emotional state and spending patterns.

[0753] "Sensor means" refers to devices for collecting personal health data, including step sensors, heart rate sensors, sleep sensors, cameras, etc.

[0754] "Communication means" refers to the technology used to transmit collected data to the server. This includes wireless communication technologies such as Wi-Fi and Bluetooth.

[0755] "Generative AI methods" are artificial intelligence technologies used to analyze submitted data and generate personalized advice, including natural language processing and machine learning algorithms.

[0756] The "display means" is a function for displaying the generated advice on the user terminal, such as the screen of a smartphone or the display of a smartwatch.

[0757] "Data collection and analysis means" refers to technology for collecting and analyzing user spending data, including electronic payment systems and purchase history management systems.

[0758] "Generative AI modeling tools" are artificial intelligence technologies, including machine learning models and generative adversarial networks (GANs), that provide personalized advice based on health and expenditure data.

[0759] This invention is a system that supports users in managing their health and improving their consumption behavior. This system collects personal health and emotional data and provides personalized health advice based on the analysis results. It also collects and analyzes the user's expenditure data and generates advice to promote healthy consumption behavior.

[0760] Hardware and software used

[0761] Sensor devices: Health data is collected through wearable devices such as the Apple Watch and Fitbit, which incorporate step, heart rate, and sleep sensors.

[0762] Communication technology: Data is transmitted using wireless communication technologies such as Wi-Fi and Bluetooth, which allows for smooth data transfer from the device to the server.

[0763] Server: AWS Lambda and AWS RDS are used to analyze and store data. AWS S3 is used to temporarily store and encrypt data.

[0764] Generative AI model: OpenAI's GPT-3 is used to generate personalized health advice, and Affectiva SDK is used to analyze emotion data.

[0765] System processing overview

[0766] The device collects personal health data (step count, heart rate, sleep data) and emotional data. Emotional data is extracted from the user's facial expressions and voice using the Affectiva SDK. This data is sent to a server at regular intervals. The sent data is stored in AWS S3 via AWS Lambda. The stored data is then integrated into a database using AWS RDS.

[0767] The server analyzes the health and emotional data and generates personalized health advice using a generative AI model (GPT-3). For example, if a user's step count is low and emotional data indicates a high stress level, the server generates advice such as, "You should walk a little more today. Take time to relax."

[0768] Spending data will also be collected and integrated with health data for analysis. Purchase history collected through electronic payment systems will be used to analyze consumer behavior. This will generate specific, healthy spending advice, such as, "Your food expenses have tended to be high this month. Consider making healthy choices the next time you buy food."

[0769] The generated advice is sent to the user's device in real time, allowing the user to improve their health and consumption behavior in their daily lives.

[0770] Specific examples

[0771] For example, if a user only walked 3,000 steps on a particular day and the emotion engine detects that the user's stress level is high, the following example prompts will be fed into the generative AI model:

[0772] Example prompt sentence:

[0773] Health data: Steps: 3000, Heart rate: 80 / min, Sleep time: 6 hours

[0774] Emotional data: Stress level: High, Mood: Unstable

[0775] Generation advice: With your step count low and stress levels high, take time to relax and take a short walk to refresh your mind.

[0776] In this way, the generative AI model takes into account the user's health and emotional state to provide appropriate advice. Furthermore, the user's spending data is also analyzed, and specific instructions are given to promote a healthy lifestyle. For example, advice such as "This month's food expenses are above average. Next time, choose a menu that includes lots of vegetables" is provided.

[0777] By using this system, users can improve their overall health management and consumption habits, leading to a healthier lifestyle.

[0778] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0779] Step 1:

[0780] The device collects the user's health data (step count, heart rate, sleep data) and emotional data. The sensor device (e.g., wearable device) used measures the user's biometric data in real time. The input is the user's biometric data, and the output is the collected health data and emotional data.

[0781] Step 2:

[0782] The device sends the collected health and emotion data to a server via Wi-Fi or Bluetooth. The input is the collected data, and the output is the encrypted data sent to the server.

[0783] Step 3:

[0784] The server uses AWS Lambda to temporarily store the submitted data in AWS S3, where the input is the data submitted to the server and the output is the data stored in AWS S3.

[0785] Step 4:

[0786] The server uses AWS RDS to integrate data stored in S3 into a database, where the input is data stored in AWS S3 and the output is the integrated data in the database.

[0787] Step 5:

[0788] The server uses AWS Lambda to analyze the health and emotional data. The analysis is performed using the Affectiva SDK. The input is the data integrated into the database, and the output is the analyzed results of the user's health and emotional state.

[0789] Step 6:

[0790] The server uses a generative AI model (GPT-3) to generate personalized health advice based on the analysis results, where the input is the health and emotional state, and the output is the generated personalized health advice.

[0791] Step 7:

[0792] The server stores the generated individual health advice in AWS RDS and simultaneously sends it to the user's device. The input here is the generated advice, and the output is a notification of the advice to the device.

[0793] Step 8:

[0794] The user terminal notifies the user of the generated advice and displays it on the display. The input here is the advice sent from the server, and the output is the advice displayed to the user.

[0795] Step 9:

[0796] The terminal collects the user's spending data from the electronic payment system and sends it to the server. At this stage, the input is the user's purchase history and the output is the collected spending data.

[0797] Step 10:

[0798] The server analyzes the expenditure data and integrates it with the health data for analysis, where the inputs are expenditure data and health data, and the output is the integrated analysis results.

[0799] Step 11:

[0800] Based on the integrated analysis results, the server generates advice to promote healthy spending behaviors using a generative AI model, where the input is the integrated analysis results and the output is the generated spending advice.

[0801] Step 12:

[0802] The server stores the generated spending advice in AWS RDS and sends it to the user's device. The input here is the generated spending advice, and the output is a notification of the advice to the device.

[0803] Step 13:

[0804] The user terminal notifies the generated spending advice to the user and displays it on a display, where the input is the spending advice sent from the server and the output is the advice displayed to the user.

[0805] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0806] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0807] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0808] [Third embodiment]

[0809] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0810] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0811] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0812] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0813] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0814] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0815] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0816] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0817] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0818] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0819] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0820] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0821] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal.

[0822] Overall system overview

[0823] This system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice. The advice is then sent to the terminal and notified to the user.

[0824] Data collection

[0825] The device is equipped with a step sensor, heart rate sensor, sleep sensor, etc., and collects the user's step count, heart rate, sleep time, etc. in real time. It also has a built-in camera, so it can collect meal data by allowing the user to take photos of their meals.

[0826] Data transmission

[0827] The device periodically sends the collected health and dietary data to a server, and the transmission is encrypted to ensure security.

[0828] Data analysis and generative AI

[0829] The server analyzes the received data and generates personalized health advice using AI. Specifically, if the step count data is low, it will point out a lack of exercise and advise, "You should walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake.

[0830] Sending and viewing advice

[0831] The advice generated by the server is delivered to the device. The device notifies the user of the advice and displays it on a display or app. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0832] Specific examples

[0833] Scenario 1: Step data collection and advice

[0834] Device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, such as 3,000.

[0835] Device: Sends this step count data to the server.

[0836] Server: Analyzes the received step count data and determines that the amount of exercise is insufficient.

[0837] Server: Using generative AI, generate advice such as "You should walk a little more today."

[0838] Server: Sends the generated advice to the terminal.

[0839] Terminal: Notify the user.

[0840] Scenario 2: Food image analysis and calorie management

[0841] Device: The user takes a photo of the food they had for lunch.

[0842] Device: Sends this food image to the server.

[0843] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0844] Server: Determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0845] Server: Sends the generated advice to the terminal.

[0846] Terminal: Notify the user.

[0847] In this way, this system provides comprehensive support for the user's health through a series of processes, from data collection by sensors to data analysis, and the generation and notification of individualized advice. Users can easily understand their health status in their daily lives and improve their lifestyle habits based on the advice.

[0848] The processing flow will be explained below.

[0849] Step 1:

[0850] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, meal times, etc.) in real time. This includes a step sensor, heart rate sensor, sleep sensor, and camera. During this time, the device periodically reads the sensor values ​​and records them in its internal memory.

[0851] Step 2:

[0852] User: When eating, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. At this time, a confirmation message for the photo-taking procedure and saving is displayed, allowing the user to check the contents of the photo.

[0853] Step 3:

[0854] Device: The collected health data and meal images are sent to the server at regular intervals. All information is encrypted before transmission to ensure communication security.

[0855] Step 4:

[0856] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. The analysis includes the process of evaluating data such as the number of steps, heart rate, and sleep time over time.

[0857] Step 5:

[0858] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. In this process, an image analysis model recognizes the contents of the meal and compares it with a general calorie database to estimate the calories.

[0859] Step 6:

[0860] Server: Using generation AI, it generates personalized health advice based on the analysis results. For example, if the number of steps is low, it generates advice such as "You should walk a little more today," and if the calorie intake is high, it generates advice such as "You should eat more vegetables at your next meal."

[0861] Step 7:

[0862] Server: The server sends the generated advice to the device. At this time, the advice is customized for each user and organized in an appropriate format. The content of the advice is concise and organized in a format that is easy for the user to understand.

[0863] Step 8:

[0864] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" is displayed.

[0865] Step 9:

[0866] User: Improves lifestyle habits based on the advice received, for example, taking an extra walk after receiving a notification or adding more vegetables to their next meal.

[0867] Through these steps, the system provides comprehensive support for users in maintaining their health and offers specific, practical advice.

[0868] Example 1

[0869] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0870] Conventional health management systems have had difficulty effectively analyzing collected health data and providing personalized health advice. They also lacked the functionality to analyze users' dietary data in detail and accurately calculate calorie intake. Furthermore, they lacked mechanisms for notifying users of generated advice in real time. For these reasons, there was a need for the development of a system that could comprehensively support users' health management.

[0871] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0872] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data to the server, a generation AI means for analyzing the transmitted health data and generating personalized advice, a generation AI model means for inputting a prompt sentence into the generation AI model and generating health advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. This makes it possible to analyze the user's health data and dietary data in detail and quickly and effectively generate and notify personalized health advice.

[0873] "Sensor means" refers to a device used to collect personal health data, and includes a step sensor, a heart rate sensor, a sleep sensor, etc.

[0874] "Communication means" refers to the devices and technologies used to transmit collected data to the server, including wireless communication modules and internet connections.

[0875] "Generative AI Means" are the artificial intelligence algorithms and systems used to analyze the submitted data and generate personalized advice.

[0876] A "generative AI model means" is an artificial intelligence model used to generate health advice in response to a prompt statement.

[0877] "Display means" refers to the devices or techniques used to notify advice that is displayed on a user terminal, including displays and notification systems.

[0878] "Image recognition means" is a technology used to analyze collected dietary data and calculate calorie intake.

[0879] "Data storage means" refers to a system used to store and manage collected health and dietary data over the long term and provide ongoing health management advice.

[0880] This invention is a system that provides comprehensive support for the health of users in their daily lives. The system consists of a wearable device (hereinafter referred to as a "terminal") worn by the user and a server that works in conjunction with the device.

[0881] Data collection

[0882] Device: The device is equipped with various sensors, such as a step sensor, heart rate sensor, and sleep sensor, and collects health data such as the user's number of steps, heart rate, and sleep time in real time.The device also has a camera, which collects dietary data by allowing the user to take photos of their meals.

[0883] Data transmission

[0884] Device: All collected health and dietary data is encrypted and sent to a server at regular intervals using encryption technology such as AES-256. Transmission is via Wi-Fi or mobile data.

[0885] Data analysis and generative AI

[0886] Server: The server stores the received data in a database, and uses an analysis module to detect patterns and outliers in the data. It then inputs prompts into a generative AI model to generate health advice.

[0887] An example of a specific prompt: "Generate advice for the user on days when their step count data is low."

[0888] For example, the generative AI model might generate advice such as, "You should walk a little more today. Why not try a walk in a nearby park?"

[0889] Generative AI model: The generative AI model used utilizes the latest natural language processing techniques to generate detailed and relevant advice based on the prompt.

[0890] Sending and viewing advice

[0891] Server: The generated advice is encrypted and sent to the terminal.

[0892] Device: Interprets the received advice and notifies the user. Notification methods include push notifications and banner displays. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed on the screen.

[0893] Specific examples

[0894] Step 1: Collecting step data and giving advice

[0895] The device records the user's steps throughout the day, and if the user takes only 3,000 steps in a day, the data is stored on the device.

[0896] The terminal transmits this step count data to the server.

[0897] The server analyzes the received step count data and determines that the amount of exercise is insufficient.

[0898] The server uses a generative AI to generate advice such as, "You should walk a little more today."

[0899] The server transmits the generated advice to the terminal.

[0900] The terminal notifies the user and displays advice on the display.

[0901] Step 2: Food image analysis and calorie management

[0902] The terminal takes a photo of the food the user had for lunch.

[0903] The device sends this food image to the server.

[0904] The server analyzes the food image and calculates the calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[0905] The server determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[0906] The server transmits the generated advice to the terminal.

[0907] The terminal notifies the user and displays advice on the display.

[0908] As described above, the present invention combines various technologies such as sensor means, communication means, and generation AI means to grasp a user's health condition in real time and provide personalized health advice, allowing users to easily manage their health condition in their daily lives and continuously improve their lifestyle habits.

[0909] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0910] Step 1: Data collection

[0911] Device: Using an activity tracker, heart rate sensor, and sleep sensor, the device collects real-time health data (number of steps, heart rate, sleep time, etc.) from the user's daily life. For example, a smartwatch measures the number of steps a user takes in a day and stores the data on the device when they reach 10,000 steps.

[0912] Device: The user takes a photo of their lunch using their smartphone camera to collect meal data. The device stores the captured image in its internal storage.

[0913] Input: Raw data from sensors and food photos.

[0914] Output: Health and dietary data stored on the device.

[0915] Step 2: Send data

[0916] Terminal: All collected data is sent to the server at regular intervals. Before being sent, the data is securely protected using AES-256 encryption technology.

[0917] On your device: Using Wi-Fi or mobile data, the encrypted data is sent to a server.

[0918] Input: Health and dietary data stored on the device.

[0919] Output: The encrypted data sent to the server.

[0920] Step 3: Receiving and storing data

[0921] Server: Decrypts the received encrypted data and stores it in a database. The database uses a storage system that allows for quick search and analysis.

[0922] Input: Encrypted data.

[0923] Output: Decrypted data stored in a database.

[0924] Step 4: Data analysis

[0925] Server: Passes the data stored in the database to the analysis module, which detects patterns in the step count, heart rate, and sleep data and identifies outliers, such as a low current step count compared to the average step count over the past week.

[0926] Input: Health data from the database.

[0927] Output: Analysis results (e.g., information indicating low step count).

[0928] Step 5: Generative AI model generates advice

[0929] Server: Based on the analysis results, input a prompt to the generative AI model. For example, "Please generate advice for days when the user's step count data is low."

[0930] Generative AI model: Generates detailed and relevant health advice based on prompts. Generative AI takes into account context and past data to provide optimal advice.

[0931] Input: prompt statement, analysis result.

[0932] Output: Health advice (e.g., "You should walk a little more today").

[0933] Step 6: Send advice

[0934] Server: Encrypts the generated advice and sends it to the terminal.

[0935] Input: The generated advice.

[0936] Output: Encrypted advice data to the terminal.

[0937] Step 7: Display Advice

[0938] Device: Interprets the received advice and notifies the user. This can be done using push notifications or in-app banners. For example, the device might display advice such as "Your heart rate tends to be high. Take some time to relax."

[0939] Input: Encrypted advice data sent by the server.

[0940] Output: Displaying information and advice messages to the user.

[0941] (Application example 1)

[0942] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0943] In modern society, personal health management is an important issue. In particular, lack of exercise and improper diet often have negative effects on health, so there is a need for effective means of managing these. Furthermore, when users use food delivery services, there is a lack of functionality that provides optimal meal options based on their health status. For this reason, a system is needed that can collect and analyze individual health data in real time and provide appropriate advice to comprehensively support users' health management.

[0944] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0945] In this invention, the server includes sensor means for collecting personal health data, communication means for transmitting the collected health data to the server, generation AI means for analyzing the transmitted health data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, display means for notifying the user of the advice to be displayed on the user terminal, and suggestion means for suggesting optimal meal options based on the user's health data. This allows users to understand their own health status in real time and enables them to make health-conscious meal choices even when using food delivery services.

[0946] The "sensor means" is a device for collecting personal health data, and has the function of detecting the number of steps taken, heart rate, sleep time, etc. in real time.

[0947] "Communication means" refers to a device or system that has the function of transmitting collected health data to a server, and is capable of encrypting data communication.

[0948] "Generative AI means" refers to a device or system that uses artificial intelligence technology to analyze health data sent to the server and generate personalized health advice.

[0949] The "display means" refers to a display or application that notifies and displays advice sent from the server on the user terminal.

[0950] "Dietary data" is information about the meals consumed by the user, and includes photos of the meals and manually entered lists of ingredients.

[0951] The "image recognition means" is a device or system that has the function of analyzing collected dietary data and calculating calorie intake.

[0952] "Data storage means" refers to a device or system that has the function of storing and accumulating collected health data and calorie intake data over the long term.

[0953] The "suggestion means" is a device or system for suggesting optimal meal options based on the user's health data, and has the function of assisting in meal selection based on the generated advice.

[0954] Overall system overview

[0955] This invention is a system that comprehensively supports personal health management. The system includes a sensor means for collecting health data, a communication means for transmitting the data to a server, an AI generation means for analyzing the data and generating personalized advice, a means for transmitting the advice to a terminal and displaying it, and a suggestion means for suggesting optimal meal options based on the user's health data. The system works in conjunction with a wearable device to collect the user's health data in real time. This data is analyzed by the server and provided to the user as advice.

[0956] Hardware and software used

[0957] Wearable device: Equipped with step sensors, heart rate sensors, and sleep sensors.

[0958] Server: Uses data analysis and generative AI models (e.g., GPT-4).

[0959] User device: Advice is displayed and notified using a display or application.

[0960] Data processing and calculation

[0961] 1. Health Data Collection:

[0962] Wearable devices collect real-time health data such as the user's steps, heart rate, and sleep duration.

[0963] The device sends the collected data to a server at regular intervals, and the data is encrypted for security purposes.

[0964] 2. Data transmission:

[0965] The collected health data is transmitted to a server via a communication means and analyzed by the server.

[0966] 3. Data analysis and advice generation:

[0967] The server analyzes the received health data and generates personalized health advice using a generative AI model (e.g., GPT-4).

[0968] For example, if the number of steps taken is low, the system will generate advice such as, "You should walk a little more today." It will also analyze photos of meals, calculate calorie intake, and provide advice on calorie management.

[0969] 4. Send and display advice to user device:

[0970] The advice generated by the server is transmitted to the user terminal via a communication means.

[0971] The user terminal notifies the user of the advice and displays it in an application or on a display.

[0972] 5. Suggestions based on health advice:

[0973] The suggestion tool provides optimal meal options based on the user's health data, for example, healthy meal options if the user is not getting enough exercise.

[0974] Specific examples

[0975] Scenario 1: Step data collection and advice

[0976] On the device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, say 3,000.

[0977] Server: Analyzes the received step count data, determines that the amount of exercise is insufficient, and generates advice such as, "You should walk a little more today."

[0978] Terminal: The generated advice is notified to the user.

[0979] Scenario 2: Food image analysis and calorie management

[0980] Device: The user takes a photo of the food they had for lunch.

[0981] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal, it generates specific advice such as "Increase the amount of vegetables at your next meal."

[0982] Terminal: The generated advice is notified to the user.

[0983] Prompt Sentence Examples

[0984] User health data: 3000 steps (daily), heart rate 80 (average), sleep time 6 hours (average)

[0985] User's recent meals: pizza, hamburger

[0986] Prompt to spawn AI:

[0987] "The user is not very active, so please suggest a lower calorie option for their next meal."

[0988] This allows users to understand their health status in real time and choose healthy meals when using food delivery services.In addition, by using the suggestion method, optimal advice based on the user's health data is provided, allowing for efficient health management.

[0989] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0990] Step 1:

[0991] A user wears a wearable device while going about their daily life. This device is equipped with a step sensor, a heart rate sensor, and a sleep sensor, and has the function of collecting health data such as the user's steps, heart rate, and sleep time in real time. The input of the data collection is the user's physical activity, and the output is the collected health data.

[0992] Step 2:

[0993] The device sends the collected health data to the server at regular intervals. The data is encrypted before transmission to ensure security. The input is the collected health data, and the output is the transmission of the encrypted health data to the server. This ensures that the user's health data reaches the server safely.

[0994] Step 3:

[0995] The server analyzes the received health data. The input is the received health data, which is analyzed using a generation AI method. Specific operations of data analysis include pattern recognition, such as low step counts and high heart rate. Based on the results of this analysis, optimal health advice is generated for each user. The output is the generated health advice.

[0996] Step 4:

[0997] The server sends the generated health advice to the user terminal. The input is the generated health advice, and the output is the transmission of the advice to the user terminal. The data is encrypted and transferred securely by the communication means.

[0998] Step 5:

[0999] The device receives the advice sent from the server and notifies the user through a display or application. The input is the health advice sent from the server, and the output is the content to be notified to the user. Specific actions can include a pop-up notification in the application or an email notification.

[1000] Step 6:

[1001] A user launches a food delivery app. The app references the health data and advice previously received from the device. The input is the user's health data and generated advice, and the output is the health-conscious meal options displayed on the food delivery menu screen.

[1002] Step 7:

[1003] The suggestion mechanism suggests optimal meal options based on the user's health data. For example, if the user is not exercising enough, it suggests "salads" or "low-calorie dishes." The generative AI model then uses a prompt such as "The user is not exercising much, so please suggest low-calorie options" to make optimal suggestions. The input is the user's health data and the prompt, and the output is a suggestion of a healthy meal option.

[1004] This will enable the entire system to consistently handle everything from data collection to advice suggestions and meal selection, providing comprehensive support for users' health management.

[1005] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1006] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. Furthermore, an emotion engine that recognizes the user's emotions is also incorporated.

[1007] Overall system overview

[1008] This system consists of a wearable device (hereafter referred to as the terminal) and a server linked to it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates individualized advice that takes the emotional data into account. The generated advice is then sent to the terminal and notified to the user.

[1009] Data collection

[1010] The device is equipped with a step sensor, heart rate sensor, sleep sensor, camera, and emotion engine. These sensors collect the user's health data (step count, heart rate, sleep time, meal time) and emotion data in real time. The emotion engine analyzes the user's facial expressions and voice to determine their mood and emotional state for the day.

[1011] Data transmission

[1012] The device periodically transmits the collected health data, food images, and emotion data to a server, and the transmission is encrypted to ensure security.

[1013] Data analysis and generative AI

[1014] The server analyzes the received data and uses AI generation to generate personalized health advice. Specifically, if the step count data is low, it will point out a lack of exercise and advise the user to "walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake. Furthermore, it adjusts the advice based on emotional data. For example, if the emotion engine detects the user's stress level, it will add the advice "Take time to relax."

[1015] Sending and viewing advice

[1016] The advice generated by the server is sent to the device. The device notifies the user of the advice and displays it on a display or in an app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" may be displayed.

[1017] Specific examples

[1018] Scenario 1: Collecting step count data and emotion data, giving advice

[1019] Device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps today, the emotion engine detects that the user's stress level is high.

[1020] Terminal: Sends collected step count data and emotion data to the server.

[1021] Server: Analyzes data and detects lack of exercise and stress.

[1022] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[1023] Server: Sends the generated advice to the terminal.

[1024] Terminal: Notify the user.

[1025] Scenario 2: Calorie management based on food image analysis and emotional state

[1026] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[1027] Device: Sends food images and emotion data to the server.

[1028] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1029] Server: Considers emotional data, determines that the calorie intake is high, and generates specific advice such as, "Increase the amount of vegetables at your next meal. Eat healthily to maintain your great mood today."

[1030] Server: Sends the generated advice to the terminal.

[1031] Terminal: Notify the user.

[1032] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[1033] The processing flow will be explained below.

[1034] Step 1:

[1035] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, etc.) in real time. This includes a step sensor, heart rate sensor, and sleep sensor. The device periodically records the data obtained from these sensors in its internal memory.

[1036] Step 2:

[1037] User: When eating a meal, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. The user is prompted to confirm that they want to save the photo.

[1038] Step 3:

[1039] On the device: The emotion engine collects emotion data from the user's facial expressions and voice. For example, it uses a facial recognition algorithm to analyze the user's facial expressions and determine their emotional state, such as joy, sadness, or anger.

[1040] Step 4:

[1041] Device: The device periodically transmits collected health data, food images, and emotion data to the server. The transmitted data is encrypted to ensure communication security.

[1042] Step 5:

[1043] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. Analysis includes time-series evaluation of health data such as number of steps, heart rate, and sleep time, calorie calculation of meals, and analysis of emotional data.

[1044] Step 6:

[1045] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. The food photos are analyzed to determine the type and amount of food, and the calorie intake is estimated.

[1046] Step 7:

[1047] Server: Using the generative AI, it generates personalized health advice based on the analysis results. For example, if the analysis results indicate a lack of exercise, it generates the advice "You should walk a little more today." If the emotion engine determines that the user is feeling stressed, it adds the advice "Take time to relax."

[1048] Step 8:

[1049] Server: Sends the generated advice to the device. The advice is customized for each user and organized in an appropriate format.

[1050] Step 9:

[1051] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app, and the user can check the content. For example, a notification may be displayed saying, "Your heart rate tends to be high. Take time to relax."

[1052] Step 10:

[1053] User: Improves lifestyle habits based on the advice received. For example, taking an extra walk or adding more vegetables to their next meal. At this stage, users can understand their own health and emotional state and optimize their behavior based on high-quality advice.

[1054] Through this series of processing steps, users receive specific advice tailored to their individual health and emotional state, enabling them to effectively manage their own health in their daily lives.

[1055] Example 2

[1056] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1057] In modern society, busy lifestyles and irregular eating habits make it difficult to manage personal health. In addition to simple health data, emotional states are also important factors in comprehensive health management. However, current systems struggle to effectively collect and analyze this data and provide users with personalized advice. Furthermore, there is a lack of systems that can manage and analyze health and emotional data together and provide users with specific behavioral guidance.

[1058] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to the server, and generation AI means for analyzing the transmitted health data and emotional data and generating individual advice. This makes it possible to provide comprehensive and personalized health management based on the user's health condition and emotional state.

[1059] "Sensor means" refers to devices or mechanisms for collecting health and emotional data of an individual.

[1060] "Communication means" refers to the network connection capability for transmitting collected health and emotion data to a server.

[1061] "Generative AI means" refers to the artificial intelligence models and algorithms used to analyze submitted data and generate personalized advice.

[1062] "Display means" refers to a display or application function for notifying advice displayed on a user terminal.

[1063] "Image recognition means" refers to image recognition technology and software used to analyze collected dietary data and calculate calorie intake.

[1064] "Data storage means" refers to a database or storage system for long-term storage and management of collected health data, emotional data, and calorie intake data.

[1065] This invention provides a system for supporting a user's health management. The system includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to a server, AI generation means for analyzing the transmitted health data and emotional data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, and display means for displaying the advice on the user terminal.

[1066] System Configuration

[1067] The entire system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice that takes the emotional data into account. The advice is then sent to the terminal and notified to the user.

[1068] Data collection

[1069] The device is equipped with the following sensors:

[1070] Pedometer: Records the number of steps taken by the user. Example: Records 5,000 steps in a day.

[1071] Heart Rate Sensor: Measures the user's heart rate. Example: Heart rate is 70 bpm on average.

[1072] Sleep sensor: Records the user's sleep time. Example: The previous night's sleep time was 7 hours.

[1073] Camera: Takes a photo of the user's meal. Example: Takes a photo of lunch.

[1074] Emotion engine: Analyzes the user's facial expressions and voice to determine their emotional state. Example: Detects that the user is smiling and determines that they are feeling happy.

[1075] The emotion engine is equipped with advanced analytical technology and can identify the user's mood and stress level for the day through facial expression and voice analysis.

[1076] Data transmission

[1077] The device sends the collected data to the server at regular intervals. This transmission is performed using encryption technology (e.g., TLS / SSL) to ensure security. The data is sent every 15 minutes, for example.

[1078] Data analysis and generative AI

[1079] The server uses advanced analysis algorithms and generative AI models (e.g., GPT-3) to analyze the received data.

[1080] Analysis of step count data: Analyze daily step count data and evaluate the amount of exercise. For example, if the number of steps taken in a day is less than 3,000, it is determined that the person is not exercising enough.

[1081] Food image analysis: Using image recognition technology (e.g., Google Cloud Vision API), calculate calorie intake from food photos. Example: Estimate 800 kcal from a photo of lunch.

[1082] Emotion data analysis: The emotion engine considers the analyzed emotion data. For example, the stress level is determined to be high.

[1083] Example prompt sentence:

[1084] "Please send the collected health and emotional data to the server." "Please analyze the received data and evaluate the health and emotional state." "Please generate appropriate health advice for the user based on the analysis results." "Please send the generated advice to the device and notify the user."

[1085] Advice Generation

[1086] Based on the analysis results, the server generates personalized health advice using a generative AI model:

[1087] Example 1: If step count data is low and stress levels are high, generate advice such as "Walk a little more today. Take time to relax."

[1088] Example 2: If your calorie intake is high, generate advice like "Increase your vegetables at your next meal. Eat healthily to keep feeling great today."

[1089] Sending and viewing advice

[1090] The advice generated by the server is sent back to the device, which then notifies the user of the received advice and displays it on the display or app:

[1091] Example 1: The notification "Your heart rate tends to be high. Take time to relax" appears on the display.

[1092] Example 2: The app displays a notification saying, "You've consumed a lot of calories, so try eating a lighter meal next time."

[1093] The system allows users to receive specific advice based on their health and emotional data to improve their lifestyle habits.

[1094] Specific examples

[1095] As a concrete example, consider the following scenario:

[1096] Scenario 1: Collecting step count data and emotion data, giving advice

[1097] On the device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps, the emotion engine detects that the user's stress level is high.

[1098] Terminal: Sends collected data to the server.

[1099] Server: Analyzes data and detects lack of exercise and stress.

[1100] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[1101] Server: Sends the generated advice to the device.

[1102] Terminal: Notify the user.

[1103] Scenario 2: Calorie management based on food image analysis and emotional state

[1104] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[1105] Device: Sends food images and emotion data to the server.

[1106] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1107] Server: Considers emotional data, determines that your calorie intake is high, and generates specific advice such as, "Increase your vegetables at your next meal. Eat healthily to maintain your great mood today."

[1108] Server: Sends the generated advice to the device.

[1109] Terminal: Notify the user.

[1110] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[1111] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1112] Step 1: Data collection

[1113] The device uses the following sensors to collect health and emotional data. Specifically, the device uses a pedometer to record the number of steps taken each day, a heart rate sensor to measure the heart rate, and a sleep sensor to record the amount of sleep. It also uses a camera to take photos of meals, and an emotion engine to analyze the user's facial expressions and voice to determine their emotional state for the day. For example, if a user records 5,000 steps, an average heart rate of 70 bpm, and seven hours of sleep, and then takes a photo of the meal, the device determines their emotional state as "high happiness."

[1114] Input: Raw data collected by sensors (number of steps, heart rate, sleep time, meal photos, facial expression and voice data)

[1115] Output: Collected health and emotional data (e.g., steps = 5000, heart rate = 70 bpm, sleep time = 7 hours, emotional state = high happiness)

[1116] Examples of using prompt statements

[1117] "Collect user health and emotional data."

[1118] Step 2: Send data

[1119] The device sends the collected health and emotional data to the server. Specifically, the device sends the collected data to the server every 15 minutes using encryption technology (e.g., TLS / SSL). For example, the number of steps taken (5,000), heart rate (70 bpm), sleep time (7 hours), emotional state ("high happiness"), and meal photos are sent to the server.

[1120] Input: Collected health and emotion data

[1121] Output: Data sent to the server

[1122] Examples of using prompt statements

[1123] "Please send the collected health and emotion data to the server."

[1124] Step 3: Data analysis

[1125] The server analyzes the received data. Specifically, it first analyzes the step count data and evaluates the amount of exercise. It also uses image recognition technology to calculate calorie intake from meal photos and takes into account the emotional data provided by the emotion engine. For example, if the number of steps taken in a day is less than 3,000, it determines that the person is not exercising enough, identifies the calorie intake of the meal as 800 kcal from a photo of lunch, and determines the stress level as high from the emotional data.

[1126] Input: Health and emotion data sent to the server

[1127] Output: Analysis results (lack of exercise, calorie intake 800 kcal, high stress level)

[1128] Examples of using prompt statements

[1129] "Analyze the data received and assess your health and emotional state."

[1130] Step 4: Advice Generation

[1131] The server uses the generative AI model to generate personalized health advice based on the analysis results. Specifically, it generates advice such as "You should walk a little more today" and "Take time to relax" based on the analysis results. If the calorie intake is high, it also generates specific advice such as "Increase the amount of vegetables at your next meal."

[1132] Input: Analysis results

[1133] Output: Personalized health advice (e.g., "You should walk a little more today" or "Take time to relax")

[1134] Examples of using prompt statements

[1135] "Based on the analysis results, generate appropriate health advice for the user."

[1136] Step 5: Submitting and viewing advice

[1137] The server sends the generated advice to the device. The device notifies the user of the received advice and displays it on a display or in an app. For example, a notification saying "Your heart rate tends to be high. Take time to relax" may appear on the display. Another notification saying "Your calorie intake is high, so have a lighter meal next time" may appear in the app.

[1138] Input: Generated advice

[1139] Output: Advice given to the user

[1140] Examples of using prompt statements

[1141] Send the generated advice to the device and notify the user.

[1142] (Application example 2)

[1143] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] Conventional health management systems collect health and emotional data and provide users with health advice, but they are unable to provide specific improvement suggestions based on users' spending patterns and lifestyle habits. Therefore, there is a need for a system that supports users in improving their lifestyle habits through healthy consumption behavior.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1146] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data and emotional data to the server, a generation AI means for analyzing the transmitted health data and emotional data and generating personalized advice, a communication means for transmitting the generated advice to a user terminal, a display means for notifying the user of the advice to be displayed on the user terminal, a data collection and analysis means for collecting and analyzing expenditure data, and a generation AI model means for providing personalized advice to promote healthy consumption based on the health data and expenditure data. This enables comprehensive health management and improvement of consumption behavior based on the user's health and emotional state and spending patterns.

[1147] "Sensor means" refers to devices for collecting personal health data, including step sensors, heart rate sensors, sleep sensors, cameras, etc.

[1148] "Communication means" refers to the technology used to transmit collected data to the server. This includes wireless communication technologies such as Wi-Fi and Bluetooth.

[1149] "Generative AI methods" are artificial intelligence technologies used to analyze submitted data and generate personalized advice, including natural language processing and machine learning algorithms.

[1150] The "display means" is a function for displaying the generated advice on the user terminal, such as the screen of a smartphone or the display of a smartwatch.

[1151] "Data collection and analysis means" refers to technology for collecting and analyzing user spending data, including electronic payment systems and purchase history management systems.

[1152] "Generative AI modeling tools" are artificial intelligence technologies, including machine learning models and generative adversarial networks (GANs), that provide personalized advice based on health and expenditure data.

[1153] This invention is a system that supports users in managing their health and improving their consumption behavior. This system collects personal health and emotional data and provides personalized health advice based on the analysis results. It also collects and analyzes the user's expenditure data and generates advice to promote healthy consumption behavior.

[1154] Hardware and software used

[1155] Sensor devices: Health data is collected through wearable devices such as the Apple Watch and Fitbit, which incorporate step, heart rate, and sleep sensors.

[1156] Communication technology: Data is transmitted using wireless communication technologies such as Wi-Fi and Bluetooth, which allows for smooth data transfer from the device to the server.

[1157] Server: AWS Lambda and AWS RDS are used to analyze and store data. AWS S3 is used to temporarily store and encrypt data.

[1158] Generative AI model: OpenAI's GPT-3 is used to generate personalized health advice, and Affectiva SDK is used to analyze emotion data.

[1159] System processing overview

[1160] The device collects personal health data (step count, heart rate, sleep data) and emotional data. Emotional data is extracted from the user's facial expressions and voice using the Affectiva SDK. This data is sent to a server at regular intervals. The sent data is stored in AWS S3 via AWS Lambda. The stored data is then integrated into a database using AWS RDS.

[1161] The server analyzes the health and emotional data and generates personalized health advice using a generative AI model (GPT-3). For example, if a user's step count is low and emotional data indicates a high stress level, the server generates advice such as, "You should walk a little more today. Take time to relax."

[1162] Spending data will also be collected and integrated with health data for analysis. Purchase history collected through electronic payment systems will be used to analyze consumer behavior. This will generate specific, healthy spending advice, such as, "Your food expenses have tended to be high this month. Consider making healthy choices the next time you buy food."

[1163] The generated advice is sent to the user's device in real time, allowing the user to improve their health and consumption behavior in their daily lives.

[1164] Specific examples

[1165] For example, if a user only walked 3,000 steps on a particular day and the emotion engine detects that the user's stress level is high, the following example prompts will be fed into the generative AI model:

[1166] Example prompt sentence:

[1167] Health data: Steps: 3000, Heart rate: 80 / min, Sleep time: 6 hours

[1168] Emotional data: Stress level: High, Mood: Unstable

[1169] Generation advice: With your step count low and stress levels high, take time to relax and take a short walk to refresh your mind.

[1170] In this way, the generative AI model takes into account the user's health and emotional state to provide appropriate advice. Furthermore, the user's spending data is also analyzed, and specific instructions are given to promote a healthy lifestyle. For example, advice such as "This month's food expenses are above average. Next time, choose a menu that includes lots of vegetables" is provided.

[1171] By using this system, users can improve their overall health management and consumption habits, leading to a healthier lifestyle.

[1172] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1173] Step 1:

[1174] The device collects the user's health data (step count, heart rate, sleep data) and emotional data. The sensor device (e.g., wearable device) used measures the user's biometric data in real time. The input is the user's biometric data, and the output is the collected health data and emotional data.

[1175] Step 2:

[1176] The device sends the collected health and emotion data to a server via Wi-Fi or Bluetooth. The input is the collected data, and the output is the encrypted data sent to the server.

[1177] Step 3:

[1178] The server uses AWS Lambda to temporarily store the submitted data in AWS S3, where the input is the data submitted to the server and the output is the data stored in AWS S3.

[1179] Step 4:

[1180] The server uses AWS RDS to integrate data stored in S3 into a database, where the input is data stored in AWS S3 and the output is the integrated data in the database.

[1181] Step 5:

[1182] The server uses AWS Lambda to analyze the health and emotional data. The analysis is performed using the Affectiva SDK. The input is the data integrated into the database, and the output is the analyzed results of the user's health and emotional state.

[1183] Step 6:

[1184] The server uses a generative AI model (GPT-3) to generate personalized health advice based on the analysis results, where the input is the health and emotional state, and the output is the generated personalized health advice.

[1185] Step 7:

[1186] The server stores the generated individual health advice in AWS RDS and simultaneously sends it to the user's device. The input here is the generated advice, and the output is a notification of the advice to the device.

[1187] Step 8:

[1188] The user terminal notifies the user of the generated advice and displays it on the display. The input here is the advice sent from the server, and the output is the advice displayed to the user.

[1189] Step 9:

[1190] The terminal collects the user's spending data from the electronic payment system and sends it to the server. At this stage, the input is the user's purchase history and the output is the collected spending data.

[1191] Step 10:

[1192] The server analyzes the expenditure data and integrates it with the health data for analysis, where the inputs are expenditure data and health data, and the output is the integrated analysis results.

[1193] Step 11:

[1194] Based on the integrated analysis results, the server generates advice to promote healthy spending behaviors using a generative AI model, where the input is the integrated analysis results and the output is the generated spending advice.

[1195] Step 12:

[1196] The server stores the generated spending advice in AWS RDS and sends it to the user's device. The input here is the generated spending advice, and the output is a notification of the advice to the device.

[1197] Step 13:

[1198] The user terminal notifies the generated spending advice to the user and displays it on a display, where the input is the spending advice sent from the server and the output is the advice displayed to the user.

[1199] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1200] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1201] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1202] [Fourth embodiment]

[1203] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1204] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1205] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1206] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1207] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1209] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1210] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1211] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1212] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1213] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1214] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1215] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1216] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal.

[1217] Overall system overview

[1218] This system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice. The advice is then sent to the terminal and notified to the user.

[1219] Data collection

[1220] The device is equipped with a step sensor, heart rate sensor, sleep sensor, etc., and collects the user's step count, heart rate, sleep time, etc. in real time. It also has a built-in camera, so it can collect meal data by allowing the user to take photos of their meals.

[1221] Data transmission

[1222] The device periodically sends the collected health and dietary data to a server, and the transmission is encrypted to ensure security.

[1223] Data analysis and generative AI

[1224] The server analyzes the received data and generates personalized health advice using AI. Specifically, if the step count data is low, it will point out a lack of exercise and advise, "You should walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake.

[1225] Sending and viewing advice

[1226] The advice generated by the server is delivered to the device. The device notifies the user of the advice and displays it on a display or app. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed.

[1227] Specific examples

[1228] Scenario 1: Step data collection and advice

[1229] Device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, such as 3,000.

[1230] Device: Sends this step count data to the server.

[1231] Server: Analyzes the received step count data and determines that the amount of exercise is insufficient.

[1232] Server: Using generative AI, generate advice such as "You should walk a little more today."

[1233] Server: Sends the generated advice to the terminal.

[1234] Terminal: Notify the user.

[1235] Scenario 2: Food image analysis and calorie management

[1236] Device: The user takes a photo of the food they had for lunch.

[1237] Device: Sends this food image to the server.

[1238] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1239] Server: Determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[1240] Server: Sends the generated advice to the terminal.

[1241] Terminal: Notify the user.

[1242] In this way, this system provides comprehensive support for the user's health through a series of processes, from data collection by sensors to data analysis, and the generation and notification of individualized advice. Users can easily understand their health status in their daily lives and improve their lifestyle habits based on the advice.

[1243] The processing flow will be explained below.

[1244] Step 1:

[1245] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, meal times, etc.) in real time. This includes a step sensor, heart rate sensor, sleep sensor, and camera. During this time, the device periodically reads the sensor values ​​and records them in its internal memory.

[1246] Step 2:

[1247] User: When eating, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. At this time, a confirmation message for the photo-taking procedure and saving is displayed, allowing the user to check the contents of the photo.

[1248] Step 3:

[1249] Device: The collected health data and meal images are sent to the server at regular intervals. All information is encrypted before transmission to ensure communication security.

[1250] Step 4:

[1251] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. The analysis includes the process of evaluating data such as the number of steps, heart rate, and sleep time over time.

[1252] Step 5:

[1253] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. In this process, an image analysis model recognizes the contents of the meal and compares it with a general calorie database to estimate the calories.

[1254] Step 6:

[1255] Server: Using generation AI, it generates personalized health advice based on the analysis results. For example, if the number of steps is low, it generates advice such as "You should walk a little more today," and if the calorie intake is high, it generates advice such as "You should eat more vegetables at your next meal."

[1256] Step 7:

[1257] Server: The server sends the generated advice to the device. At this time, the advice is customized for each user and organized in an appropriate format. The content of the advice is concise and organized in a format that is easy for the user to understand.

[1258] Step 8:

[1259] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" is displayed.

[1260] Step 9:

[1261] User: Improves lifestyle habits based on the advice received, for example, taking an extra walk after receiving a notification or adding more vegetables to their next meal.

[1262] Through these steps, the system provides comprehensive support for users in maintaining their health and offers specific, practical advice.

[1263] Example 1

[1264] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1265] Conventional health management systems have had difficulty effectively analyzing collected health data and providing personalized health advice. They also lacked the functionality to analyze users' dietary data in detail and accurately calculate calorie intake. Furthermore, they lacked mechanisms for notifying users of generated advice in real time. For these reasons, there was a need for the development of a system that could comprehensively support users' health management.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1267] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data to the server, a generation AI means for analyzing the transmitted health data and generating personalized advice, a generation AI model means for inputting a prompt sentence into the generation AI model and generating health advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. This makes it possible to analyze the user's health data and dietary data in detail and quickly and effectively generate and notify personalized health advice.

[1268] "Sensor means" refers to a device used to collect personal health data, and includes a step sensor, a heart rate sensor, a sleep sensor, etc.

[1269] "Communication means" refers to the devices and technologies used to transmit collected data to the server, including wireless communication modules and internet connections.

[1270] "Generative AI Means" are the artificial intelligence algorithms and systems used to analyze the submitted data and generate personalized advice.

[1271] A "generative AI model means" is an artificial intelligence model used to generate health advice in response to a prompt statement.

[1272] "Display means" refers to the devices or techniques used to notify advice that is displayed on a user terminal, including displays and notification systems.

[1273] "Image recognition means" is a technology used to analyze collected dietary data and calculate calorie intake.

[1274] "Data storage means" refers to a system used to store and manage collected health and dietary data over the long term and provide ongoing health management advice.

[1275] This invention is a system that provides comprehensive support for the health of users in their daily lives. The system consists of a wearable device (hereinafter referred to as a "terminal") worn by the user and a server that works in conjunction with the device.

[1276] Data collection

[1277] Device: The device is equipped with various sensors, such as a step sensor, heart rate sensor, and sleep sensor, and collects health data such as the user's number of steps, heart rate, and sleep time in real time.The device also has a camera, which collects dietary data by allowing the user to take photos of their meals.

[1278] Data transmission

[1279] Device: All collected health and dietary data is encrypted and sent to a server at regular intervals using encryption technology such as AES-256. Transmission is via Wi-Fi or mobile data.

[1280] Data analysis and generative AI

[1281] Server: The server stores the received data in a database, and uses an analysis module to detect patterns and outliers in the data. It then inputs prompts into a generative AI model to generate health advice.

[1282] An example of a specific prompt: "Generate advice for the user on days when their step count data is low."

[1283] For example, the generative AI model might generate advice such as, "You should walk a little more today. Why not try a walk in a nearby park?"

[1284] Generative AI model: The generative AI model used utilizes the latest natural language processing techniques to generate detailed and relevant advice based on the prompt.

[1285] Sending and viewing advice

[1286] Server: The generated advice is encrypted and sent to the terminal.

[1287] Device: Interprets the received advice and notifies the user. Notification methods include push notifications and banner displays. For example, advice such as "Your heart rate tends to be high. Take time to relax" is displayed on the screen.

[1288] Specific examples

[1289] Step 1: Collecting step data and giving advice

[1290] The device records the user's steps throughout the day, and if the user takes only 3,000 steps in a day, the data is stored on the device.

[1291] The terminal transmits this step count data to the server.

[1292] The server analyzes the received step count data and determines that the amount of exercise is insufficient.

[1293] The server uses a generative AI to generate advice such as, "You should walk a little more today."

[1294] The server transmits the generated advice to the terminal.

[1295] The terminal notifies the user and displays advice on the display.

[1296] Step 2: Food image analysis and calorie management

[1297] The terminal takes a photo of the food the user had for lunch.

[1298] The device sends this food image to the server.

[1299] The server analyzes the food image and calculates the calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1300] The server determines that the calorie intake is high and generates specific advice such as "Increase the amount of vegetables at your next meal."

[1301] The server transmits the generated advice to the terminal.

[1302] The terminal notifies the user and displays advice on the display.

[1303] As described above, the present invention combines various technologies such as sensor means, communication means, and generation AI means to grasp a user's health condition in real time and provide personalized health advice, allowing users to easily manage their health condition in their daily lives and continuously improve their lifestyle habits.

[1304] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1305] Step 1: Data collection

[1306] Device: Using an activity tracker, heart rate sensor, and sleep sensor, the device collects real-time health data (number of steps, heart rate, sleep time, etc.) from the user's daily life. For example, a smartwatch measures the number of steps a user takes in a day and stores the data on the device when they reach 10,000 steps.

[1307] Device: The user takes a photo of their lunch using their smartphone camera to collect meal data. The device stores the captured image in its internal storage.

[1308] Input: Raw data from sensors and food photos.

[1309] Output: Health and dietary data stored on the device.

[1310] Step 2: Send data

[1311] Terminal: All collected data is sent to the server at regular intervals. Before being sent, the data is securely protected using AES-256 encryption technology.

[1312] On your device: Using Wi-Fi or mobile data, the encrypted data is sent to a server.

[1313] Input: Health and dietary data stored on the device.

[1314] Output: The encrypted data sent to the server.

[1315] Step 3: Receiving and storing data

[1316] Server: Decrypts the received encrypted data and stores it in a database. The database uses a storage system that allows for quick search and analysis.

[1317] Input: Encrypted data.

[1318] Output: Decrypted data stored in a database.

[1319] Step 4: Data analysis

[1320] Server: Passes the data stored in the database to the analysis module, which detects patterns in the step count, heart rate, and sleep data and identifies outliers, such as a low current step count compared to the average step count over the past week.

[1321] Input: Health data from the database.

[1322] Output: Analysis results (e.g., information indicating low step count).

[1323] Step 5: Generative AI model generates advice

[1324] Server: Based on the analysis results, input a prompt to the generative AI model. For example, "Please generate advice for days when the user's step count data is low."

[1325] Generative AI model: Generates detailed and relevant health advice based on prompts. Generative AI takes into account context and past data to provide optimal advice.

[1326] Input: prompt statement, analysis result.

[1327] Output: Health advice (e.g., "You should walk a little more today").

[1328] Step 6: Send advice

[1329] Server: Encrypts the generated advice and sends it to the terminal.

[1330] Input: The generated advice.

[1331] Output: Encrypted advice data to the terminal.

[1332] Step 7: Display Advice

[1333] Device: Interprets the received advice and notifies the user. This can be done using push notifications or in-app banners. For example, the device might display advice such as "Your heart rate tends to be high. Take some time to relax."

[1334] Input: Encrypted advice data sent by the server.

[1335] Output: Displaying information and advice messages to the user.

[1336] (Application example 1)

[1337] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1338] In modern society, personal health management is an important issue. In particular, lack of exercise and improper diet often have negative effects on health, so there is a need for effective means of managing these. Furthermore, when users use food delivery services, there is a lack of functionality that provides optimal meal options based on their health status. For this reason, a system is needed that can collect and analyze individual health data in real time and provide appropriate advice to comprehensively support users' health management.

[1339] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1340] In this invention, the server includes sensor means for collecting personal health data, communication means for transmitting the collected health data to the server, generation AI means for analyzing the transmitted health data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, display means for notifying the user of the advice to be displayed on the user terminal, and suggestion means for suggesting optimal meal options based on the user's health data. This allows users to understand their own health status in real time and enables them to make health-conscious meal choices even when using food delivery services.

[1341] The "sensor means" is a device for collecting personal health data, and has the function of detecting the number of steps taken, heart rate, sleep time, etc. in real time.

[1342] "Communication means" refers to a device or system that has the function of transmitting collected health data to a server, and is capable of encrypting data communication.

[1343] "Generative AI means" refers to a device or system that uses artificial intelligence technology to analyze health data sent to the server and generate personalized health advice.

[1344] The "display means" refers to a display or application that notifies and displays advice sent from the server on the user terminal.

[1345] "Dietary data" is information about the meals consumed by the user, and includes photos of the meals and manually entered lists of ingredients.

[1346] The "image recognition means" is a device or system that has the function of analyzing collected dietary data and calculating calorie intake.

[1347] "Data storage means" refers to a device or system that has the function of storing and accumulating collected health data and calorie intake data over the long term.

[1348] The "suggestion means" is a device or system for suggesting optimal meal options based on the user's health data, and has the function of assisting in meal selection based on the generated advice.

[1349] Overall system overview

[1350] This invention is a system that comprehensively supports personal health management. The system includes a sensor means for collecting health data, a communication means for transmitting the data to a server, an AI generation means for analyzing the data and generating personalized advice, a means for transmitting the advice to a terminal and displaying it, and a suggestion means for suggesting optimal meal options based on the user's health data. The system works in conjunction with a wearable device to collect the user's health data in real time. This data is analyzed by the server and provided to the user as advice.

[1351] Hardware and software used

[1352] Wearable device: Equipped with step sensors, heart rate sensors, and sleep sensors.

[1353] Server: Uses data analysis and generative AI models (e.g., GPT-4).

[1354] User device: Advice is displayed and notified using a display or application.

[1355] Data processing and calculation

[1356] 1. Health Data Collection:

[1357] Wearable devices collect real-time health data such as the user's steps, heart rate, and sleep duration.

[1358] The device sends the collected data to a server at regular intervals, and the data is encrypted for security purposes.

[1359] 2. Data transmission:

[1360] The collected health data is transmitted to a server via a communication means and analyzed by the server.

[1361] 3. Data analysis and advice generation:

[1362] The server analyzes the received health data and generates personalized health advice using a generative AI model (e.g., GPT-4).

[1363] For example, if the number of steps taken is low, the system will generate advice such as, "You should walk a little more today." It will also analyze photos of meals, calculate calorie intake, and provide advice on calorie management.

[1364] 4. Send and display advice to user device:

[1365] The advice generated by the server is transmitted to the user terminal via a communication means.

[1366] The user terminal notifies the user of the advice and displays it in an application or on a display.

[1367] 5. Suggestions based on health advice:

[1368] The suggestion tool provides optimal meal options based on the user's health data, for example, healthy meal options if the user is not getting enough exercise.

[1369] Specific examples

[1370] Scenario 1: Step data collection and advice

[1371] On the device: The activity tracker records the user's steps throughout the day. For example, if the number of steps taken in a day is low, say 3,000.

[1372] Server: Analyzes the received step count data, determines that the amount of exercise is insufficient, and generates advice such as, "You should walk a little more today."

[1373] Terminal: The generated advice is notified to the user.

[1374] Scenario 2: Food image analysis and calorie management

[1375] Device: The user takes a photo of the food they had for lunch.

[1376] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal, it generates specific advice such as "Increase the amount of vegetables at your next meal."

[1377] Terminal: The generated advice is notified to the user.

[1378] Prompt Sentence Examples

[1379] User health data: 3000 steps (daily), heart rate 80 (average), sleep time 6 hours (average)

[1380] User's recent meals: pizza, hamburger

[1381] Prompt to spawn AI:

[1382] "The user is not very active, so please suggest a lower calorie option for their next meal."

[1383] This allows users to understand their health status in real time and choose healthy meals when using food delivery services.In addition, by using the suggestion method, optimal advice based on the user's health data is provided, allowing for efficient health management.

[1384] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1385] Step 1:

[1386] A user wears a wearable device while going about their daily life. This device is equipped with a step sensor, a heart rate sensor, and a sleep sensor, and has the function of collecting health data such as the user's steps, heart rate, and sleep time in real time. The input of the data collection is the user's physical activity, and the output is the collected health data.

[1387] Step 2:

[1388] The device sends the collected health data to the server at regular intervals. The data is encrypted before transmission to ensure security. The input is the collected health data, and the output is the transmission of the encrypted health data to the server. This ensures that the user's health data reaches the server safely.

[1389] Step 3:

[1390] The server analyzes the received health data. The input is the received health data, which is analyzed using a generation AI method. Specific operations of data analysis include pattern recognition, such as low step counts and high heart rate. Based on the results of this analysis, optimal health advice is generated for each user. The output is the generated health advice.

[1391] Step 4:

[1392] The server sends the generated health advice to the user terminal. The input is the generated health advice, and the output is the transmission of the advice to the user terminal. The data is encrypted and transferred securely by the communication means.

[1393] Step 5:

[1394] The device receives the advice sent from the server and notifies the user through a display or application. The input is the health advice sent from the server, and the output is the content to be notified to the user. Specific actions can include a pop-up notification in the application or an email notification.

[1395] Step 6:

[1396] A user launches a food delivery app. The app references the health data and advice previously received from the device. The input is the user's health data and generated advice, and the output is the health-conscious meal options displayed on the food delivery menu screen.

[1397] Step 7:

[1398] The suggestion mechanism suggests optimal meal options based on the user's health data. For example, if the user is not exercising enough, it suggests "salads" or "low-calorie dishes." The generative AI model then uses a prompt such as "The user is not exercising much, so please suggest low-calorie options" to make optimal suggestions. The input is the user's health data and the prompt, and the output is a suggestion of a healthy meal option.

[1399] This will enable the entire system to consistently handle everything from data collection to advice suggestions and meal selection, providing comprehensive support for users' health management.

[1400] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1401] This invention is a system that supports a user's health management, and includes a sensor means for collecting personal health data, a communication means for transmitting the collected data to a server, a generation AI means for analyzing the transmitted data and generating individual advice, a communication means for transmitting the generated advice to a user terminal, and a display means for displaying the advice on the user terminal. Furthermore, an emotion engine that recognizes the user's emotions is also incorporated.

[1402] Overall system overview

[1403] This system consists of a wearable device (hereafter referred to as the terminal) and a server linked to it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates individualized advice that takes the emotional data into account. The generated advice is then sent to the terminal and notified to the user.

[1404] Data collection

[1405] The device is equipped with a step sensor, heart rate sensor, sleep sensor, camera, and emotion engine. These sensors collect the user's health data (step count, heart rate, sleep time, meal time) and emotion data in real time. The emotion engine analyzes the user's facial expressions and voice to determine their mood and emotional state for the day.

[1406] Data transmission

[1407] The device periodically transmits the collected health data, food images, and emotion data to a server, and the transmission is encrypted to ensure security.

[1408] Data analysis and generative AI

[1409] The server analyzes the received data and uses AI generation to generate personalized health advice. Specifically, if the step count data is low, it will point out a lack of exercise and advise the user to "walk a little more today." It also uses image recognition technology to analyze photos of meals and calculate calorie intake. Furthermore, it adjusts the advice based on emotional data. For example, if the emotion engine detects the user's stress level, it will add the advice "Take time to relax."

[1410] Sending and viewing advice

[1411] The advice generated by the server is sent to the device. The device notifies the user of the advice and displays it on a display or in an app. For example, a notification such as "Your heart rate tends to be high. Take time to relax" may be displayed.

[1412] Specific examples

[1413] Scenario 1: Collecting step count data and emotion data, giving advice

[1414] Device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps today, the emotion engine detects that the user's stress level is high.

[1415] Terminal: Sends collected step count data and emotion data to the server.

[1416] Server: Analyzes data and detects lack of exercise and stress.

[1417] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[1418] Server: Sends the generated advice to the terminal.

[1419] Terminal: Notify the user.

[1420] Scenario 2: Calorie management based on food image analysis and emotional state

[1421] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[1422] Device: Sends food images and emotion data to the server.

[1423] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1424] Server: Considers emotional data, determines that the calorie intake is high, and generates specific advice such as, "Increase the amount of vegetables at your next meal. Eat healthily to maintain your great mood today."

[1425] Server: Sends the generated advice to the terminal.

[1426] Terminal: Notify the user.

[1427] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[1428] The processing flow will be explained below.

[1429] Step 1:

[1430] Device: The activity tracker collects the user's health data (number of steps, heart rate, sleep time, etc.) in real time. This includes a step sensor, heart rate sensor, and sleep sensor. The device periodically records the data obtained from these sensors in its internal memory.

[1431] Step 2:

[1432] User: When eating a meal, the user takes a photo of the meal using the device's camera. This photo is temporarily saved in the device's memory. The user is prompted to confirm that they want to save the photo.

[1433] Step 3:

[1434] On the device: The emotion engine collects emotion data from the user's facial expressions and voice. For example, it uses a facial recognition algorithm to analyze the user's facial expressions and determine their emotional state, such as joy, sadness, or anger.

[1435] Step 4:

[1436] Device: The device periodically transmits collected health data, food images, and emotion data to the server. The transmitted data is encrypted to ensure communication security.

[1437] Step 5:

[1438] Server: Receives the transmitted data and stores it in a database. At the same time, the data analysis module reads the data and begins analysis. Analysis includes time-series evaluation of health data such as number of steps, heart rate, and sleep time, calorie calculation of meals, and analysis of emotional data.

[1439] Step 6:

[1440] Server: Analyzes the received food images using image recognition technology to calculate calorie intake. The food photos are analyzed to determine the type and amount of food, and the calorie intake is estimated.

[1441] Step 7:

[1442] Server: Using the generative AI, it generates personalized health advice based on the analysis results. For example, if the analysis results indicate a lack of exercise, it generates the advice "You should walk a little more today." If the emotion engine determines that the user is feeling stressed, it adds the advice "Take time to relax."

[1443] Step 8:

[1444] Server: Sends the generated advice to the device. The advice is customized for each user and organized in an appropriate format.

[1445] Step 9:

[1446] Device: The user is notified of the advice received from the server. The advice is displayed on the device display or in the app, and the user can check the content. For example, a notification may be displayed saying, "Your heart rate tends to be high. Take time to relax."

[1447] Step 10:

[1448] User: Improves lifestyle habits based on the advice received. For example, taking an extra walk or adding more vegetables to their next meal. At this stage, users can understand their own health and emotional state and optimize their behavior based on high-quality advice.

[1449] Through this series of processing steps, users receive specific advice tailored to their individual health and emotional state, enabling them to effectively manage their own health in their daily lives.

[1450] Example 2

[1451] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] In modern society, busy lifestyles and irregular eating habits make it difficult to manage personal health. In addition to simple health data, emotional states are also important factors in comprehensive health management. However, current systems struggle to effectively collect and analyze this data and provide users with personalized advice. Furthermore, there is a lack of systems that can manage and analyze health and emotional data together and provide users with specific behavioral guidance.

[1453] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to the server, and generation AI means for analyzing the transmitted health data and emotional data and generating individual advice. This makes it possible to provide comprehensive and personalized health management based on the user's health condition and emotional state.

[1454] "Sensor means" refers to devices or mechanisms for collecting health and emotional data of an individual.

[1455] "Communication means" refers to the network connection capability for transmitting collected health and emotion data to a server.

[1456] "Generative AI means" refers to the artificial intelligence models and algorithms used to analyze submitted data and generate personalized advice.

[1457] "Display means" refers to a display or application function for notifying advice displayed on a user terminal.

[1458] "Image recognition means" refers to image recognition technology and software used to analyze collected dietary data and calculate calorie intake.

[1459] "Data storage means" refers to a database or storage system for long-term storage and management of collected health data, emotional data, and calorie intake data.

[1460] This invention provides a system for supporting a user's health management. The system includes sensor means for collecting personal health data and emotional data, communication means for transmitting the collected health data and emotional data to a server, AI generation means for analyzing the transmitted health data and emotional data and generating personalized advice, communication means for transmitting the generated advice to a user terminal, and display means for displaying the advice on the user terminal.

[1461] System Configuration

[1462] The entire system consists of a wearable device (hereafter referred to as the terminal) and a server that works in conjunction with it. The terminal collects health and emotional data from the user's daily life and sends it to the server. The server analyzes the received data and generates personalized advice that takes the emotional data into account. The advice is then sent to the terminal and notified to the user.

[1463] Data collection

[1464] The device is equipped with the following sensors:

[1465] Pedometer: Records the number of steps taken by the user. Example: Records 5,000 steps in a day.

[1466] Heart Rate Sensor: Measures the user's heart rate. Example: Heart rate is 70 bpm on average.

[1467] Sleep sensor: Records the user's sleep time. Example: The previous night's sleep time was 7 hours.

[1468] Camera: Takes a photo of the user's meal. Example: Takes a photo of lunch.

[1469] Emotion engine: Analyzes the user's facial expressions and voice to determine their emotional state. Example: Detects that the user is smiling and determines that they are feeling happy.

[1470] The emotion engine is equipped with advanced analytical technology and can identify the user's mood and stress level for the day through facial expression and voice analysis.

[1471] Data transmission

[1472] The device sends the collected data to the server at regular intervals. This transmission is performed using encryption technology (e.g., TLS / SSL) to ensure security. The data is sent every 15 minutes, for example.

[1473] Data analysis and generative AI

[1474] The server uses advanced analysis algorithms and generative AI models (e.g., GPT-3) to analyze the received data.

[1475] Analysis of step count data: Analyze daily step count data and evaluate the amount of exercise. For example, if the number of steps taken in a day is less than 3,000, it is determined that the person is not exercising enough.

[1476] Food image analysis: Using image recognition technology (e.g., Google Cloud Vision API), calculate calorie intake from food photos. Example: Estimate 800 kcal from a photo of lunch.

[1477] Emotion data analysis: The emotion engine considers the analyzed emotion data. For example, the stress level is determined to be high.

[1478] Example prompt sentence:

[1479] "Please send the collected health and emotional data to the server." "Please analyze the received data and evaluate the health and emotional state." "Please generate appropriate health advice for the user based on the analysis results." "Please send the generated advice to the device and notify the user."

[1480] Advice Generation

[1481] Based on the analysis results, the server generates personalized health advice using a generative AI model:

[1482] Example 1: If step count data is low and stress levels are high, generate advice such as "Walk a little more today. Take time to relax."

[1483] Example 2: If your calorie intake is high, generate advice like "Increase your vegetables at your next meal. Eat healthily to keep feeling great today."

[1484] Sending and viewing advice

[1485] The advice generated by the server is sent back to the device, which then notifies the user of the received advice and displays it on the display or app:

[1486] Example 1: The notification "Your heart rate tends to be high. Take time to relax" appears on the display.

[1487] Example 2: The app displays a notification saying, "You've consumed a lot of calories, so try eating a lighter meal next time."

[1488] The system allows users to receive specific advice based on their health and emotional data to improve their lifestyle habits.

[1489] Specific examples

[1490] As a concrete example, consider the following scenario:

[1491] Scenario 1: Collecting step count data and emotion data, giving advice

[1492] On the device: The activity tracker records the user's steps throughout the day, and the emotion engine analyzes the user's emotional state. For example, if the user only walked 3,000 steps, the emotion engine detects that the user's stress level is high.

[1493] Terminal: Sends collected data to the server.

[1494] Server: Analyzes data and detects lack of exercise and stress.

[1495] Server: Using generative AI, generate advice such as "You should walk a little more today. Take some time to relax."

[1496] Server: Sends the generated advice to the device.

[1497] Terminal: Notify the user.

[1498] Scenario 2: Calorie management based on food image analysis and emotional state

[1499] Device: The user takes a photo of the food they had for lunch, and the emotion engine analyzes the user's happiness level.

[1500] Device: Sends food images and emotion data to the server.

[1501] Server: Analyzes food images and calculates calorie intake. For example, if the calorie intake calculated from the photo is 800 kcal.

[1502] Server: Considers emotional data, determines that your calorie intake is high, and generates specific advice such as, "Increase your vegetables at your next meal. Eat healthily to maintain your great mood today."

[1503] Server: Sends the generated advice to the device.

[1504] Terminal: Notify the user.

[1505] In this way, this system, which combines an emotion engine, provides advice that takes the user's emotional state into consideration, enabling more comprehensive and personalized health management. Users can understand their own health and emotional state in their daily lives and improve their lifestyle habits based on the advice.

[1506] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1507] Step 1: Data collection

[1508] The device uses the following sensors to collect health and emotional data. Specifically, the device uses a pedometer to record the number of steps taken each day, a heart rate sensor to measure the heart rate, and a sleep sensor to record the amount of sleep. It also uses a camera to take photos of meals, and an emotion engine to analyze the user's facial expressions and voice to determine their emotional state for the day. For example, if a user records 5,000 steps, an average heart rate of 70 bpm, and seven hours of sleep, and then takes a photo of the meal, the device determines their emotional state as "high happiness."

[1509] Input: Raw data collected by sensors (number of steps, heart rate, sleep time, meal photos, facial expression and voice data)

[1510] Output: Collected health and emotional data (e.g., steps = 5000, heart rate = 70 bpm, sleep time = 7 hours, emotional state = high happiness)

[1511] Examples of using prompt statements

[1512] "Collect user health and emotional data."

[1513] Step 2: Send data

[1514] The device sends the collected health and emotional data to the server. Specifically, the device sends the collected data to the server every 15 minutes using encryption technology (e.g., TLS / SSL). For example, the number of steps taken (5,000), heart rate (70 bpm), sleep time (7 hours), emotional state ("high happiness"), and meal photos are sent to the server.

[1515] Input: Collected health and emotion data

[1516] Output: Data sent to the server

[1517] Examples of using prompt statements

[1518] "Please send the collected health and emotion data to the server."

[1519] Step 3: Data analysis

[1520] The server analyzes the received data. Specifically, it first analyzes the step count data and evaluates the amount of exercise. It also uses image recognition technology to calculate calorie intake from meal photos and takes into account the emotional data provided by the emotion engine. For example, if the number of steps taken in a day is less than 3,000, it determines that the person is not exercising enough, identifies the calorie intake of the meal as 800 kcal from a photo of lunch, and determines the stress level as high from the emotional data.

[1521] Input: Health and emotion data sent to the server

[1522] Output: Analysis results (lack of exercise, calorie intake 800 kcal, high stress level)

[1523] Examples of using prompt statements

[1524] "Analyze the data received and assess your health and emotional state."

[1525] Step 4: Advice Generation

[1526] The server uses the generative AI model to generate personalized health advice based on the analysis results. Specifically, it generates advice such as "You should walk a little more today" and "Take time to relax" based on the analysis results. If the calorie intake is high, it also generates specific advice such as "Increase the amount of vegetables at your next meal."

[1527] Input: Analysis results

[1528] Output: Personalized health advice (e.g., "You should walk a little more today" or "Take time to relax")

[1529] Examples of using prompt statements

[1530] "Based on the analysis results, generate appropriate health advice for the user."

[1531] Step 5: Submitting and viewing advice

[1532] The server sends the generated advice to the device. The device notifies the user of the received advice and displays it on a display or in an app. For example, a notification saying "Your heart rate tends to be high. Take time to relax" may appear on the display. Another notification saying "Your calorie intake is high, so have a lighter meal next time" may appear in the app.

[1533] Input: Generated advice

[1534] Output: Advice given to the user

[1535] Examples of using prompt statements

[1536] Send the generated advice to the device and notify the user.

[1537] (Application example 2)

[1538] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1539] Conventional health management systems collect health and emotional data and provide users with health advice, but they are unable to provide specific improvement suggestions based on users' spending patterns and lifestyle habits. Therefore, there is a need for a system that supports users in improving their lifestyle habits through healthy consumption behavior.

[1540] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1541] In this invention, the server includes a sensor means for collecting personal health data, a communication means for transmitting the collected health data and emotional data to the server, a generation AI means for analyzing the transmitted health data and emotional data and generating personalized advice, a communication means for transmitting the generated advice to a user terminal, a display means for notifying the user of the advice to be displayed on the user terminal, a data collection and analysis means for collecting and analyzing expenditure data, and a generation AI model means for providing personalized advice to promote healthy consumption based on the health data and expenditure data. This enables comprehensive health management and improvement of consumption behavior based on the user's health and emotional state and spending patterns.

[1542] "Sensor means" refers to devices for collecting personal health data, including step sensors, heart rate sensors, sleep sensors, cameras, etc.

[1543] "Communication means" refers to the technology used to transmit collected data to the server. This includes wireless communication technologies such as Wi-Fi and Bluetooth.

[1544] "Generative AI methods" are artificial intelligence technologies used to analyze submitted data and generate personalized advice, including natural language processing and machine learning algorithms.

[1545] The "display means" is a function for displaying the generated advice on the user terminal, such as the screen of a smartphone or the display of a smartwatch.

[1546] "Data collection and analysis means" refers to technology for collecting and analyzing user spending data, including electronic payment systems and purchase history management systems.

[1547] "Generative AI modeling tools" are artificial intelligence technologies, including machine learning models and generative adversarial networks (GANs), that provide personalized advice based on health and expenditure data.

[1548] This invention is a system that supports users in managing their health and improving their consumption behavior. This system collects personal health and emotional data and provides personalized health advice based on the analysis results. It also collects and analyzes the user's expenditure data and generates advice to promote healthy consumption behavior.

[1549] Hardware and software used

[1550] Sensor devices: Health data is collected through wearable devices such as the Apple Watch and Fitbit, which incorporate step, heart rate, and sleep sensors.

[1551] Communication technology: Data is transmitted using wireless communication technologies such as Wi-Fi and Bluetooth, which allows for smooth data transfer from the device to the server.

[1552] Server: AWS Lambda and AWS RDS are used to analyze and store data. AWS S3 is used to temporarily store and encrypt data.

[1553] Generative AI model: OpenAI's GPT-3 is used to generate personalized health advice, and Affectiva SDK is used to analyze emotion data.

[1554] System processing overview

[1555] The device collects personal health data (step count, heart rate, sleep data) and emotional data. Emotional data is extracted from the user's facial expressions and voice using the Affectiva SDK. This data is sent to a server at regular intervals. The sent data is stored in AWS S3 via AWS Lambda. The stored data is then integrated into a database using AWS RDS.

[1556] The server analyzes the health and emotional data and generates personalized health advice using a generative AI model (GPT-3). For example, if a user's step count is low and emotional data indicates a high stress level, the server generates advice such as, "You should walk a little more today. Take time to relax."

[1557] Spending data will also be collected and integrated with health data for analysis. Purchase history collected through electronic payment systems will be used to analyze consumer behavior. This will generate specific, healthy spending advice, such as, "Your food expenses have tended to be high this month. Consider making healthy choices the next time you buy food."

[1558] The generated advice is sent to the user's device in real time, allowing the user to improve their health and consumption behavior in their daily lives.

[1559] Specific examples

[1560] For example, if a user only walked 3,000 steps on a particular day and the emotion engine detects that the user's stress level is high, the following example prompts will be fed into the generative AI model:

[1561] Example prompt sentence:

[1562] Health data: Steps: 3000, Heart rate: 80 / min, Sleep time: 6 hours

[1563] Emotional data: Stress level: High, Mood: Unstable

[1564] Generation advice: With your step count low and stress levels high, take time to relax and take a short walk to refresh your mind.

[1565] In this way, the generative AI model takes into account the user's health and emotional state to provide appropriate advice. Furthermore, the user's spending data is also analyzed, and specific instructions are given to promote a healthy lifestyle. For example, advice such as "This month's food expenses are above average. Next time, choose a menu that includes lots of vegetables" is provided.

[1566] By using this system, users can improve their overall health management and consumption habits, leading to a healthier lifestyle.

[1567] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1568] Step 1:

[1569] The device collects the user's health data (step count, heart rate, sleep data) and emotional data. The sensor device (e.g., wearable device) used measures the user's biometric data in real time. The input is the user's biometric data, and the output is the collected health data and emotional data.

[1570] Step 2:

[1571] The device sends the collected health and emotion data to a server via Wi-Fi or Bluetooth. The input is the collected data, and the output is the encrypted data sent to the server.

[1572] Step 3:

[1573] The server uses AWS Lambda to temporarily store the submitted data in AWS S3, where the input is the data submitted to the server and the output is the data stored in AWS S3.

[1574] Step 4:

[1575] The server uses AWS RDS to integrate data stored in S3 into a database, where the input is data stored in AWS S3 and the output is the integrated data in the database.

[1576] Step 5:

[1577] The server uses AWS Lambda to analyze the health and emotional data. The analysis is performed using the Affectiva SDK. The input is the data integrated into the database, and the output is the analyzed results of the user's health and emotional state.

[1578] Step 6:

[1579] The server uses a generative AI model (GPT-3) to generate personalized health advice based on the analysis results, where the input is the health and emotional state, and the output is the generated personalized health advice.

[1580] Step 7:

[1581] The server stores the generated individual health advice in AWS RDS and simultaneously sends it to the user's device. The input here is the generated advice, and the output is a notification of the advice to the device.

[1582] Step 8:

[1583] The user terminal notifies the user of the generated advice and displays it on the display. The input here is the advice sent from the server, and the output is the advice displayed to the user.

[1584] Step 9:

[1585] The terminal collects the user's spending data from the electronic payment system and sends it to the server. At this stage, the input is the user's purchase history and the output is the collected spending data.

[1586] Step 10:

[1587] The server analyzes the expenditure data and integrates it with the health data for analysis, where the inputs are expenditure data and health data, and the output is the integrated analysis results.

[1588] Step 11:

[1589] Based on the integrated analysis results, the server generates advice to promote healthy spending behaviors using a generative AI model, where the input is the integrated analysis results and the output is the generated spending advice.

[1590] Step 12:

[1591] The server stores the generated spending advice in AWS RDS and sends it to the user's device. The input here is the generated spending advice, and the output is a notification of the advice to the device.

[1592] Step 13:

[1593] The user terminal notifies the generated spending advice to the user and displays it on a display, where the input is the spending advice sent from the server and the output is the advice displayed to the user.

[1594] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1595] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1596] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1597] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1598] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1599] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1600] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1601] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1602] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1603] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1604] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1605] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1606] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1607] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1608] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1609] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1610] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1611] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1612] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1613] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1614] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1615] The following is further disclosed regarding the above embodiment.

[1616] (Claim 1)

[1617] sensor means for collecting personal health data;

[1618] a communication means for transmitting the collected health data to a server;

[1619] A generative AI means for analyzing the transmitted health data and generating personalized advice;

[1620] a communication means for transmitting the generated advice to a user terminal;

[1621] a display means for notifying the user of the advice to be displayed on the user terminal;

[1622] A system including:

[1623] (Claim 2)

[1624] 10. The system of claim 1, further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

[1625] (Claim 3)

[1626] 10. The system of claim 1, further comprising a data accumulation means for providing long-term health management advice based on the collected health data and calorie intake data.

[1627] "Example 1"

[1628] (Claim 1)

[1629] sensor means for collecting personal health data;

[1630] a communication means for transmitting the collected health data to a server;

[1631] A generative AI means for analyzing the transmitted health data and generating personalized advice;

[1632] a generative AI model means for inputting a prompt sentence into the generative AI model and generating health advice;

[1633] a communication means for transmitting the generated advice to a user terminal;

[1634] a display means for notifying the user of the advice to be displayed on the user terminal;

[1635] A system including:

[1636] (Claim 2)

[1637] 10. The system of claim 1, further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

[1638] (Claim 3)

[1639] 10. The system of claim 1, further comprising a data accumulation means for providing long-term health management advice based on the collected health data and calorie intake data.

[1640] "Application Example 1"

[1641] (Claim 1)

[1642] sensor means for collecting personal health data;

[1643] a communication means for transmitting the collected health data to a server;

[1644] A generative AI means for analyzing the transmitted health data and generating personalized advice;

[1645] a communication means for transmitting the generated advice to a user terminal;

[1646] A system including a display means for notifying advice to be displayed on a user terminal,

[1647] The system further includes a suggestion means for suggesting optimal meal options based on the user's health data.

[1648] (Claim 2)

[1649] 10. The system of claim 1, further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

[1650] (Claim 3)

[1651] 10. The system of claim 1, further comprising a data accumulation means for providing long-term health management advice based on the collected health data and calorie intake data.

[1652] "Example 2: Combining Emotion Engines"

[1653] (Claim 1)

[1654] sensor means for collecting personal health and emotional data;

[1655] communication means for transmitting the collected health data and emotion data to a server;

[1656] A generative AI means for analyzing the transmitted health data and emotional data and generating personalized advice;

[1657] a communication means for transmitting the generated advice to a user terminal;

[1658] a display means for notifying the user of the advice to be displayed on the user terminal;

[1659] A system including:

[1660] (Claim 2)

[1661] 10. The system of claim 1, further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

[1662] (Claim 3)

[1663] 10. The system of claim 1, further comprising a data accumulation means for providing long-term health management advice based on the collected health data, emotional data, and calorie intake data.

[1664] "Application example 2 when combining emotion engines"

[1665] (Claim 1)

[1666] sensor means for collecting personal health data;

[1667] communication means for transmitting the collected health data and emotion data to a server;

[1668] A generative AI means for analyzing the transmitted health data and emotional data and generating personalized advice;

[1669] a communication means for transmitting the generated advice to a user terminal;

[1670] a display means for notifying the user of the advice to be displayed on the user terminal;

[1671] a data collection and analysis tool for collecting and analyzing expenditure data;

[1672] a generative AI model means for providing personalized advice to promote healthy consumption based on health data and expenditure data;

[1673] A system including:

[1674] (Claim 2)

[1675] 10. The system of claim 1, further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

[1676] (Claim 3)

[1677] 10. The system of claim 1, further comprising a data accumulation means for providing long-term health management advice based on the collected health data and calorie intake data. [Explanation of symbols]

[1678] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. sensor means for collecting personal health data; a communication means for transmitting the collected health data to a server; A generative AI means for analyzing the transmitted health data and generating personalized advice; a communication means for transmitting the generated advice to a user terminal; a display means for notifying the user of the advice to be displayed on the user terminal; A system including:

2. The system of claim 1 , further comprising image recognition means for analyzing the collected dietary data and calculating calorie intake.

3. 10. The system of claim 1, further comprising a data storage means for providing long-term health management advice based on the collected health data and calorie intake data.

Images