System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining physiological information of a user; means for calculating calories burned by the user based on the obtained physiological information; means for generating an exercise plan based on a goal self-set by the user; means for presenting the generated exercise plan to the user; and means for monitoring progress of the user and adaptively updating the exercise plan as needed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today, users face the challenge of consistently exercising. In particular, it is difficult for users to maintain an exercise plan tailored to their goals and monitor their progress. As a result, users often give up midway through their exercise routine. It is also difficult to provide personalized exercise plans that reflect the physiological and nutritional information that varies for each individual user. To address these challenges, the present invention develops a system that analyzes a user's physiological information and dietary habits to provide an individually optimized exercise plan. [Means for solving the problem]

[0005] The present invention solves the above problems by providing the following means: A means for acquiring a user's physiological information is provided. A means for calculating the user's calorie expenditure based on the acquired physiological information is provided. A means for generating an exercise plan based on the user's self-set goals is also provided. This allows the user to receive a personalized exercise plan. A means for presenting the generated exercise plan to the user and a means for monitoring the user's progress and adaptively updating the exercise plan as needed are also provided. This makes it easier for the user to continuously exercise toward their goal. A means for analyzing the user's diet and a means for incorporating the analyzed nutritional information of the diet into the exercise plan are also provided, enabling more precise feedback. A means for acquiring the user's physiological information in real time using a wearable device and adjusting exercise intensity based on the acquired physiological information is also provided. This provides real-time feedback, allowing the user to always perform optimal exercise.

[0006] "User" refers to an individual who utilizes the system to receive and execute an exercise plan.

[0007] "Physiological information" refers to data that indicates the user's internal physical condition and activity level, such as heart rate, body temperature, and number of steps taken.

[0008] "Calories burned" refers to the amount of energy consumed by a user through exercise.

[0009] An "exercise plan" refers to the specific exercise content and schedule generated by AI to help users achieve their goals.

[0010] "Self-defined goals" refers to fitness goals that a user sets within the system that they want to achieve, such as losing weight or gaining muscle.

[0011] "Analysis" refers to the process in which AI processes images and data of meals and exercise to extract information such as nutrients and calories burned.

[0012] "Personalized" refers to exercise plans and feedback that are customized to each user's specific needs and conditions.

[0013] A "wearable device" refers to an electronic device that can acquire physiological information in real time by being worn by the user.

[0014] "Real-time" means that data is processed and used almost immediately after it is generated.

[0015] "Monitoring" refers to the process of continuously observing, recording, and evaluating a user's progress and status. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] Overall overview

[0038] This system is designed to help users maintain their exercise habits. Users use a smartphone app and a wearable device to collect their exercise and dietary data, which is then analyzed by a server to provide an individually optimized exercise plan. The plan is dynamically updated based on the user's goals.

[0039] Data collection

[0040] Users take photos of their meals using a smartphone app. The device (smartphone) then sends these photos to a server. Users also wear a wearable device while exercising daily, and the data is sent to the server via the device. The wearable device collects physiological information such as heart rate, steps, and distance traveled in real time. By collecting this data, the server can grasp the user's condition in detail.

[0041] Data analysis

[0042] The server uses an AI algorithm to analyze the received photos of the meal. Specifically, it uses image recognition technology to identify the contents of the meal and extract information about each nutrient. It also analyzes data sent from the wearable device to calculate the type and intensity of exercise and calories burned. This allows for detailed measurements of the user's calories burned and nutritional intake.

[0043] Generate exercise plans

[0044] Based on the user's self-defined goals (e.g., weight loss or muscle gain), the server generates an exercise plan. This plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level. The generated plan details the type, duration, and frequency of exercise.

[0045] Feedback and Monitoring

[0046] The server sends the generated exercise plan to the device, which then presents it to the user. The user then exercises according to the presented plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. This allows the user to continue exercising effectively toward their set goals.

[0047] Specific examples

[0048] Examples of data collection

[0049] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0050] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0051] Specific examples of data analysis

[0052] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0053] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0054] Example of generating an exercise plan

[0055] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of light jogging on Monday and 45 minutes of strength training on Tuesday."

[0056] The terminal notifies the user of this exercise plan and displays it as feedback.

[0057] Examples of feedback and monitoring

[0058] The user follows the proposed plan and jogs for 30 minutes on Monday.

[0059] The terminal transmits data of the jogging that has been performed to the server.

[0060] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the device.

[0061] In this way, the system manages the user's behavior and physiological information in an integrated manner, supporting continuous exercise and health management.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] Users open the app on their smartphone and take a photo of their meal.

[0065] The device (smartphone) automatically sends the photograph of the meal to the server.

[0066] Step 2:

[0067] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[0068] The collected data is sent to the server in real time via the terminal.

[0069] Step 3:

[0070] The server analyzes the received meal photos using an AI algorithm to identify the meal contents and extract information on each nutrient (e.g., calories, protein, fat, and carbohydrate content).

[0071] Step 4:

[0072] The server analyzes the exercise data sent from the wearable device, including the type of exercise (e.g., running, walking), exercise intensity, and calorie consumption calculations.

[0073] Step 5:

[0074] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[0075] This exercise plan includes specific exercise content (e.g., exercise schedule for each day of the week), time, frequency, etc.

[0076] Step 6:

[0077] The server transmits the generated exercise plan to the terminal.

[0078] The device will notify the user of the exercise plan and display it as feedback, where the user can see the details of the exercise plan (e.g., 30 minutes of jogging on Mondays).

[0079] Step 7:

[0080] The user follows the notified exercise plan and performs the exercise.

[0081] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[0082] Step 8:

[0083] The server periodically monitors the user's progress, including analyzing changes in body composition and exercise performance.

[0084] Based on the analysis results, the exercise plan is adjusted as needed.

[0085] Step 9:

[0086] The server again transmits the updated exercise plan to the terminal.

[0087] The device notifies the user of the updated exercise plan and displays it as feedback, and the user continues exercising again based on the new plan.

[0088] Example 1

[0089] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0090] Conventional exercise plan generation systems only acquire a user's physiological information and generate an exercise plan based on that information, making it difficult to provide a plan optimized for the user's self-set goals. In particular, they lack real-time feedback and adaptive updating of exercise plans, and efficient exercise guidance based on the user's progress is required.

[0091] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0092] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for utilizing a generative AI model to generate an exercise plan based on the user's self-set goals, means for presenting the generated exercise plan to the user, and means for monitoring the user's progress and adaptively updating the exercise plan as needed, thereby providing an exercise plan optimized for each individual user and enabling real-time feedback and adaptive plan changes.

[0093] "User's physiological information" is data that indicates the user's physical activity and health condition, such as heart rate, number of steps, and distance traveled.

[0094] The "means for calculating calories burned" is a device or system that calculates the amount of energy burned by the user through exercise based on the acquired physiological information.

[0095] A "generative AI model" is an artificial intelligence algorithm used to generate an optimized exercise plan based on a user's goals and current exercise data.

[0096] "Image recognition technology" is a technology for analyzing photographs and image data and identifying the objects and content contained within them.

[0097] An "exercise plan" is a specific exercise content and schedule that is planned based on the goal that the user wants to achieve.

[0098] "Monitoring" is the process of continuously observing and collecting data about a user's progress and exercise performance.

[0099] An "adaptive updating means" is a device or system for dynamically changing or adjusting an exercise plan based on the user's latest progress and feedback.

[0100] A "wearable device" is a small electronic device that can be worn by a user and is used to collect physiological information in real time.

[0101] System configuration

[0102] This system is designed to help users maintain their exercise habits and is primarily composed of a smartphone app and wearable device used by the user, as well as a server that analyzes data and generates exercise plans.

[0103] Data collection details

[0104] The user takes a photo of their meal using a smartphone app, and the device (smartphone) sends the photo to a server. The wearable device collects physiological information such as the user's heart rate, number of steps, and distance traveled in real time, and sends the data to the server via the smartphone. This data collection uses the smartphone's camera function and common communication methods such as Bluetooth and Wi-Fi.

[0105] Data analysis details

[0106] The server analyzes the received food photos using AI algorithms (e.g., Google Cloud Vision API) to identify the meal contents and extract information on each nutrient. It also uses Python and TensorFlow to analyze physiological data sent from the wearable device and calculate the type and intensity of exercise and calories burned.

[0107] Generate exercise plans

[0108] The server generates an exercise plan based on the user's set goals (e.g., weight loss or muscle gain). This process uses a generative AI model (e.g., OpenAI GPT-4). The generated exercise plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level.

[0109] Feedback and Monitoring

[0110] The server sends the generated exercise plan to the device, which then presents it to the user. As the user exercises according to the plan, the data is collected again and sent to the server. The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed.

[0111] Specific examples

[0112] Examples of data collection

[0113] When a user eats an omelet for breakfast, they take a photo of it with their smartphone and send it to the server via the app.

[0114] The wearable device on the terminal measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0115] Specific examples of data analysis

[0116] The server analyzes the breakfast photo and extracts nutritional information (e.g., calories, protein, fat, and carbohydrate content). An example of a specific prompt is, "Please identify the nutrients contained in this meal."

[0117] The server analyzes the jogging data acquired from the wearable device and calculates the calorie consumption of the user. An example of a specific prompt sentence is "Please calculate the calorie consumption from this jogging data."

[0118] Example of generating an exercise plan

[0119] The server creates a weekly exercise plan based on the user's weight loss goal. Specific exercise content is suggested using a generative AI model. An example prompt is, "Please generate an exercise plan that will help the user achieve their weight goal."

[0120] The device notifies the user of the generated exercise plan and displays feedback as a dashboard.

[0121] Examples of feedback and monitoring

[0122] The user goes for a 30-minute jog on Monday, and the data is collected by the wearable device and sent to the server via the smartphone.

[0123] The server analyzes the acquired jogging data, evaluates the user's progress based on their heart rate and calories burned, and adjusts the exercise plan for the next week and sends it back to the device.

[0124] Through these processes, the system comprehensively manages the user's behavior and physiological information, supporting continuous exercise and health management.

[0125] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0126] Step 1: Data collection

[0127] A user takes a photo of a meal using a smartphone app. The input is the photo of the meal, and the output is the photo data.

[0128] The device (smartphone) previews the captured photo within the app and asks the user for confirmation. Once the user has finished checking, they press the "Send" button to send the photo data to the server. Specifically, data is sent using the HTTP protocol.

[0129] When a user starts exercising, the terminal (wearable device) collects physiological information such as heart rate, number of steps, and distance traveled in real time. The input is the user's physiological information, and the output is the collected physiological data.

[0130] The device (smartphone) sends the collected physiological data in bulk to a server. The data is synchronized to the smartphone using Bluetooth or Wi-Fi, and then transmitted to the server using the HTTP protocol.

[0131] Step 2: Data analysis

[0132] The server analyzes the received meal photos using image recognition technology (e.g., Google Cloud Vision API). The input is the photo data of the meal, and the output is the analysis results, such as the types and amounts of ingredients. For example, the prompt might be, "Please identify the nutrients contained in this meal."

[0133] The server extracts nutrient information (calories, protein, fat, carbohydrates) based on the image recognition results. Specifically, it identifies the nutrient information by referencing a food database.

[0134] The server analyzes physiological data (heart rate, steps, distance traveled) sent from the wearable device. The input is physiological data, and the output is analysis results such as the type and intensity of exercise, and calories burned. Python and TensorFlow are used to clean and preprocess the data, and a machine learning model (e.g., regression model) is used to calculate calories burned. The prompt is "Please calculate calories burned from this jogging data."

[0135] Step 3: Generate an exercise plan

[0136] The server generates an exercise plan based on the user's self-defined goals (e.g., weight loss or muscle gain). The input is the user's goals and analyzed physiological and nutritional data, and the output is a customized exercise plan.

[0137] A generative AI model (e.g., OpenAI GPT-4) is used to create detailed exercise plans. The prompt is, "Generate an exercise plan that will help the user achieve their goal weight."

[0138] The server generates a plan that takes into account the type of exercise (e.g., jogging, strength training), time, and frequency, and sends it to the device in JSON format.

[0139] Step 4: Feedback and monitoring

[0140] The terminal notifies the user of the exercise plan received from the server. The input is the generated exercise plan, and the output is the notification of the exercise plan presented to the user.

[0141] The user exercises based on the presented plan. After the exercise is completed, the wearable device collects data again and sends it to the server via the smartphone. The input is the exercise data performed, and the output is an updated database in the server.

[0142] The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed. The prompt is, "Please adjust next week's exercise plan based on your exercise progress." The updated exercise plan is then notified to the user via their device.

[0143] (Application example 1)

[0144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] In today's busy lifestyles, it is difficult for users to find and consistently follow an exercise plan that is optimal for them. Furthermore, balancing proper nutritional management and exercise is not easy, and users require specialized knowledge to achieve effective results. Therefore, there is a need for a system that provides individually customized exercise plans and nutritional management and effectively supports users in achieving their goals.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0147] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as needed, means for analyzing photo data of meals to extract nutrient information, and means for providing training feedback in real time. This allows users to receive an optimal exercise plan and appropriate nutritional management even without specialized knowledge, and to be effectively supported continuously toward achieving their goals.

[0148] "User's physiological information" refers to data related to the user's physical activity, such as heart rate, number of steps taken, and distance traveled.

[0149] "Calories burned" refers to the amount of energy consumed by a user through exercise and daily activities.

[0150] An "exercise plan" is a program that plans specific exercise content and schedules based on goals set by the user.

[0151] "Nutrient information" refers to information on ingredients such as calories, protein, fat, and carbohydrates extracted from dietary data.

[0152] "Real-time" refers to a process that instantly collects data about the user's ongoing activities and provides analysis results.

[0153] "Feedback" refers to evaluations and advice returned based on a user's activities, providing information that will be useful for the next action.

[0154] "Adaptive updating" refers to dynamically changing and adjusting the exercise plan according to the user's progress and condition.

[0155] A "wearable device" is a device that can be worn by a user and can measure physiological information such as heart rate and number of steps.

[0156] A "smartphone" is an evolved form of a mobile phone, a multi-function device that can connect to the Internet and use applications.

[0157] "Smart glasses" are devices that have the shape of glasses, are equipped with a display and sensors, and can present information to the user in real time.

[0158] A "server" is a computer that stores, manages, and processes data over a network, and serves as the central hub of the entire system.

[0159] Overall system overview

[0160] The system is designed to encourage users to continue their exercise habits and provide individually customized exercise plans and nutritional management. The system operates using the user's smartphone, wearable device, server, smart glasses, or head-mounted display.

[0161] Hardware and software used

[0162] Hardware

[0163] 1. Smartphone: Used for data collection and as a user interface.

[0164] 2. Wearable devices: Collect the user's physiological information (heart rate, steps, distance, etc.) in real time.

[0165] 3. Smart glasses or head-mounted displays: display real-time training feedback and provide exercise instructions to the user.

[0166] 4. Server: Analyzes data, generates exercise plans, and monitors user progress.

[0167] software

[0168] 1. Flask: Used as a server-side framework.

[0169] 2. Keras: Uses deep learning models (generative AI models) for image recognition and data analysis.

[0170] 3. OpenCV: Used as an image processing library.

[0171] System processing flow

[0172] Data collection

[0173] Users use a smartphone app to take photos of their meals and send them to a server. The wearable device also collects physiological information such as heart rate and number of steps taken while the user is exercising, and sends the data to the server via the smartphone.

[0174] Data analysis

[0175] The server receives photo data of meals using Flask and performs image analysis using a Keras model. Specifically, it identifies the contents of the meal and extracts information about each nutrient. Exercise data is processed using OpenCV to calculate the user's calories burned and exercise intensity.

[0176] Generate exercise plans

[0177] The server generates a personalized exercise plan based on the user's set goals (e.g., weight loss, muscle gain), detailing the type, duration, and frequency of exercise, and dynamically updates the plan to fit the user's daily routine.

[0178] Feedback and Monitoring

[0179] The server presents the generated exercise plan to the user via a smartphone or smart glasses. The user's progress is continuously monitored via the wearable device, and the exercise plan is adaptively updated as needed.

[0180] Specific examples

[0181] 1. Specific examples of data collection

[0182] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0183] The wearable device measures the user's heart rate and distance traveled while jogging and transmits the data to a server.

[0184] 2. Specific examples of data analysis

[0185] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0186] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0187] 3. Example of motor plan generation

[0188] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of jogging on Monday and 45 minutes of strength training on Tuesday."

[0189] The smartphone notifies the user of this exercise plan and displays it as feedback.

[0190] 4. Examples of feedback and monitoring

[0191] The user follows the proposed plan and jogs for 30 minutes on Monday.

[0192] The wearable device transmits data about the jogging that has been performed to a server.

[0193] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the smartphone.

[0194] Examples of prompt statements

[0195] Analyze the user's breakfast photo data and extract the nutrients (calories, protein, fat, carbohydrates, etc.) contained in it. Also, analyze the previous day's jogging data and calculate the calories burned. Generate a new exercise plan based on the user's weight loss goal, including "30 minutes of jogging on Monday" and "45 minutes of strength training on Tuesday."

[0196] In this way, a system is constructed by combining specific hardware and software to provide optimal exercise and nutrition management for users to achieve their goals.

[0197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0198] Step 1:

[0199] Data collection

[0200] Input: A user opens a smartphone app and takes a photo of their breakfast, while a wearable device monitors their heart rate, steps taken, and distance traveled in real time.

[0201] Data processing / calculation: The smartphone sends the photograph of the meal to the server via the app. The wearable device sends the measurement data to the server via the smartphone.

[0202] Output: The server receives the food photo data and physiological information data.

[0203] Specific operation: The user operates the app and takes a photo. The wearable device automatically records the measurement data and sends it to a smartphone.

[0204] Step 2:

[0205] Data Analysis - Dietary Data

[0206] Input: The server receives a photo of a meal sent by the user.

[0207] Data processing / calculation: The server uses a Keras model to analyze the photo data and identify the food contents. It then uses OpenCV to perform image processing and extract nutritional information such as calories, protein, fat, and carbohydrates.

[0208] Output: Extracted nutrient information is obtained.

[0209] Specific operation: The server uses an AI model to recognize foods in an image and output specific nutritional data.

[0210] Step 3:

[0211] Data Analysis - Exercise Data

[0212] Input: The server receives the wearable device movement data sent by the user.

[0213] Data processing / calculation: The server calculates the user's calories burned and exercise intensity based on the received physiological information.

[0214] Output: Calculated calorie consumption and exercise intensity data are obtained.

[0215] Specific operation: The server analyzes the received data and stores the calculation results.

[0216] Step 4:

[0217] Exercise plan generation

[0218] Input: The server receives the analyzed nutrition information, exercise data, and user-defined goals.

[0219] Data processing / calculation: The server uses this data to generate an exercise plan that is optimal for the user's goals. The generative AI model dynamically adjusts the exercise content and schedule.

[0220] Output: A customized exercise plan is generated.

[0221] Specific operation: The server uses the AI ​​model to combine the data analysis results, calculate and output the optimal plan.

[0222] Step 5:

[0223] Plan presentation

[0224] Input: Server-generated exercise plan.

[0225] Data processing / calculation: The exercise plan is sent from the server to the device (smartphone, smart glasses, etc.).

[0226] Output: The exercise plan is displayed on the user's device.

[0227] Specific operation: The server sends the plan data along with the notification message to the terminal, which displays it through the user interface.

[0228] Step 6:

[0229] Feedback and progress monitoring

[0230] Input: User progress data (exercise performance, heart rate variability, etc.) is collected from a wearable device.

[0231] Data processing / calculation: The server compares the planned exercise content with the actual activity based on the progress data and adjusts the exercise plan if necessary.

[0232] Output: A new, adjusted exercise plan.

[0233] How it works: The server analyzes the progress data, adjusts the plan using an AI model, and sends the new plan back to the device.

[0234] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0235] Overall overview

[0236] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect their exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[0237] Data collection

[0238] Users take photos of their meals using a smartphone app. They also use a wearable device to collect daily exercise data. The smartphone app then uses the camera and voice input to recognize the user's emotions. The device then sends this data to a server, which then collects detailed user data.

[0239] Data analysis

[0240] The server uses an AI algorithm to analyze the received food photos and extract nutritional information from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state.

[0241] Generate exercise plans

[0242] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if stress levels are high, the server may recommend exercise that has a relaxing effect. The generated plan includes detailed exercise content, schedule, and frequency.

[0243] Feedback and Monitoring

[0244] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[0245] Specific examples

[0246] Examples of data collection

[0247] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0248] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0249] Users can report their emotional state after exercise using a smartphone app, or the app can automatically collect data by analyzing emotions through facial recognition and voice tone.

[0250] Specific examples of data analysis

[0251] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0252] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0253] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[0254] Example of generating an exercise plan

[0255] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, the server might generate a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[0256] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[0257] Examples of feedback and monitoring

[0258] The user follows the plan and does 30 minutes of yoga on Mondays.

[0259] The terminal transmits data on the yoga session performed and subsequent emotional data (for example, stress reduction effect) to the server.

[0260] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[0261] In this way, the system manages the user's behavior, physiological information, and emotional information in an integrated manner, supporting continuous exercise and health management.

[0262] The processing flow will be explained below.

[0263] Step 1:

[0264] Users open the app on their smartphone and take a photo of their meal.

[0265] The device (smartphone) automatically sends the photograph of the meal to the server.

[0266] Step 2:

[0267] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[0268] The collected data is sent to the server in real time via the terminal.

[0269] Step 3:

[0270] Users can report their emotional state after exercise in the app, or the app will automatically analyze emotions through facial recognition and voice tone.

[0271] When a user taps an emotion button, the emotion information is also sent to the server.

[0272] Step 4:

[0273] The server uses an AI algorithm to analyze the received photos of the meal and identify the contents of the meal.

[0274] It also extracts information about each nutrient (e.g., amount of calories, protein, fat, carbohydrates).

[0275] Step 5:

[0276] The server analyzes the exercise data sent from the wearable device.

[0277] This includes calculating the type of exercise (e.g., running, walking), the intensity of the exercise, and the calories burned.

[0278] Step 6:

[0279] The server analyzes the received emotion data using an emotion engine.

[0280] Identify and assess your current emotional state (e.g., stress level, satisfaction, fatigue, etc.).

[0281] Step 7:

[0282] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[0283] This exercise plan is customized taking into account your emotional state as well as your physical data.

[0284] Step 8:

[0285] The server transmits the generated exercise plan to the terminal.

[0286] The device notifies the user of the exercise plan and displays detailed exercise content and schedule as feedback.

[0287] Step 9:

[0288] The user follows the notified exercise plan and performs the exercise.

[0289] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[0290] Step 10:

[0291] The server periodically monitors the user's progress.

[0292] This includes analyzing changes in body composition and exercise outcomes (e.g., weight loss, muscle mass gain).

[0293] Step 11:

[0294] The server uses an emotion engine to analyze the user's emotion data and also monitor the user's psychological state.

[0295] Adaptively update your exercise plan and make real-time adjustments as needed.

[0296] Step 12:

[0297] The server then transmits the updated exercise plan to the terminal again.

[0298] The device notifies the user of the updated exercise plan and presents detailed exercise content and schedule again as feedback.

[0299] Step 13:

[0300] The user then exercises again based on the new exercise plan and continues to receive feedback and monitoring.

[0301] By linking the device and the server, the user's physical and emotional data are managed in an integrated manner, allowing optimal exercise instruction to be provided.

[0302] Example 2

[0303] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] Conventional systems primarily generate exercise plans based on the user's physiological information and exercise data, but because they do not take the user's emotional state into account, the exercise plan is not optimized for the user's individual psychological state. This can reduce the user's motivation to continue exercising, potentially reducing the effectiveness of health management.

[0305] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the calories burned by the user based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for acquiring and analyzing emotional information of the user, and means for adjusting the exercise plan based on the analyzed emotional information. This makes it possible to provide an optimal exercise plan according to the user's individual psychological state.

[0306] 1. "User" refers to a person who uses this system to provide exercise, diet, and emotional data and receives feedback based on that data.

[0307] 2. "Physiological information" refers to the user's physical data, such as heart rate, calories burned, and type and intensity of exercise.

[0308] 3. "Exercise Plan" refers to a personalized exercise schedule or program generated based on a user's physiological information, goals, and emotional information.

[0309] 4. "Emotional information" refers to data that indicates a user's emotional state (e.g., stress, satisfaction, etc.), and is primarily collected through facial recognition, voice tone analysis, or user self-reporting.

[0310] 5. "Server" refers to a computer system that receives and analyzes data sent from users and devices, and provides the generated exercise plan to users.

[0311] 6. "Device" refers to an electronic device, such as a smartphone or wearable device, that is used to collect and transmit user data to a server.

[0312] 7. "Means for acquiring" refers to methods and devices for collecting physiological and emotional information of a user, such as sensors, cameras, microphones, etc.

[0313] 8. "Analytical means" refers to the algorithms and software used to process the collected data and evaluate and analyze the user's condition.

[0314] 9. "Means for generating" refers to the method or algorithm for generating an exercise plan based on the analysis results.

[0315] 10. "Presentation means" refers to a method or device for notifying the user of the generated exercise plan, such as a smartphone notification function or app.

[0316] 11. "Monitoring means" refers to the methods and functions that continuously observe and collect user progress and data, and adjust exercise plans as needed.

[0317] Overall overview

[0318] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[0319] Data collection

[0320] Users use a smartphone app to take photos of their meals, which are then analyzed to record the details of their meals. The user also uses a wearable device to collect daily exercise data, including heart rate, distance traveled, and calories burned. Furthermore, the smartphone app recognizes the user's emotions through camera and voice input. All of this data is sent from the device to a server, where detailed user data is collected.

[0321] Data analysis

[0322] The server analyzes the received food photos using AI algorithms (e.g., TensorFlow or PyTorch) and extracts nutritional information from the meal contents. At the same time, it analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. It uses an emotion engine to analyze the user's emotional data and identify their current psychological state. This allows the server to comprehensively evaluate the user's physiological information, exercise data, and emotional data.

[0323] Generate exercise plans

[0324] The server generates an exercise plan based on the user's self-set goals (e.g., target weight, exercise time) and emotional data. This exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if the user is in a high-stress state, it will recommend exercise that has a relaxing effect (e.g., yoga or relaxing walking). The generated exercise plan includes detailed exercise content, schedule, and frequency.

[0325] Feedback and Monitoring

[0326] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[0327] Specific examples

[0328] Examples of data collection

[0329] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0330] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0331] Users report their emotional state after exercise using a smartphone app, which can automatically collect emotions using facial recognition and voice tone analysis.

[0332] Specific examples of data analysis

[0333] The server analyzes photos of breakfast and uses AI algorithms to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate amounts).

[0334] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0335] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[0336] Example of generating an exercise plan

[0337] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, it generates a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[0338] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[0339] Examples of feedback and monitoring

[0340] The user follows the plan and does 30 minutes of yoga on Mondays.

[0341] The terminal transmits data on the yoga performed and subsequent emotional data (e.g., stress reduction effect) to the server.

[0342] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[0343] This system uses a generative AI model to integrate and analyze the user's physiological and emotional information, generate an individually optimized exercise plan, and provide feedback to support continuous exercise and health management.

[0344] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0345] Step 1:

[0346] A user takes a photo of their meal using a smartphone app.

[0347] Input: Food photo

[0348] Output: Food photo data sent to a smartphone app

[0349] How it works: A user opens a smartphone app, takes a photo of a meal, and sends the data through the app.

[0350] Step 2:

[0351] A user wears a wearable device to collect exercise data.

[0352] Input: Physiological data during exercise (heart rate, distance traveled, etc.)

[0353] Output: Exercise data recorded on a wearable device

[0354] How it works: A user puts on the wearable device and starts exercising. The device continuously records exercise data.

[0355] Step 3:

[0356] A smartphone app recognizes the user's emotional state.

[0357] Input: User facial expressions and tone of voice

[0358] Output: Emotional state data

[0359] How it works: A smartphone app uses the camera and microphone to analyze the user's facial expressions and voice to determine their emotional state.

[0360] Step 4:

[0361] The device sends the collected data (meal photos, exercise data, emotional data) to the server.

[0362] Input: Food photo data, exercise data, emotion data

[0363] Output: Detailed user data sent to the server

[0364] How it works: Your device sends data over the internet to a server.

[0365] Step 5:

[0366] The server analyzes the received food photos and extracts nutritional information.

[0367] Input: Food photo data

[0368] Output: Nutrition information (calories, protein, fat, carbohydrates, etc.)

[0369] How it works: The server uses AI algorithms to analyze the image, identify the contents of the meal, and extract nutritional information.

[0370] Step 6:

[0371] The server analyzes the received exercise data and calculates the calories burned.

[0372] Input: Exercise data (heart rate, distance traveled, etc.)

[0373] Output: Calories burned, exercise type, intensity, etc.

[0374] Operation: The server analyzes the exercise data and performs various data calculations to calculate detailed exercise information and calories burned.

[0375] Step 7:

[0376] The server uses an emotion engine to analyze the user's emotion data.

[0377] Input: Emotional state data

[0378] Output: Current psychological state (e.g., stress level, satisfaction, etc.)

[0379] How it works: The server uses the emotion engine to analyze the user's emotion data and identify their psychological state.

[0380] Step 8:

[0381] The server generates an exercise plan based on the user's goals and the analyzed data.

[0382] Input: Goal data, Nutrition information, Exercise data, Emotion data

[0383] Output: Individually optimized exercise plans

[0384] How it works: The server comprehensively analyzes all the data it collects and creates the optimal exercise plan for the user.

[0385] Step 9:

[0386] The server sends the generated exercise plan to the terminal.

[0387] Input: Personalized exercise plan

[0388] Output: Exercise plan sent to device

[0389] Operation: The server sends the generated exercise plan to the device and notifies the user.

[0390] Step 10:

[0391] The user exercises according to the exercise plan presented to them.

[0392] Input: Exercise plan

[0393] Output: Exercise data performed

[0394] Action: The user performs the specified exercise according to the exercise plan received from the device.

[0395] Step 11:

[0396] The terminal transmits the exercise data and emotion data that have been performed to the server again.

[0397] Input: Exercise data, emotion data

[0398] Output: Data to be resent to the server

[0399] Operation: The device sends the exercise data and emotion data back to the server.

[0400] Step 12:

[0401] The server analyzes the collected data and adaptively updates the exercise plan.

[0402] Input: Exercise data, emotion data

[0403] Output: Updated exercise plan

[0404] How it works: The server analyzes the newly collected data, adaptively updates the exercise plan as needed, and resends it to the device.

[0405] The above is a specific operation of the processing steps of this system.

[0406] (Application example 2)

[0407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0408] Maintaining health and establishing exercise habits are important issues in modern society. However, there are few systems that provide optimal exercise and meal plans that take into account the user's physiological information and emotional state. In particular, there are no systems that integrate meal suggestions and automatic ordering functions, making it difficult for users to consistently manage their health.

[0409] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on goals set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for recognizing the user's emotional state and reflecting it in the exercise plan, means for proposing an optimal meal plan, and means for automatically ordering meals based on the proposed meal plan. This integrates the user's physiological information and emotional information to provide an optimal exercise and meal plan, enabling the user to maintain a healthy lifestyle and establish exercise habits.

[0410] "User's physiological information" is data that indicates the user's health condition, such as physical activity, heart rate, calories burned, and sleep patterns.

[0411] The "means for calculating calories burned" is a system or device that calculates the amount of energy consumed by the user based on the user's exercise data and physiological information.

[0412] An "exercise plan" is a plan that shows an exercise schedule and content that is optimized based on the user's health goals and physiological information.

[0413] "User goals" are health goals set by the user, such as weight loss, muscle gain, etc.

[0414] A "progress monitoring means" is a system or device that collects a user's exercise and health data in real time or at regular intervals and checks their progress.

[0415] An "adaptive updating means" is a system or device that dynamically modifies the exercise plan based on the user's progress or new physiological information.

[0416] "Emotional state" is data that indicates the user's psychological state, including stress level, happiness, and the like.

[0417] The "means for reflecting in exercise plan" is a system or device that customizes an exercise plan taking into account the user's emotional state.

[0418] A "meal plan" is a plan that shows optimized dietary content and nutrient balance that takes into account the user's health goals, physiological information, and emotional state.

[0419] An "automated ordering means" is a system or device that automatically orders meals based on a proposed meal plan.

[0420] The present invention revolves around a system for optimizing a user's health management. The system is realized mainly using a smartphone, a wearable device, and a server. Specific embodiments for carrying out the invention are described below.

[0421] Data collection methods

[0422] Users use a smartphone app to take photos of their meals and send them to a server via the app. In addition, users use wearable devices (e.g., fitness trackers) to collect daily exercise data (e.g., steps taken, heart rate). The smartphone app also uses camera and voice input to recognize and collect data on the user's emotional state (e.g., stress level, happiness). The device then sends this data to a server, which then stores detailed user data.

[0423] Data Analysis Methods

[0424] The server uses an AI algorithm to analyze the received food photos and extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state. The software used includes Python programs and machine learning models (e.g., TensorFlow, PyTorch).

[0425] Movement plan generation means

[0426] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized by taking into account not only the user's physical data (e.g., daily calorie consumption, exercise history) but also their emotional state. For example, if stress levels are high, relaxing exercises (e.g., yoga or relaxing walking) will be recommended. The generated plan includes detailed exercise content, schedule, and frequency.

[0427] Meal planning and automated ordering tools

[0428] The server proposes an optimal meal plan (e.g., grilled chicken salad) based on the user's exercise data, emotional data, and nutritional analysis results. The server then provides the user with a one-click meal ordering function. Based on the proposed meal plan, the server automatically orders the meal.

[0429] Feedback and monitoring measures

[0430] The user exercises according to the presented exercise plan and eats according to the meal plan. Afterwards, they record their emotional state and exercise progress using a wearable device or smartphone app, and the data is sent to the server. The server analyzes this data and reflects it in their next exercise and meal plan.

[0431] Examples and prompts

[0432] For example, if the user jogs 5,000 steps and their stress level is 3, they will be suggested to order a "grilled chicken salad." An example prompt is as follows:

[0433] Consider today's exercise data and emotional state to suggest the optimal meal plan. The user's exercise data is 5000 steps of jogging, and their emotional state is stress level 3.

[0434] In this way, the system of the present invention can comprehensively manage a user's behavior, physiological information, and emotional information, and support continuous health management.

[0435] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0436] Step 1:

[0437] A user takes a photo of a meal using a smartphone app and sends it to the server through the app. Specifically, the user takes a photo of the meal and taps the send button. The input is the photo data of the meal, and the output is that this data is sent to the server.

[0438] Step 2:

[0439] The terminal (wearable device) acquires daily exercise data (e.g., number of steps, heart rate) and sends it to a server. Specifically, the wearable device records the user's exercise information using sensors. The input is the user's exercise data, and the output is that this data is sent to the server.

[0440] Step 3:

[0441] The smartphone app uses a camera or voice input to recognize the user's emotional state and sends the data to a server. Specifically, the app recognizes the user's face and analyzes their voice to extract emotional data. The input is the user's face image or voice data, and the output is data indicating the user's emotional state, which is sent to the server.

[0442] Step 4:

[0443] The server analyzes the received food photos using an AI algorithm to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. Specifically, the AI ​​model analyzes the food photos and retrieves nutritional information from a database. The input is food photo data, and the output is nutritional information.

[0444] Step 5:

[0445] The server analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. Specifically, it processes the exercise data using a data analysis tool and calculates calories burned. The input is the exercise data, and the output is the exercise analysis results, such as calories burned.

[0446] Step 6:

[0447] The server uses an emotion engine to analyze the user's emotion data and identify their current psychological state. Specifically, it applies an emotion analysis algorithm to classify the emotional state. The input is emotion data, and the output is data indicating the user's psychological state.

[0448] Step 7:

[0449] The server generates an exercise plan based on the user's self-defined goals, emotional data, and movement data. Specifically, a custom algorithm integrates various data to generate an optimal exercise plan. The inputs are goal data, movement data, and emotional data, and the output is the exercise plan.

[0450] Step 8:

[0451] The server proposes an optimal meal plan based on the user's exercise data, emotional data, and nutrient analysis results. Specifically, it generates a meal plan using a nutrition calculation algorithm. The inputs are exercise data, emotional data, and nutrient data, and the output is a meal plan.

[0452] Step 9:

[0453] It provides an interface for users to order meals with one click, and the server automatically orders meals based on the proposed meal plan. Specifically, it calls the ordering API to confirm the meal order. The input is the meal plan, and the output is a notification that the order has been completed.

[0454] Step 10:

[0455] The user exercises according to the presented exercise plan and records their emotional state and exercise progress using a wearable device or smartphone app. The data is then sent to a server. Specifically, the user exercises, and the wearable device then records the data. The input is exercise data and emotional data, and the output is the data sent to the server.

[0456] Step 11:

[0457] The server analyzes the user's exercise and emotional data and dynamically updates the exercise and meal plans as needed. Specifically, it runs a reanalysis algorithm to generate new plans. The input is the latest exercise and emotional data, and the output is the updated exercise and meal plans.

[0458] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0460] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0461] [Second embodiment]

[0462] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0463] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0464] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0465] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0466] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0467] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0468] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0469] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0470] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0471] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0472] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0473] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0474] Overall overview

[0475] This system is designed to help users maintain their exercise habits. Users use a smartphone app and a wearable device to collect their exercise and dietary data, which is then analyzed by a server to provide an individually optimized exercise plan. The plan is dynamically updated based on the user's goals.

[0476] Data collection

[0477] Users take photos of their meals using a smartphone app. The device (smartphone) then sends these photos to a server. Users also wear a wearable device while exercising daily, and the data is sent to the server via the device. The wearable device collects physiological information such as heart rate, steps, and distance traveled in real time. By collecting this data, the server can grasp the user's condition in detail.

[0478] Data analysis

[0479] The server uses an AI algorithm to analyze the received photos of the meal. Specifically, it uses image recognition technology to identify the contents of the meal and extract information about each nutrient. It also analyzes data sent from the wearable device to calculate the type and intensity of exercise and calories burned. This allows for detailed measurements of the user's calories burned and nutritional intake.

[0480] Generate exercise plans

[0481] Based on the user's self-defined goals (e.g., weight loss or muscle gain), the server generates an exercise plan. This plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level. The generated plan details the type, duration, and frequency of exercise.

[0482] Feedback and Monitoring

[0483] The server sends the generated exercise plan to the device, which then presents it to the user. The user then exercises according to the presented plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. This allows the user to continue exercising effectively toward their set goals.

[0484] Specific examples

[0485] Examples of data collection

[0486] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0487] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0488] Specific examples of data analysis

[0489] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0490] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0491] Example of generating an exercise plan

[0492] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of light jogging on Monday and 45 minutes of strength training on Tuesday."

[0493] The terminal notifies the user of this exercise plan and displays it as feedback.

[0494] Examples of feedback and monitoring

[0495] The user follows the proposed plan and jogs for 30 minutes on Monday.

[0496] The terminal transmits data of the jogging that has been performed to the server.

[0497] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the device.

[0498] In this way, the system manages the user's behavior and physiological information in an integrated manner, supporting continuous exercise and health management.

[0499] The processing flow will be explained below.

[0500] Step 1:

[0501] Users open the app on their smartphone and take a photo of their meal.

[0502] The device (smartphone) automatically sends the photograph of the meal to the server.

[0503] Step 2:

[0504] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[0505] The collected data is sent to the server in real time via the terminal.

[0506] Step 3:

[0507] The server analyzes the received meal photos using an AI algorithm to identify the meal contents and extract information on each nutrient (e.g., calories, protein, fat, and carbohydrate content).

[0508] Step 4:

[0509] The server analyzes the exercise data sent from the wearable device, including the type of exercise (e.g., running, walking), exercise intensity, and calorie consumption calculations.

[0510] Step 5:

[0511] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[0512] This exercise plan includes specific exercise content (e.g., exercise schedule for each day of the week), time, frequency, etc.

[0513] Step 6:

[0514] The server transmits the generated exercise plan to the terminal.

[0515] The device will notify the user of the exercise plan and display it as feedback, where the user can see the details of the exercise plan (e.g., 30 minutes of jogging on Mondays).

[0516] Step 7:

[0517] The user follows the notified exercise plan and performs the exercise.

[0518] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[0519] Step 8:

[0520] The server periodically monitors the user's progress, including analyzing changes in body composition and exercise performance.

[0521] Based on the analysis results, the exercise plan is adjusted as needed.

[0522] Step 9:

[0523] The server again transmits the updated exercise plan to the terminal.

[0524] The device notifies the user of the updated exercise plan and displays it as feedback, and the user continues exercising again based on the new plan.

[0525] Example 1

[0526] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] Conventional exercise plan generation systems only acquire a user's physiological information and generate an exercise plan based on that information, making it difficult to provide a plan optimized for the user's self-set goals. In particular, they lack real-time feedback and adaptive updating of exercise plans, and efficient exercise guidance based on the user's progress is required.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0529] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for utilizing a generative AI model to generate an exercise plan based on the user's self-set goals, means for presenting the generated exercise plan to the user, and means for monitoring the user's progress and adaptively updating the exercise plan as needed, thereby providing an exercise plan optimized for each individual user and enabling real-time feedback and adaptive plan changes.

[0530] "User's physiological information" is data that indicates the user's physical activity and health condition, such as heart rate, number of steps, and distance traveled.

[0531] The "means for calculating calories burned" is a device or system that calculates the amount of energy burned by the user through exercise based on the acquired physiological information.

[0532] A "generative AI model" is an artificial intelligence algorithm used to generate an optimized exercise plan based on a user's goals and current exercise data.

[0533] "Image recognition technology" is a technology for analyzing photographs and image data and identifying the objects and content contained within them.

[0534] An "exercise plan" is a specific exercise content and schedule that is planned based on the goal that the user wants to achieve.

[0535] "Monitoring" is the process of continuously observing and collecting data about a user's progress and exercise performance.

[0536] An "adaptive updating means" is a device or system for dynamically changing or adjusting an exercise plan based on the user's latest progress and feedback.

[0537] A "wearable device" is a small electronic device that can be worn by a user and is used to collect physiological information in real time.

[0538] System configuration

[0539] This system is designed to help users maintain their exercise habits and is primarily composed of a smartphone app and wearable device used by the user, as well as a server that analyzes data and generates exercise plans.

[0540] Data collection details

[0541] The user takes a photo of their meal using a smartphone app, and the device (smartphone) sends the photo to a server. The wearable device collects physiological information such as the user's heart rate, number of steps, and distance traveled in real time, and sends the data to the server via the smartphone. This data collection uses the smartphone's camera function and common communication methods such as Bluetooth and Wi-Fi.

[0542] Data analysis details

[0543] The server analyzes the received food photos using AI algorithms (e.g., Google Cloud Vision API) to identify the meal contents and extract information on each nutrient. It also uses Python and TensorFlow to analyze physiological data sent from the wearable device and calculate the type and intensity of exercise and calories burned.

[0544] Generate exercise plans

[0545] The server generates an exercise plan based on the user's set goals (e.g., weight loss or muscle gain). This process uses a generative AI model (e.g., OpenAI GPT-4). The generated exercise plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level.

[0546] Feedback and Monitoring

[0547] The server sends the generated exercise plan to the device, which then presents it to the user. As the user exercises according to the plan, the data is collected again and sent to the server. The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed.

[0548] Specific examples

[0549] Examples of data collection

[0550] When a user eats an omelet for breakfast, they take a photo of it with their smartphone and send it to the server via the app.

[0551] The wearable device on the terminal measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0552] Specific examples of data analysis

[0553] The server analyzes the breakfast photo and extracts nutritional information (e.g., calories, protein, fat, and carbohydrate content). An example of a specific prompt is, "Please identify the nutrients contained in this meal."

[0554] The server analyzes the jogging data acquired from the wearable device and calculates the calorie consumption of the user. An example of a specific prompt sentence is "Please calculate the calorie consumption from this jogging data."

[0555] Example of generating an exercise plan

[0556] The server creates a weekly exercise plan based on the user's weight loss goal. Specific exercise content is suggested using a generative AI model. An example prompt is, "Please generate an exercise plan that will help the user achieve their weight goal."

[0557] The device notifies the user of the generated exercise plan and displays feedback as a dashboard.

[0558] Examples of feedback and monitoring

[0559] The user goes for a 30-minute jog on Monday, and the data is collected by the wearable device and sent to the server via the smartphone.

[0560] The server analyzes the acquired jogging data, evaluates the user's progress based on their heart rate and calories burned, and adjusts the exercise plan for the next week and sends it back to the device.

[0561] Through these processes, the system comprehensively manages the user's behavior and physiological information, supporting continuous exercise and health management.

[0562] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0563] Step 1: Data collection

[0564] A user takes a photo of a meal using a smartphone app. The input is the photo of the meal, and the output is the photo data.

[0565] The device (smartphone) previews the captured photo within the app and asks the user for confirmation. Once the user has finished checking, they press the "Send" button to send the photo data to the server. Specifically, data is sent using the HTTP protocol.

[0566] When a user starts exercising, the terminal (wearable device) collects physiological information such as heart rate, number of steps, and distance traveled in real time. The input is the user's physiological information, and the output is the collected physiological data.

[0567] The device (smartphone) sends the collected physiological data in bulk to a server. The data is synchronized to the smartphone using Bluetooth or Wi-Fi, and then transmitted to the server using the HTTP protocol.

[0568] Step 2: Data analysis

[0569] The server analyzes the received meal photos using image recognition technology (e.g., Google Cloud Vision API). The input is the photo data of the meal, and the output is the analysis results, such as the types and amounts of ingredients. For example, the prompt might be, "Please identify the nutrients contained in this meal."

[0570] The server extracts nutrient information (calories, protein, fat, carbohydrates) based on the image recognition results. Specifically, it identifies the nutrient information by referencing a food database.

[0571] The server analyzes physiological data (heart rate, steps, distance traveled) sent from the wearable device. The input is physiological data, and the output is analysis results such as the type and intensity of exercise, and calories burned. Python and TensorFlow are used to clean and preprocess the data, and a machine learning model (e.g., regression model) is used to calculate calories burned. The prompt is "Please calculate calories burned from this jogging data."

[0572] Step 3: Generate an exercise plan

[0573] The server generates an exercise plan based on the user's self-defined goals (e.g., weight loss or muscle gain). The input is the user's goals and analyzed physiological and nutritional data, and the output is a customized exercise plan.

[0574] A generative AI model (e.g., OpenAI GPT-4) is used to create detailed exercise plans. The prompt is, "Generate an exercise plan that will help the user achieve their goal weight."

[0575] The server generates a plan that takes into account the type of exercise (e.g., jogging, strength training), time, and frequency, and sends it to the device in JSON format.

[0576] Step 4: Feedback and monitoring

[0577] The terminal notifies the user of the exercise plan received from the server. The input is the generated exercise plan, and the output is the notification of the exercise plan presented to the user.

[0578] The user exercises based on the presented plan. After the exercise is completed, the wearable device collects data again and sends it to the server via the smartphone. The input is the exercise data performed, and the output is an updated database in the server.

[0579] The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed. The prompt is, "Please adjust next week's exercise plan based on your exercise progress." The updated exercise plan is then notified to the user via their device.

[0580] (Application example 1)

[0581] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0582] In today's busy lifestyles, it is difficult for users to find and consistently follow an exercise plan that is optimal for them. Furthermore, balancing proper nutritional management and exercise is not easy, and users require specialized knowledge to achieve effective results. Therefore, there is a need for a system that provides individually customized exercise plans and nutritional management and effectively supports users in achieving their goals.

[0583] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0584] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as needed, means for analyzing photo data of meals to extract nutrient information, and means for providing training feedback in real time. This allows users to receive an optimal exercise plan and appropriate nutritional management even without specialized knowledge, and to be effectively supported continuously toward achieving their goals.

[0585] "User's physiological information" refers to data related to the user's physical activity, such as heart rate, number of steps taken, and distance traveled.

[0586] "Calories burned" refers to the amount of energy consumed by a user through exercise and daily activities.

[0587] An "exercise plan" is a program that plans specific exercise content and schedules based on goals set by the user.

[0588] "Nutrient information" refers to information on ingredients such as calories, protein, fat, and carbohydrates extracted from dietary data.

[0589] "Real-time" refers to a process that instantly collects data about the user's ongoing activities and provides analysis results.

[0590] "Feedback" refers to evaluations and advice returned based on a user's activities, providing information that will be useful for the next action.

[0591] "Adaptive updating" refers to dynamically changing and adjusting the exercise plan according to the user's progress and condition.

[0592] A "wearable device" is a device that can be worn by a user and can measure physiological information such as heart rate and number of steps.

[0593] A "smartphone" is an evolved form of a mobile phone, a multi-function device that can connect to the Internet and use applications.

[0594] "Smart glasses" are devices that have the shape of glasses, are equipped with a display and sensors, and can present information to the user in real time.

[0595] A "server" is a computer that stores, manages, and processes data over a network, and serves as the central hub of the entire system.

[0596] Overall system overview

[0597] The system is designed to encourage users to continue their exercise habits and provide individually customized exercise plans and nutritional management. The system operates using the user's smartphone, wearable device, server, smart glasses, or head-mounted display.

[0598] Hardware and software used

[0599] Hardware

[0600] 1. Smartphone: Used for data collection and as a user interface.

[0601] 2. Wearable devices: Collect the user's physiological information (heart rate, steps, distance, etc.) in real time.

[0602] 3. Smart glasses or head-mounted displays: display real-time training feedback and provide exercise instructions to the user.

[0603] 4. Server: Analyzes data, generates exercise plans, and monitors user progress.

[0604] software

[0605] 1. Flask: Used as a server-side framework.

[0606] 2. Keras: Uses deep learning models (generative AI models) for image recognition and data analysis.

[0607] 3. OpenCV: Used as an image processing library.

[0608] System processing flow

[0609] Data collection

[0610] Users use a smartphone app to take photos of their meals and send them to a server. The wearable device also collects physiological information such as heart rate and number of steps taken while the user is exercising, and sends the data to the server via the smartphone.

[0611] Data analysis

[0612] The server receives photo data of meals using Flask and performs image analysis using a Keras model. Specifically, it identifies the contents of the meal and extracts information about each nutrient. Exercise data is processed using OpenCV to calculate the user's calories burned and exercise intensity.

[0613] Generate exercise plans

[0614] The server generates a personalized exercise plan based on the user's set goals (e.g., weight loss, muscle gain), detailing the type, duration, and frequency of exercise, and dynamically updates the plan to fit the user's daily routine.

[0615] Feedback and Monitoring

[0616] The server presents the generated exercise plan to the user via a smartphone or smart glasses. The user's progress is continuously monitored via the wearable device, and the exercise plan is adaptively updated as needed.

[0617] Specific examples

[0618] 1. Specific examples of data collection

[0619] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0620] The wearable device measures the user's heart rate and distance traveled while jogging and transmits the data to a server.

[0621] 2. Specific examples of data analysis

[0622] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0623] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0624] 3. Example of motor plan generation

[0625] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of jogging on Monday and 45 minutes of strength training on Tuesday."

[0626] The smartphone notifies the user of this exercise plan and displays it as feedback.

[0627] 4. Examples of feedback and monitoring

[0628] The user follows the proposed plan and jogs for 30 minutes on Monday.

[0629] The wearable device transmits data about the jogging that has been performed to a server.

[0630] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the smartphone.

[0631] Examples of prompt statements

[0632] Analyze the user's breakfast photo data and extract the nutrients (calories, protein, fat, carbohydrates, etc.) contained in it. Also, analyze the previous day's jogging data and calculate the calories burned. Generate a new exercise plan based on the user's weight loss goal, including "30 minutes of jogging on Monday" and "45 minutes of strength training on Tuesday."

[0633] In this way, a system is constructed by combining specific hardware and software to provide optimal exercise and nutrition management for users to achieve their goals.

[0634] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0635] Step 1:

[0636] Data collection

[0637] Input: A user opens a smartphone app and takes a photo of their breakfast, while a wearable device monitors their heart rate, steps taken, and distance traveled in real time.

[0638] Data processing / calculation: The smartphone sends the photograph of the meal to the server via the app. The wearable device sends the measurement data to the server via the smartphone.

[0639] Output: The server receives the food photo data and physiological information data.

[0640] Specific operation: The user operates the app and takes a photo. The wearable device automatically records the measurement data and sends it to a smartphone.

[0641] Step 2:

[0642] Data Analysis - Dietary Data

[0643] Input: The server receives a photo of a meal sent by the user.

[0644] Data processing / calculation: The server uses a Keras model to analyze the photo data and identify the food contents. It then uses OpenCV to perform image processing and extract nutritional information such as calories, protein, fat, and carbohydrates.

[0645] Output: Extracted nutrient information is obtained.

[0646] Specific operation: The server uses an AI model to recognize foods in an image and output specific nutritional data.

[0647] Step 3:

[0648] Data Analysis - Exercise Data

[0649] Input: The server receives the wearable device movement data sent by the user.

[0650] Data processing / calculation: The server calculates the user's calories burned and exercise intensity based on the received physiological information.

[0651] Output: Calculated calorie consumption and exercise intensity data are obtained.

[0652] Specific operation: The server analyzes the received data and stores the calculation results.

[0653] Step 4:

[0654] Exercise plan generation

[0655] Input: The server receives the analyzed nutrition information, exercise data, and user-defined goals.

[0656] Data processing / calculation: The server uses this data to generate an exercise plan that is optimal for the user's goals. The generative AI model dynamically adjusts the exercise content and schedule.

[0657] Output: A customized exercise plan is generated.

[0658] Specific operation: The server uses the AI ​​model to combine the data analysis results, calculate and output the optimal plan.

[0659] Step 5:

[0660] Plan presentation

[0661] Input: Server-generated exercise plan.

[0662] Data processing / calculation: The exercise plan is sent from the server to the device (smartphone, smart glasses, etc.).

[0663] Output: The exercise plan is displayed on the user's device.

[0664] Specific operation: The server sends the plan data along with the notification message to the terminal, which displays it through the user interface.

[0665] Step 6:

[0666] Feedback and progress monitoring

[0667] Input: User progress data (exercise performance, heart rate variability, etc.) is collected from a wearable device.

[0668] Data processing / calculation: The server compares the planned exercise content with the actual activity based on the progress data and adjusts the exercise plan if necessary.

[0669] Output: A new, adjusted exercise plan.

[0670] How it works: The server analyzes the progress data, adjusts the plan using an AI model, and sends the new plan back to the device.

[0671] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0672] Overall overview

[0673] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect their exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[0674] Data collection

[0675] Users take photos of their meals using a smartphone app. They also use a wearable device to collect daily exercise data. The smartphone app then uses the camera and voice input to recognize the user's emotions. The device then sends this data to a server, which then collects detailed user data.

[0676] Data analysis

[0677] The server uses an AI algorithm to analyze the received food photos and extract nutritional information from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state.

[0678] Generate exercise plans

[0679] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if stress levels are high, the server may recommend exercise that has a relaxing effect. The generated plan includes detailed exercise content, schedule, and frequency.

[0680] Feedback and Monitoring

[0681] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[0682] Specific examples

[0683] Examples of data collection

[0684] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0685] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0686] Users can report their emotional state after exercise using a smartphone app, or the app can automatically collect data by analyzing emotions through facial recognition and voice tone.

[0687] Specific examples of data analysis

[0688] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0689] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0690] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[0691] Example of generating an exercise plan

[0692] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, the server might generate a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[0693] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[0694] Examples of feedback and monitoring

[0695] The user follows the plan and does 30 minutes of yoga on Mondays.

[0696] The terminal transmits data on the yoga session performed and subsequent emotional data (for example, stress reduction effect) to the server.

[0697] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[0698] In this way, the system manages the user's behavior, physiological information, and emotional information in an integrated manner, supporting continuous exercise and health management.

[0699] The processing flow will be explained below.

[0700] Step 1:

[0701] Users open the app on their smartphone and take a photo of their meal.

[0702] The device (smartphone) automatically sends the photograph of the meal to the server.

[0703] Step 2:

[0704] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[0705] The collected data is sent to the server in real time via the terminal.

[0706] Step 3:

[0707] Users can report their emotional state after exercise in the app, or the app will automatically analyze emotions through facial recognition and voice tone.

[0708] When a user taps an emotion button, the emotion information is also sent to the server.

[0709] Step 4:

[0710] The server uses an AI algorithm to analyze the received photos of the meal and identify the contents of the meal.

[0711] It also extracts information about each nutrient (e.g., amount of calories, protein, fat, carbohydrates).

[0712] Step 5:

[0713] The server analyzes the exercise data sent from the wearable device.

[0714] This includes calculating the type of exercise (e.g., running, walking), the intensity of the exercise, and the calories burned.

[0715] Step 6:

[0716] The server analyzes the received emotion data using an emotion engine.

[0717] Identify and assess your current emotional state (e.g., stress level, satisfaction, fatigue, etc.).

[0718] Step 7:

[0719] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[0720] This exercise plan is customized taking into account your emotional state as well as your physical data.

[0721] Step 8:

[0722] The server transmits the generated exercise plan to the terminal.

[0723] The device notifies the user of the exercise plan and displays detailed exercise content and schedule as feedback.

[0724] Step 9:

[0725] The user follows the notified exercise plan and performs the exercise.

[0726] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[0727] Step 10:

[0728] The server periodically monitors the user's progress.

[0729] This includes analyzing changes in body composition and exercise outcomes (e.g., weight loss, muscle mass gain).

[0730] Step 11:

[0731] The server uses an emotion engine to analyze the user's emotion data and also monitor the user's psychological state.

[0732] Adaptively update your exercise plan and make real-time adjustments as needed.

[0733] Step 12:

[0734] The server then transmits the updated exercise plan to the terminal again.

[0735] The device notifies the user of the updated exercise plan and presents detailed exercise content and schedule again as feedback.

[0736] Step 13:

[0737] The user then exercises again based on the new exercise plan and continues to receive feedback and monitoring.

[0738] By linking the device and the server, the user's physical and emotional data are managed in an integrated manner, allowing optimal exercise instruction to be provided.

[0739] Example 2

[0740] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0741] Conventional systems primarily generate exercise plans based on the user's physiological information and exercise data, but because they do not take the user's emotional state into account, the exercise plan is not optimized for the user's individual psychological state. This can reduce the user's motivation to continue exercising, potentially reducing the effectiveness of health management.

[0742] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the calories burned by the user based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for acquiring and analyzing emotional information of the user, and means for adjusting the exercise plan based on the analyzed emotional information. This makes it possible to provide an optimal exercise plan according to the user's individual psychological state.

[0743] 1. "User" refers to a person who uses this system to provide exercise, diet, and emotional data and receives feedback based on that data.

[0744] 2. "Physiological information" refers to the user's physical data, such as heart rate, calories burned, and type and intensity of exercise.

[0745] 3. "Exercise Plan" refers to a personalized exercise schedule or program generated based on a user's physiological information, goals, and emotional information.

[0746] 4. "Emotional information" refers to data that indicates a user's emotional state (e.g., stress, satisfaction, etc.), and is primarily collected through facial recognition, voice tone analysis, or user self-reporting.

[0747] 5. "Server" refers to a computer system that receives and analyzes data sent from users and devices, and provides the generated exercise plan to users.

[0748] 6. "Device" refers to an electronic device, such as a smartphone or wearable device, that is used to collect and transmit user data to a server.

[0749] 7. "Means for acquiring" refers to methods and devices for collecting physiological and emotional information of a user, such as sensors, cameras, microphones, etc.

[0750] 8. "Analytical means" refers to the algorithms and software used to process the collected data and evaluate and analyze the user's condition.

[0751] 9. "Means for generating" refers to the method or algorithm for generating an exercise plan based on the analysis results.

[0752] 10. "Presentation means" refers to a method or device for notifying the user of the generated exercise plan, such as a smartphone notification function or app.

[0753] 11. "Monitoring means" refers to the methods and functions that continuously observe and collect user progress and data, and adjust exercise plans as needed.

[0754] Overall overview

[0755] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[0756] Data collection

[0757] Users use a smartphone app to take photos of their meals, which are then analyzed to record the details of their meals. The user also uses a wearable device to collect daily exercise data, including heart rate, distance traveled, and calories burned. Furthermore, the smartphone app recognizes the user's emotions through camera and voice input. All of this data is sent from the device to a server, where detailed user data is collected.

[0758] Data analysis

[0759] The server analyzes the received food photos using AI algorithms (e.g., TensorFlow or PyTorch) and extracts nutritional information from the meal contents. At the same time, it analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. It uses an emotion engine to analyze the user's emotional data and identify their current psychological state. This allows the server to comprehensively evaluate the user's physiological information, exercise data, and emotional data.

[0760] Generate exercise plans

[0761] The server generates an exercise plan based on the user's self-set goals (e.g., target weight, exercise time) and emotional data. This exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if the user is in a high-stress state, it will recommend exercise that has a relaxing effect (e.g., yoga or relaxing walking). The generated exercise plan includes detailed exercise content, schedule, and frequency.

[0762] Feedback and Monitoring

[0763] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[0764] Specific examples

[0765] Examples of data collection

[0766] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0767] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0768] Users report their emotional state after exercise using a smartphone app, which can automatically collect emotions using facial recognition and voice tone analysis.

[0769] Specific examples of data analysis

[0770] The server analyzes photos of breakfast and uses AI algorithms to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate amounts).

[0771] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0772] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[0773] Example of generating an exercise plan

[0774] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, it generates a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[0775] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[0776] Examples of feedback and monitoring

[0777] The user follows the plan and does 30 minutes of yoga on Mondays.

[0778] The terminal transmits data on the yoga performed and subsequent emotional data (e.g., stress reduction effect) to the server.

[0779] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[0780] This system uses a generative AI model to integrate and analyze the user's physiological and emotional information, generate an individually optimized exercise plan, and provide feedback to support continuous exercise and health management.

[0781] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0782] Step 1:

[0783] A user takes a photo of their meal using a smartphone app.

[0784] Input: Food photo

[0785] Output: Food photo data sent to a smartphone app

[0786] How it works: A user opens a smartphone app, takes a photo of a meal, and sends the data through the app.

[0787] Step 2:

[0788] A user wears a wearable device to collect exercise data.

[0789] Input: Physiological data during exercise (heart rate, distance traveled, etc.)

[0790] Output: Exercise data recorded on a wearable device

[0791] How it works: A user puts on the wearable device and starts exercising. The device continuously records exercise data.

[0792] Step 3:

[0793] A smartphone app recognizes the user's emotional state.

[0794] Input: User facial expressions and tone of voice

[0795] Output: Emotional state data

[0796] How it works: A smartphone app uses the camera and microphone to analyze the user's facial expressions and voice to determine their emotional state.

[0797] Step 4:

[0798] The device sends the collected data (meal photos, exercise data, emotional data) to the server.

[0799] Input: Food photo data, exercise data, emotion data

[0800] Output: Detailed user data sent to the server

[0801] How it works: Your device sends data over the internet to a server.

[0802] Step 5:

[0803] The server analyzes the received food photos and extracts nutritional information.

[0804] Input: Food photo data

[0805] Output: Nutrition information (calories, protein, fat, carbohydrates, etc.)

[0806] How it works: The server uses AI algorithms to analyze the image, identify the contents of the meal, and extract nutritional information.

[0807] Step 6:

[0808] The server analyzes the received exercise data and calculates the calories burned.

[0809] Input: Exercise data (heart rate, distance traveled, etc.)

[0810] Output: Calories burned, exercise type, intensity, etc.

[0811] Operation: The server analyzes the exercise data and performs various data calculations to calculate detailed exercise information and calories burned.

[0812] Step 7:

[0813] The server uses an emotion engine to analyze the user's emotion data.

[0814] Input: Emotional state data

[0815] Output: Current psychological state (e.g., stress level, satisfaction, etc.)

[0816] How it works: The server uses the emotion engine to analyze the user's emotion data and identify their psychological state.

[0817] Step 8:

[0818] The server generates an exercise plan based on the user's goals and the analyzed data.

[0819] Input: Goal data, Nutrition information, Exercise data, Emotion data

[0820] Output: Individually optimized exercise plans

[0821] How it works: The server comprehensively analyzes all the data it collects and creates the optimal exercise plan for the user.

[0822] Step 9:

[0823] The server sends the generated exercise plan to the terminal.

[0824] Input: Personalized exercise plan

[0825] Output: Exercise plan sent to device

[0826] Operation: The server sends the generated exercise plan to the device and notifies the user.

[0827] Step 10:

[0828] The user exercises according to the exercise plan presented to them.

[0829] Input: Exercise plan

[0830] Output: Exercise data performed

[0831] Action: The user performs the specified exercise according to the exercise plan received from the device.

[0832] Step 11:

[0833] The terminal transmits the exercise data and emotion data that have been performed to the server again.

[0834] Input: Exercise data, emotion data

[0835] Output: Data to be resent to the server

[0836] Operation: The device sends the exercise data and emotion data back to the server.

[0837] Step 12:

[0838] The server analyzes the collected data and adaptively updates the exercise plan.

[0839] Input: Exercise data, emotion data

[0840] Output: Updated exercise plan

[0841] How it works: The server analyzes the newly collected data, adaptively updates the exercise plan as needed, and resends it to the device.

[0842] The above is a specific operation of the processing steps of this system.

[0843] (Application example 2)

[0844] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0845] Maintaining health and establishing exercise habits are important issues in modern society. However, there are few systems that provide optimal exercise and meal plans that take into account the user's physiological information and emotional state. In particular, there are no systems that integrate meal suggestions and automatic ordering functions, making it difficult for users to consistently manage their health.

[0846] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on goals set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for recognizing the user's emotional state and reflecting it in the exercise plan, means for proposing an optimal meal plan, and means for automatically ordering meals based on the proposed meal plan. This integrates the user's physiological information and emotional information to provide an optimal exercise and meal plan, enabling the user to maintain a healthy lifestyle and establish exercise habits.

[0847] "User's physiological information" is data that indicates the user's health condition, such as physical activity, heart rate, calories burned, and sleep patterns.

[0848] The "means for calculating calories burned" is a system or device that calculates the amount of energy consumed by the user based on the user's exercise data and physiological information.

[0849] An "exercise plan" is a plan that shows an exercise schedule and content that is optimized based on the user's health goals and physiological information.

[0850] "User goals" are health goals set by the user, such as weight loss, muscle gain, etc.

[0851] A "progress monitoring means" is a system or device that collects a user's exercise and health data in real time or at regular intervals and checks their progress.

[0852] An "adaptive updating means" is a system or device that dynamically modifies the exercise plan based on the user's progress or new physiological information.

[0853] "Emotional state" is data that indicates the user's psychological state, including stress level, happiness, and the like.

[0854] The "means for reflecting in exercise plan" is a system or device that customizes an exercise plan taking into account the user's emotional state.

[0855] A "meal plan" is a plan that shows optimized dietary content and nutrient balance that takes into account the user's health goals, physiological information, and emotional state.

[0856] An "automated ordering means" is a system or device that automatically orders meals based on a proposed meal plan.

[0857] The present invention revolves around a system for optimizing a user's health management. The system is realized mainly using a smartphone, a wearable device, and a server. Specific embodiments for carrying out the invention are described below.

[0858] Data collection methods

[0859] Users use a smartphone app to take photos of their meals and send them to a server via the app. In addition, users use wearable devices (e.g., fitness trackers) to collect daily exercise data (e.g., steps taken, heart rate). The smartphone app also uses camera and voice input to recognize and collect data on the user's emotional state (e.g., stress level, happiness). The device then sends this data to a server, which then stores detailed user data.

[0860] Data Analysis Methods

[0861] The server uses an AI algorithm to analyze the received food photos and extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state. The software used includes Python programs and machine learning models (e.g., TensorFlow, PyTorch).

[0862] Movement plan generation means

[0863] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized by taking into account not only the user's physical data (e.g., daily calorie consumption, exercise history) but also their emotional state. For example, if stress levels are high, relaxing exercises (e.g., yoga or relaxing walking) will be recommended. The generated plan includes detailed exercise content, schedule, and frequency.

[0864] Meal planning and automated ordering tools

[0865] The server proposes an optimal meal plan (e.g., grilled chicken salad) based on the user's exercise data, emotional data, and nutritional analysis results. The server then provides the user with a one-click meal ordering function. Based on the proposed meal plan, the server automatically orders the meal.

[0866] Feedback and monitoring measures

[0867] The user exercises according to the presented exercise plan and eats according to the meal plan. Afterwards, they record their emotional state and exercise progress using a wearable device or smartphone app, and the data is sent to the server. The server analyzes this data and reflects it in their next exercise and meal plan.

[0868] Examples and prompts

[0869] For example, if the user jogs 5,000 steps and their stress level is 3, they will be suggested to order a "grilled chicken salad." An example prompt is as follows:

[0870] Consider today's exercise data and emotional state to suggest the optimal meal plan. The user's exercise data is 5000 steps of jogging, and their emotional state is stress level 3.

[0871] In this way, the system of the present invention can comprehensively manage a user's behavior, physiological information, and emotional information, and support continuous health management.

[0872] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0873] Step 1:

[0874] A user takes a photo of a meal using a smartphone app and sends it to the server through the app. Specifically, the user takes a photo of the meal and taps the send button. The input is the photo data of the meal, and the output is that this data is sent to the server.

[0875] Step 2:

[0876] The terminal (wearable device) acquires daily exercise data (e.g., number of steps, heart rate) and sends it to a server. Specifically, the wearable device records the user's exercise information using sensors. The input is the user's exercise data, and the output is that this data is sent to the server.

[0877] Step 3:

[0878] The smartphone app uses a camera or voice input to recognize the user's emotional state and sends the data to a server. Specifically, the app recognizes the user's face and analyzes their voice to extract emotional data. The input is the user's face image or voice data, and the output is data indicating the user's emotional state, which is sent to the server.

[0879] Step 4:

[0880] The server analyzes the received food photos using an AI algorithm to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. Specifically, the AI ​​model analyzes the food photos and retrieves nutritional information from a database. The input is food photo data, and the output is nutritional information.

[0881] Step 5:

[0882] The server analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. Specifically, it processes the exercise data using a data analysis tool and calculates calories burned. The input is the exercise data, and the output is the exercise analysis results, such as calories burned.

[0883] Step 6:

[0884] The server uses an emotion engine to analyze the user's emotion data and identify their current psychological state. Specifically, it applies an emotion analysis algorithm to classify the emotional state. The input is emotion data, and the output is data indicating the user's psychological state.

[0885] Step 7:

[0886] The server generates an exercise plan based on the user's self-defined goals, emotional data, and movement data. Specifically, a custom algorithm integrates various data to generate an optimal exercise plan. The inputs are goal data, movement data, and emotional data, and the output is the exercise plan.

[0887] Step 8:

[0888] The server proposes an optimal meal plan based on the user's exercise data, emotional data, and nutrient analysis results. Specifically, it generates a meal plan using a nutrition calculation algorithm. The inputs are exercise data, emotional data, and nutrient data, and the output is a meal plan.

[0889] Step 9:

[0890] It provides an interface for users to order meals with one click, and the server automatically orders meals based on the proposed meal plan. Specifically, it calls the ordering API to confirm the meal order. The input is the meal plan, and the output is a notification that the order has been completed.

[0891] Step 10:

[0892] The user exercises according to the presented exercise plan and records their emotional state and exercise progress using a wearable device or smartphone app. The data is then sent to a server. Specifically, the user exercises, and the wearable device then records the data. The input is exercise data and emotional data, and the output is the data sent to the server.

[0893] Step 11:

[0894] The server analyzes the user's exercise and emotional data and dynamically updates the exercise and meal plans as needed. Specifically, it runs a reanalysis algorithm to generate new plans. The input is the latest exercise and emotional data, and the output is the updated exercise and meal plans.

[0895] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0896] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0897] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0898] [Third embodiment]

[0899] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0900] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0901] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0902] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0903] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0904] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0905] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0906] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0907] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0908] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0909] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0910] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0911] Overall overview

[0912] This system is designed to help users maintain their exercise habits. Users use a smartphone app and a wearable device to collect their exercise and dietary data, which is then analyzed by a server to provide an individually optimized exercise plan. The plan is dynamically updated based on the user's goals.

[0913] Data collection

[0914] Users take photos of their meals using a smartphone app. The device (smartphone) then sends these photos to a server. Users also wear a wearable device while exercising daily, and the data is sent to the server via the device. The wearable device collects physiological information such as heart rate, steps, and distance traveled in real time. By collecting this data, the server can grasp the user's condition in detail.

[0915] Data analysis

[0916] The server uses an AI algorithm to analyze the received photos of the meal. Specifically, it uses image recognition technology to identify the contents of the meal and extract information about each nutrient. It also analyzes data sent from the wearable device to calculate the type and intensity of exercise and calories burned. This allows for detailed measurements of the user's calories burned and nutritional intake.

[0917] Generate exercise plans

[0918] Based on the user's self-defined goals (e.g., weight loss or muscle gain), the server generates an exercise plan. This plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level. The generated plan details the type, duration, and frequency of exercise.

[0919] Feedback and Monitoring

[0920] The server sends the generated exercise plan to the device, which then presents it to the user. The user then exercises according to the presented plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. This allows the user to continue exercising effectively toward their set goals.

[0921] Specific examples

[0922] Examples of data collection

[0923] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[0924] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0925] Specific examples of data analysis

[0926] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[0927] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[0928] Example of generating an exercise plan

[0929] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of light jogging on Monday and 45 minutes of strength training on Tuesday."

[0930] The terminal notifies the user of this exercise plan and displays it as feedback.

[0931] Examples of feedback and monitoring

[0932] The user follows the proposed plan and jogs for 30 minutes on Monday.

[0933] The terminal transmits data of the jogging that has been performed to the server.

[0934] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the device.

[0935] In this way, the system manages the user's behavior and physiological information in an integrated manner, supporting continuous exercise and health management.

[0936] The processing flow will be explained below.

[0937] Step 1:

[0938] Users open the app on their smartphone and take a photo of their meal.

[0939] The device (smartphone) automatically sends the photograph of the meal to the server.

[0940] Step 2:

[0941] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[0942] The collected data is sent to the server in real time via the terminal.

[0943] Step 3:

[0944] The server analyzes the received meal photos using an AI algorithm to identify the meal contents and extract information on each nutrient (e.g., calories, protein, fat, and carbohydrate content).

[0945] Step 4:

[0946] The server analyzes the exercise data sent from the wearable device, including the type of exercise (e.g., running, walking), exercise intensity, and calorie consumption calculations.

[0947] Step 5:

[0948] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[0949] This exercise plan includes specific exercise content (e.g., exercise schedule for each day of the week), time, frequency, etc.

[0950] Step 6:

[0951] The server transmits the generated exercise plan to the terminal.

[0952] The device will notify the user of the exercise plan and display it as feedback, where the user can see the details of the exercise plan (e.g., 30 minutes of jogging on Mondays).

[0953] Step 7:

[0954] The user follows the notified exercise plan and performs the exercise.

[0955] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[0956] Step 8:

[0957] The server periodically monitors the user's progress, including analyzing changes in body composition and exercise performance.

[0958] Based on the analysis results, the exercise plan is adjusted as needed.

[0959] Step 9:

[0960] The server again transmits the updated exercise plan to the terminal.

[0961] The device notifies the user of the updated exercise plan and displays it as feedback, and the user continues exercising again based on the new plan.

[0962] Example 1

[0963] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0964] Conventional exercise plan generation systems only acquire a user's physiological information and generate an exercise plan based on that information, making it difficult to provide a plan optimized for the user's self-set goals. In particular, they lack real-time feedback and adaptive updating of exercise plans, and efficient exercise guidance based on the user's progress is required.

[0965] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0966] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for utilizing a generative AI model to generate an exercise plan based on the user's self-set goals, means for presenting the generated exercise plan to the user, and means for monitoring the user's progress and adaptively updating the exercise plan as needed, thereby providing an exercise plan optimized for each individual user and enabling real-time feedback and adaptive plan changes.

[0967] "User's physiological information" is data that indicates the user's physical activity and health condition, such as heart rate, number of steps, and distance traveled.

[0968] The "means for calculating calories burned" is a device or system that calculates the amount of energy burned by the user through exercise based on the acquired physiological information.

[0969] A "generative AI model" is an artificial intelligence algorithm used to generate an optimized exercise plan based on a user's goals and current exercise data.

[0970] "Image recognition technology" is a technology for analyzing photographs and image data and identifying the objects and content contained within them.

[0971] An "exercise plan" is a specific exercise content and schedule that is planned based on the goal that the user wants to achieve.

[0972] "Monitoring" is the process of continuously observing and collecting data about a user's progress and exercise performance.

[0973] An "adaptive updating means" is a device or system for dynamically changing or adjusting an exercise plan based on the user's latest progress and feedback.

[0974] A "wearable device" is a small electronic device that can be worn by a user and is used to collect physiological information in real time.

[0975] System configuration

[0976] This system is designed to help users maintain their exercise habits and is primarily composed of a smartphone app and wearable device used by the user, as well as a server that analyzes data and generates exercise plans.

[0977] Data collection details

[0978] The user takes a photo of their meal using a smartphone app, and the device (smartphone) sends the photo to a server. The wearable device collects physiological information such as the user's heart rate, number of steps, and distance traveled in real time, and sends the data to the server via the smartphone. This data collection uses the smartphone's camera function and common communication methods such as Bluetooth and Wi-Fi.

[0979] Data analysis details

[0980] The server analyzes the received food photos using AI algorithms (e.g., Google Cloud Vision API) to identify the meal contents and extract information on each nutrient. It also uses Python and TensorFlow to analyze physiological data sent from the wearable device and calculate the type and intensity of exercise and calories burned.

[0981] Generate exercise plans

[0982] The server generates an exercise plan based on the user's set goals (e.g., weight loss or muscle gain). This process uses a generative AI model (e.g., OpenAI GPT-4). The generated exercise plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level.

[0983] Feedback and Monitoring

[0984] The server sends the generated exercise plan to the device, which then presents it to the user. As the user exercises according to the plan, the data is collected again and sent to the server. The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed.

[0985] Specific examples

[0986] Examples of data collection

[0987] When a user eats an omelet for breakfast, they take a photo of it with their smartphone and send it to the server via the app.

[0988] The wearable device on the terminal measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[0989] Specific examples of data analysis

[0990] The server analyzes the breakfast photo and extracts nutritional information (e.g., calories, protein, fat, and carbohydrate content). An example of a specific prompt is, "Please identify the nutrients contained in this meal."

[0991] The server analyzes the jogging data acquired from the wearable device and calculates the calorie consumption of the user. An example of a specific prompt sentence is "Please calculate the calorie consumption from this jogging data."

[0992] Example of generating an exercise plan

[0993] The server creates a weekly exercise plan based on the user's weight loss goal. Specific exercise content is suggested using a generative AI model. An example prompt is, "Please generate an exercise plan that will help the user achieve their weight goal."

[0994] The device notifies the user of the generated exercise plan and displays feedback as a dashboard.

[0995] Examples of feedback and monitoring

[0996] The user goes for a 30-minute jog on Monday, and the data is collected by the wearable device and sent to the server via the smartphone.

[0997] The server analyzes the acquired jogging data, evaluates the user's progress based on their heart rate and calories burned, and adjusts the exercise plan for the next week and sends it back to the device.

[0998] Through these processes, the system comprehensively manages the user's behavior and physiological information, supporting continuous exercise and health management.

[0999] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1000] Step 1: Data collection

[1001] A user takes a photo of a meal using a smartphone app. The input is the photo of the meal, and the output is the photo data.

[1002] The device (smartphone) previews the captured photo within the app and asks the user for confirmation. Once the user has finished checking, they press the "Send" button to send the photo data to the server. Specifically, data is sent using the HTTP protocol.

[1003] When a user starts exercising, the terminal (wearable device) collects physiological information such as heart rate, number of steps, and distance traveled in real time. The input is the user's physiological information, and the output is the collected physiological data.

[1004] The device (smartphone) sends the collected physiological data in bulk to a server. The data is synchronized to the smartphone using Bluetooth or Wi-Fi, and then transmitted to the server using the HTTP protocol.

[1005] Step 2: Data analysis

[1006] The server analyzes the received meal photos using image recognition technology (e.g., Google Cloud Vision API). The input is the photo data of the meal, and the output is the analysis results, such as the types and amounts of ingredients. For example, the prompt might be, "Please identify the nutrients contained in this meal."

[1007] The server extracts nutrient information (calories, protein, fat, carbohydrates) based on the image recognition results. Specifically, it identifies the nutrient information by referencing a food database.

[1008] The server analyzes physiological data (heart rate, steps, distance traveled) sent from the wearable device. The input is physiological data, and the output is analysis results such as the type and intensity of exercise, and calories burned. Python and TensorFlow are used to clean and preprocess the data, and a machine learning model (e.g., regression model) is used to calculate calories burned. The prompt is "Please calculate calories burned from this jogging data."

[1009] Step 3: Generate an exercise plan

[1010] The server generates an exercise plan based on the user's self-defined goals (e.g., weight loss or muscle gain). The input is the user's goals and analyzed physiological and nutritional data, and the output is a customized exercise plan.

[1011] A generative AI model (e.g., OpenAI GPT-4) is used to create detailed exercise plans. The prompt is, "Generate an exercise plan that will help the user achieve their goal weight."

[1012] The server generates a plan that takes into account the type of exercise (e.g., jogging, strength training), time, and frequency, and sends it to the device in JSON format.

[1013] Step 4: Feedback and monitoring

[1014] The terminal notifies the user of the exercise plan received from the server. The input is the generated exercise plan, and the output is the notification of the exercise plan presented to the user.

[1015] The user exercises based on the presented plan. After the exercise is completed, the wearable device collects data again and sends it to the server via the smartphone. The input is the exercise data performed, and the output is an updated database in the server.

[1016] The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed. The prompt is, "Please adjust next week's exercise plan based on your exercise progress." The updated exercise plan is then notified to the user via their device.

[1017] (Application example 1)

[1018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] In today's busy lifestyles, it is difficult for users to find and consistently follow an exercise plan that is optimal for them. Furthermore, balancing proper nutritional management and exercise is not easy, and users require specialized knowledge to achieve effective results. Therefore, there is a need for a system that provides individually customized exercise plans and nutritional management and effectively supports users in achieving their goals.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1021] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as needed, means for analyzing photo data of meals to extract nutrient information, and means for providing training feedback in real time. This allows users to receive an optimal exercise plan and appropriate nutritional management even without specialized knowledge, and to be effectively supported continuously toward achieving their goals.

[1022] "User's physiological information" refers to data related to the user's physical activity, such as heart rate, number of steps taken, and distance traveled.

[1023] "Calories burned" refers to the amount of energy consumed by a user through exercise and daily activities.

[1024] An "exercise plan" is a program that plans specific exercise content and schedules based on goals set by the user.

[1025] "Nutrient information" refers to information on ingredients such as calories, protein, fat, and carbohydrates extracted from dietary data.

[1026] "Real-time" refers to a process that instantly collects data about the user's ongoing activities and provides analysis results.

[1027] "Feedback" refers to evaluations and advice returned based on a user's activities, providing information that will be useful for the next action.

[1028] "Adaptive updating" refers to dynamically changing and adjusting the exercise plan according to the user's progress and condition.

[1029] A "wearable device" is a device that can be worn by a user and can measure physiological information such as heart rate and number of steps.

[1030] A "smartphone" is an evolved form of a mobile phone, a multi-function device that can connect to the Internet and use applications.

[1031] "Smart glasses" are devices that have the shape of glasses, are equipped with a display and sensors, and can present information to the user in real time.

[1032] A "server" is a computer that stores, manages, and processes data over a network, and serves as the central hub of the entire system.

[1033] Overall system overview

[1034] The system is designed to encourage users to continue their exercise habits and provide individually customized exercise plans and nutritional management. The system operates using the user's smartphone, wearable device, server, smart glasses, or head-mounted display.

[1035] Hardware and software used

[1036] Hardware

[1037] 1. Smartphone: Used for data collection and as a user interface.

[1038] 2. Wearable devices: Collect the user's physiological information (heart rate, steps, distance, etc.) in real time.

[1039] 3. Smart glasses or head-mounted displays: display real-time training feedback and provide exercise instructions to the user.

[1040] 4. Server: Analyzes data, generates exercise plans, and monitors user progress.

[1041] software

[1042] 1. Flask: Used as a server-side framework.

[1043] 2. Keras: Uses deep learning models (generative AI models) for image recognition and data analysis.

[1044] 3. OpenCV: Used as an image processing library.

[1045] System processing flow

[1046] Data collection

[1047] Users use a smartphone app to take photos of their meals and send them to a server. The wearable device also collects physiological information such as heart rate and number of steps taken while the user is exercising, and sends the data to the server via the smartphone.

[1048] Data analysis

[1049] The server receives photo data of meals using Flask and performs image analysis using a Keras model. Specifically, it identifies the contents of the meal and extracts information about each nutrient. Exercise data is processed using OpenCV to calculate the user's calories burned and exercise intensity.

[1050] Generate exercise plans

[1051] The server generates a personalized exercise plan based on the user's set goals (e.g., weight loss, muscle gain), detailing the type, duration, and frequency of exercise, and dynamically updates the plan to fit the user's daily routine.

[1052] Feedback and Monitoring

[1053] The server presents the generated exercise plan to the user via a smartphone or smart glasses. The user's progress is continuously monitored via the wearable device, and the exercise plan is adaptively updated as needed.

[1054] Specific examples

[1055] 1. Specific examples of data collection

[1056] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1057] The wearable device measures the user's heart rate and distance traveled while jogging and transmits the data to a server.

[1058] 2. Specific examples of data analysis

[1059] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[1060] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1061] 3. Example of motor plan generation

[1062] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of jogging on Monday and 45 minutes of strength training on Tuesday."

[1063] The smartphone notifies the user of this exercise plan and displays it as feedback.

[1064] 4. Examples of feedback and monitoring

[1065] The user follows the proposed plan and jogs for 30 minutes on Monday.

[1066] The wearable device transmits data about the jogging that has been performed to a server.

[1067] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the smartphone.

[1068] Examples of prompt statements

[1069] Analyze the user's breakfast photo data and extract the nutrients (calories, protein, fat, carbohydrates, etc.) contained in it. Also, analyze the previous day's jogging data and calculate the calories burned. Generate a new exercise plan based on the user's weight loss goal, including "30 minutes of jogging on Monday" and "45 minutes of strength training on Tuesday."

[1070] In this way, a system is constructed by combining specific hardware and software to provide optimal exercise and nutrition management for users to achieve their goals.

[1071] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1072] Step 1:

[1073] Data collection

[1074] Input: A user opens a smartphone app and takes a photo of their breakfast, while a wearable device monitors their heart rate, steps taken, and distance traveled in real time.

[1075] Data processing / calculation: The smartphone sends the photograph of the meal to the server via the app. The wearable device sends the measurement data to the server via the smartphone.

[1076] Output: The server receives the food photo data and physiological information data.

[1077] Specific operation: The user operates the app and takes a photo. The wearable device automatically records the measurement data and sends it to a smartphone.

[1078] Step 2:

[1079] Data Analysis - Dietary Data

[1080] Input: The server receives a photo of a meal sent by the user.

[1081] Data processing / calculation: The server uses a Keras model to analyze the photo data and identify the food contents. It then uses OpenCV to perform image processing and extract nutritional information such as calories, protein, fat, and carbohydrates.

[1082] Output: Extracted nutrient information is obtained.

[1083] Specific operation: The server uses an AI model to recognize foods in an image and output specific nutritional data.

[1084] Step 3:

[1085] Data Analysis - Exercise Data

[1086] Input: The server receives the wearable device movement data sent by the user.

[1087] Data processing / calculation: The server calculates the user's calories burned and exercise intensity based on the received physiological information.

[1088] Output: Calculated calorie consumption and exercise intensity data are obtained.

[1089] Specific operation: The server analyzes the received data and stores the calculation results.

[1090] Step 4:

[1091] Exercise plan generation

[1092] Input: The server receives the analyzed nutrition information, exercise data, and user-defined goals.

[1093] Data processing / calculation: The server uses this data to generate an exercise plan that is optimal for the user's goals. The generative AI model dynamically adjusts the exercise content and schedule.

[1094] Output: A customized exercise plan is generated.

[1095] Specific operation: The server uses the AI ​​model to combine the data analysis results, calculate and output the optimal plan.

[1096] Step 5:

[1097] Plan presentation

[1098] Input: Server-generated exercise plan.

[1099] Data processing / calculation: The exercise plan is sent from the server to the device (smartphone, smart glasses, etc.).

[1100] Output: The exercise plan is displayed on the user's device.

[1101] Specific operation: The server sends the plan data along with the notification message to the terminal, which displays it through the user interface.

[1102] Step 6:

[1103] Feedback and progress monitoring

[1104] Input: User progress data (exercise performance, heart rate variability, etc.) is collected from a wearable device.

[1105] Data processing / calculation: The server compares the planned exercise content with the actual activity based on the progress data and adjusts the exercise plan if necessary.

[1106] Output: A new, adjusted exercise plan.

[1107] How it works: The server analyzes the progress data, adjusts the plan using an AI model, and sends the new plan back to the device.

[1108] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1109] Overall overview

[1110] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect their exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[1111] Data collection

[1112] Users take photos of their meals using a smartphone app. They also use a wearable device to collect daily exercise data. The smartphone app then uses the camera and voice input to recognize the user's emotions. The device then sends this data to a server, which then collects detailed user data.

[1113] Data analysis

[1114] The server uses an AI algorithm to analyze the received food photos and extract nutritional information from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state.

[1115] Generate exercise plans

[1116] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if stress levels are high, the server may recommend exercise that has a relaxing effect. The generated plan includes detailed exercise content, schedule, and frequency.

[1117] Feedback and Monitoring

[1118] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[1119] Specific examples

[1120] Examples of data collection

[1121] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1122] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1123] Users can report their emotional state after exercise using a smartphone app, or the app can automatically collect data by analyzing emotions through facial recognition and voice tone.

[1124] Specific examples of data analysis

[1125] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[1126] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1127] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[1128] Example of generating an exercise plan

[1129] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, the server might generate a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[1130] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[1131] Examples of feedback and monitoring

[1132] The user follows the plan and does 30 minutes of yoga on Mondays.

[1133] The terminal transmits data on the yoga session performed and subsequent emotional data (for example, stress reduction effect) to the server.

[1134] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[1135] In this way, the system manages the user's behavior, physiological information, and emotional information in an integrated manner, supporting continuous exercise and health management.

[1136] The processing flow will be explained below.

[1137] Step 1:

[1138] Users open the app on their smartphone and take a photo of their meal.

[1139] The device (smartphone) automatically sends the photograph of the meal to the server.

[1140] Step 2:

[1141] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[1142] The collected data is sent to the server in real time via the terminal.

[1143] Step 3:

[1144] Users can report their emotional state after exercise in the app, or the app will automatically analyze emotions through facial recognition and voice tone.

[1145] When a user taps an emotion button, the emotion information is also sent to the server.

[1146] Step 4:

[1147] The server uses an AI algorithm to analyze the received photos of the meal and identify the contents of the meal.

[1148] It also extracts information about each nutrient (e.g., amount of calories, protein, fat, carbohydrates).

[1149] Step 5:

[1150] The server analyzes the exercise data sent from the wearable device.

[1151] This includes calculating the type of exercise (e.g., running, walking), the intensity of the exercise, and the calories burned.

[1152] Step 6:

[1153] The server analyzes the received emotion data using an emotion engine.

[1154] Identify and assess your current emotional state (e.g., stress level, satisfaction, fatigue, etc.).

[1155] Step 7:

[1156] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[1157] This exercise plan is customized taking into account your emotional state as well as your physical data.

[1158] Step 8:

[1159] The server transmits the generated exercise plan to the terminal.

[1160] The device notifies the user of the exercise plan and displays detailed exercise content and schedule as feedback.

[1161] Step 9:

[1162] The user follows the notified exercise plan and performs the exercise.

[1163] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[1164] Step 10:

[1165] The server periodically monitors the user's progress.

[1166] This includes analyzing changes in body composition and exercise outcomes (e.g., weight loss, muscle mass gain).

[1167] Step 11:

[1168] The server uses an emotion engine to analyze the user's emotion data and also monitor the user's psychological state.

[1169] Adaptively update your exercise plan and make real-time adjustments as needed.

[1170] Step 12:

[1171] The server then transmits the updated exercise plan to the terminal again.

[1172] The device notifies the user of the updated exercise plan and presents detailed exercise content and schedule again as feedback.

[1173] Step 13:

[1174] The user then exercises again based on the new exercise plan and continues to receive feedback and monitoring.

[1175] By linking the device and the server, the user's physical and emotional data are managed in an integrated manner, allowing optimal exercise instruction to be provided.

[1176] Example 2

[1177] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1178] Conventional systems primarily generate exercise plans based on the user's physiological information and exercise data, but because they do not take the user's emotional state into account, the exercise plan is not optimized for the user's individual psychological state. This can reduce the user's motivation to continue exercising, potentially reducing the effectiveness of health management.

[1179] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the calories burned by the user based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for acquiring and analyzing emotional information of the user, and means for adjusting the exercise plan based on the analyzed emotional information. This makes it possible to provide an optimal exercise plan according to the user's individual psychological state.

[1180] 1. "User" refers to a person who uses this system to provide exercise, diet, and emotional data and receives feedback based on that data.

[1181] 2. "Physiological information" refers to the user's physical data, such as heart rate, calories burned, and type and intensity of exercise.

[1182] 3. "Exercise Plan" refers to a personalized exercise schedule or program generated based on a user's physiological information, goals, and emotional information.

[1183] 4. "Emotional information" refers to data that indicates a user's emotional state (e.g., stress, satisfaction, etc.), and is primarily collected through facial recognition, voice tone analysis, or user self-reporting.

[1184] 5. "Server" refers to a computer system that receives and analyzes data sent from users and devices, and provides the generated exercise plan to users.

[1185] 6. "Device" refers to an electronic device, such as a smartphone or wearable device, that is used to collect and transmit user data to a server.

[1186] 7. "Means for acquiring" refers to methods and devices for collecting physiological and emotional information of a user, such as sensors, cameras, microphones, etc.

[1187] 8. "Analytical means" refers to the algorithms and software used to process the collected data and evaluate and analyze the user's condition.

[1188] 9. "Means for generating" refers to the method or algorithm for generating an exercise plan based on the analysis results.

[1189] 10. "Presentation means" refers to a method or device for notifying the user of the generated exercise plan, such as a smartphone notification function or app.

[1190] 11. "Monitoring means" refers to the methods and functions that continuously observe and collect user progress and data, and adjust exercise plans as needed.

[1191] Overall overview

[1192] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[1193] Data collection

[1194] Users use a smartphone app to take photos of their meals, which are then analyzed to record the details of their meals. The user also uses a wearable device to collect daily exercise data, including heart rate, distance traveled, and calories burned. Furthermore, the smartphone app recognizes the user's emotions through camera and voice input. All of this data is sent from the device to a server, where detailed user data is collected.

[1195] Data analysis

[1196] The server analyzes the received food photos using AI algorithms (e.g., TensorFlow or PyTorch) and extracts nutritional information from the meal contents. At the same time, it analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. It uses an emotion engine to analyze the user's emotional data and identify their current psychological state. This allows the server to comprehensively evaluate the user's physiological information, exercise data, and emotional data.

[1197] Generate exercise plans

[1198] The server generates an exercise plan based on the user's self-set goals (e.g., target weight, exercise time) and emotional data. This exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if the user is in a high-stress state, it will recommend exercise that has a relaxing effect (e.g., yoga or relaxing walking). The generated exercise plan includes detailed exercise content, schedule, and frequency.

[1199] Feedback and Monitoring

[1200] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[1201] Specific examples

[1202] Examples of data collection

[1203] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1204] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1205] Users report their emotional state after exercise using a smartphone app, which can automatically collect emotions using facial recognition and voice tone analysis.

[1206] Specific examples of data analysis

[1207] The server analyzes photos of breakfast and uses AI algorithms to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate amounts).

[1208] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1209] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[1210] Example of generating an exercise plan

[1211] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, it generates a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[1212] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[1213] Examples of feedback and monitoring

[1214] The user follows the plan and does 30 minutes of yoga on Mondays.

[1215] The terminal transmits data on the yoga performed and subsequent emotional data (e.g., stress reduction effect) to the server.

[1216] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[1217] This system uses a generative AI model to integrate and analyze the user's physiological and emotional information, generate an individually optimized exercise plan, and provide feedback to support continuous exercise and health management.

[1218] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1219] Step 1:

[1220] A user takes a photo of their meal using a smartphone app.

[1221] Input: Food photo

[1222] Output: Food photo data sent to a smartphone app

[1223] How it works: A user opens a smartphone app, takes a photo of a meal, and sends the data through the app.

[1224] Step 2:

[1225] A user wears a wearable device to collect exercise data.

[1226] Input: Physiological data during exercise (heart rate, distance traveled, etc.)

[1227] Output: Exercise data recorded on a wearable device

[1228] How it works: A user puts on the wearable device and starts exercising. The device continuously records exercise data.

[1229] Step 3:

[1230] A smartphone app recognizes the user's emotional state.

[1231] Input: User facial expressions and tone of voice

[1232] Output: Emotional state data

[1233] How it works: A smartphone app uses the camera and microphone to analyze the user's facial expressions and voice to determine their emotional state.

[1234] Step 4:

[1235] The device sends the collected data (meal photos, exercise data, emotional data) to the server.

[1236] Input: Food photo data, exercise data, emotion data

[1237] Output: Detailed user data sent to the server

[1238] How it works: Your device sends data over the internet to a server.

[1239] Step 5:

[1240] The server analyzes the received food photos and extracts nutritional information.

[1241] Input: Food photo data

[1242] Output: Nutrition information (calories, protein, fat, carbohydrates, etc.)

[1243] How it works: The server uses AI algorithms to analyze the image, identify the contents of the meal, and extract nutritional information.

[1244] Step 6:

[1245] The server analyzes the received exercise data and calculates the calories burned.

[1246] Input: Exercise data (heart rate, distance traveled, etc.)

[1247] Output: Calories burned, exercise type, intensity, etc.

[1248] Operation: The server analyzes the exercise data and performs various data calculations to calculate detailed exercise information and calories burned.

[1249] Step 7:

[1250] The server uses an emotion engine to analyze the user's emotion data.

[1251] Input: Emotional state data

[1252] Output: Current psychological state (e.g., stress level, satisfaction, etc.)

[1253] How it works: The server uses the emotion engine to analyze the user's emotion data and identify their psychological state.

[1254] Step 8:

[1255] The server generates an exercise plan based on the user's goals and the analyzed data.

[1256] Input: Goal data, Nutrition information, Exercise data, Emotion data

[1257] Output: Individually optimized exercise plans

[1258] How it works: The server comprehensively analyzes all the data it collects and creates the optimal exercise plan for the user.

[1259] Step 9:

[1260] The server sends the generated exercise plan to the terminal.

[1261] Input: Personalized exercise plan

[1262] Output: Exercise plan sent to device

[1263] Operation: The server sends the generated exercise plan to the device and notifies the user.

[1264] Step 10:

[1265] The user exercises according to the exercise plan presented to them.

[1266] Input: Exercise plan

[1267] Output: Exercise data performed

[1268] Action: The user performs the specified exercise according to the exercise plan received from the device.

[1269] Step 11:

[1270] The terminal transmits the exercise data and emotion data that have been performed to the server again.

[1271] Input: Exercise data, emotion data

[1272] Output: Data to be resent to the server

[1273] Operation: The device sends the exercise data and emotion data back to the server.

[1274] Step 12:

[1275] The server analyzes the collected data and adaptively updates the exercise plan.

[1276] Input: Exercise data, emotion data

[1277] Output: Updated exercise plan

[1278] How it works: The server analyzes the newly collected data, adaptively updates the exercise plan as needed, and resends it to the device.

[1279] The above is a specific operation of the processing steps of this system.

[1280] (Application example 2)

[1281] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1282] Maintaining health and establishing exercise habits are important issues in modern society. However, there are few systems that provide optimal exercise and meal plans that take into account the user's physiological information and emotional state. In particular, there are no systems that integrate meal suggestions and automatic ordering functions, making it difficult for users to consistently manage their health.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on goals set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for recognizing the user's emotional state and reflecting it in the exercise plan, means for proposing an optimal meal plan, and means for automatically ordering meals based on the proposed meal plan. This integrates the user's physiological information and emotional information to provide an optimal exercise and meal plan, enabling the user to maintain a healthy lifestyle and establish exercise habits.

[1284] "User's physiological information" is data that indicates the user's health condition, such as physical activity, heart rate, calories burned, and sleep patterns.

[1285] The "means for calculating calories burned" is a system or device that calculates the amount of energy consumed by the user based on the user's exercise data and physiological information.

[1286] An "exercise plan" is a plan that shows an exercise schedule and content that is optimized based on the user's health goals and physiological information.

[1287] "User goals" are health goals set by the user, such as weight loss, muscle gain, etc.

[1288] A "progress monitoring means" is a system or device that collects a user's exercise and health data in real time or at regular intervals and checks their progress.

[1289] An "adaptive updating means" is a system or device that dynamically modifies the exercise plan based on the user's progress or new physiological information.

[1290] "Emotional state" is data that indicates the user's psychological state, including stress level, happiness, and the like.

[1291] The "means for reflecting in exercise plan" is a system or device that customizes an exercise plan taking into account the user's emotional state.

[1292] A "meal plan" is a plan that shows optimized dietary content and nutrient balance that takes into account the user's health goals, physiological information, and emotional state.

[1293] An "automated ordering means" is a system or device that automatically orders meals based on a proposed meal plan.

[1294] The present invention revolves around a system for optimizing a user's health management. The system is realized mainly using a smartphone, a wearable device, and a server. Specific embodiments for carrying out the invention are described below.

[1295] Data collection methods

[1296] Users use a smartphone app to take photos of their meals and send them to a server via the app. In addition, users use wearable devices (e.g., fitness trackers) to collect daily exercise data (e.g., steps taken, heart rate). The smartphone app also uses camera and voice input to recognize and collect data on the user's emotional state (e.g., stress level, happiness). The device then sends this data to a server, which then stores detailed user data.

[1297] Data Analysis Methods

[1298] The server uses an AI algorithm to analyze the received food photos and extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state. The software used includes Python programs and machine learning models (e.g., TensorFlow, PyTorch).

[1299] Movement plan generation means

[1300] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized by taking into account not only the user's physical data (e.g., daily calorie consumption, exercise history) but also their emotional state. For example, if stress levels are high, relaxing exercises (e.g., yoga or relaxing walking) will be recommended. The generated plan includes detailed exercise content, schedule, and frequency.

[1301] Meal planning and automated ordering tools

[1302] The server proposes an optimal meal plan (e.g., grilled chicken salad) based on the user's exercise data, emotional data, and nutritional analysis results. The server then provides the user with a one-click meal ordering function. Based on the proposed meal plan, the server automatically orders the meal.

[1303] Feedback and monitoring measures

[1304] The user exercises according to the presented exercise plan and eats according to the meal plan. Afterwards, they record their emotional state and exercise progress using a wearable device or smartphone app, and the data is sent to the server. The server analyzes this data and reflects it in their next exercise and meal plan.

[1305] Examples and prompts

[1306] For example, if the user jogs 5,000 steps and their stress level is 3, they will be suggested to order a "grilled chicken salad." An example prompt is as follows:

[1307] Consider today's exercise data and emotional state to suggest the optimal meal plan. The user's exercise data is 5000 steps of jogging, and their emotional state is stress level 3.

[1308] In this way, the system of the present invention can comprehensively manage a user's behavior, physiological information, and emotional information, and support continuous health management.

[1309] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1310] Step 1:

[1311] A user takes a photo of a meal using a smartphone app and sends it to the server through the app. Specifically, the user takes a photo of the meal and taps the send button. The input is the photo data of the meal, and the output is that this data is sent to the server.

[1312] Step 2:

[1313] The terminal (wearable device) acquires daily exercise data (e.g., number of steps, heart rate) and sends it to a server. Specifically, the wearable device records the user's exercise information using sensors. The input is the user's exercise data, and the output is that this data is sent to the server.

[1314] Step 3:

[1315] The smartphone app uses a camera or voice input to recognize the user's emotional state and sends the data to a server. Specifically, the app recognizes the user's face and analyzes their voice to extract emotional data. The input is the user's face image or voice data, and the output is data indicating the user's emotional state, which is sent to the server.

[1316] Step 4:

[1317] The server analyzes the received food photos using an AI algorithm to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. Specifically, the AI ​​model analyzes the food photos and retrieves nutritional information from a database. The input is food photo data, and the output is nutritional information.

[1318] Step 5:

[1319] The server analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. Specifically, it processes the exercise data using a data analysis tool and calculates calories burned. The input is the exercise data, and the output is the exercise analysis results, such as calories burned.

[1320] Step 6:

[1321] The server uses an emotion engine to analyze the user's emotion data and identify their current psychological state. Specifically, it applies an emotion analysis algorithm to classify the emotional state. The input is emotion data, and the output is data indicating the user's psychological state.

[1322] Step 7:

[1323] The server generates an exercise plan based on the user's self-defined goals, emotional data, and movement data. Specifically, a custom algorithm integrates various data to generate an optimal exercise plan. The inputs are goal data, movement data, and emotional data, and the output is the exercise plan.

[1324] Step 8:

[1325] The server proposes an optimal meal plan based on the user's exercise data, emotional data, and nutrient analysis results. Specifically, it generates a meal plan using a nutrition calculation algorithm. The inputs are exercise data, emotional data, and nutrient data, and the output is a meal plan.

[1326] Step 9:

[1327] It provides an interface for users to order meals with one click, and the server automatically orders meals based on the proposed meal plan. Specifically, it calls the ordering API to confirm the meal order. The input is the meal plan, and the output is a notification that the order has been completed.

[1328] Step 10:

[1329] The user exercises according to the presented exercise plan and records their emotional state and exercise progress using a wearable device or smartphone app. The data is then sent to a server. Specifically, the user exercises, and the wearable device then records the data. The input is exercise data and emotional data, and the output is the data sent to the server.

[1330] Step 11:

[1331] The server analyzes the user's exercise and emotional data and dynamically updates the exercise and meal plans as needed. Specifically, it runs a reanalysis algorithm to generate new plans. The input is the latest exercise and emotional data, and the output is the updated exercise and meal plans.

[1332] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1333] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1334] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1335] [Fourth embodiment]

[1336] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1337] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1338] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1339] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1340] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1341] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1342] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1343] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1344] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1347] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1348] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1349] Overall overview

[1350] This system is designed to help users maintain their exercise habits. Users use a smartphone app and a wearable device to collect their exercise and dietary data, which is then analyzed by a server to provide an individually optimized exercise plan. The plan is dynamically updated based on the user's goals.

[1351] Data collection

[1352] Users take photos of their meals using a smartphone app. The device (smartphone) then sends these photos to a server. Users also wear a wearable device while exercising daily, and the data is sent to the server via the device. The wearable device collects physiological information such as heart rate, steps, and distance traveled in real time. By collecting this data, the server can grasp the user's condition in detail.

[1353] Data analysis

[1354] The server uses an AI algorithm to analyze the received photos of the meal. Specifically, it uses image recognition technology to identify the contents of the meal and extract information about each nutrient. It also analyzes data sent from the wearable device to calculate the type and intensity of exercise and calories burned. This allows for detailed measurements of the user's calories burned and nutritional intake.

[1355] Generate exercise plans

[1356] Based on the user's self-defined goals (e.g., weight loss or muscle gain), the server generates an exercise plan. This plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level. The generated plan details the type, duration, and frequency of exercise.

[1357] Feedback and Monitoring

[1358] The server sends the generated exercise plan to the device, which then presents it to the user. The user then exercises according to the presented plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. This allows the user to continue exercising effectively toward their set goals.

[1359] Specific examples

[1360] Examples of data collection

[1361] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1362] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1363] Specific examples of data analysis

[1364] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[1365] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1366] Example of generating an exercise plan

[1367] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of light jogging on Monday and 45 minutes of strength training on Tuesday."

[1368] The terminal notifies the user of this exercise plan and displays it as feedback.

[1369] Examples of feedback and monitoring

[1370] The user follows the proposed plan and jogs for 30 minutes on Monday.

[1371] The terminal transmits data of the jogging that has been performed to the server.

[1372] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the device.

[1373] In this way, the system manages the user's behavior and physiological information in an integrated manner, supporting continuous exercise and health management.

[1374] The processing flow will be explained below.

[1375] Step 1:

[1376] Users open the app on their smartphone and take a photo of their meal.

[1377] The device (smartphone) automatically sends the photograph of the meal to the server.

[1378] Step 2:

[1379] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[1380] The collected data is sent to the server in real time via the terminal.

[1381] Step 3:

[1382] The server analyzes the received meal photos using an AI algorithm to identify the meal contents and extract information on each nutrient (e.g., calories, protein, fat, and carbohydrate content).

[1383] Step 4:

[1384] The server analyzes the exercise data sent from the wearable device, including the type of exercise (e.g., running, walking), exercise intensity, and calorie consumption calculations.

[1385] Step 5:

[1386] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[1387] This exercise plan includes specific exercise content (e.g., exercise schedule for each day of the week), time, frequency, etc.

[1388] Step 6:

[1389] The server transmits the generated exercise plan to the terminal.

[1390] The device will notify the user of the exercise plan and display it as feedback, where the user can see the details of the exercise plan (e.g., 30 minutes of jogging on Mondays).

[1391] Step 7:

[1392] The user follows the notified exercise plan and performs the exercise.

[1393] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[1394] Step 8:

[1395] The server periodically monitors the user's progress, including analyzing changes in body composition and exercise performance.

[1396] Based on the analysis results, the exercise plan is adjusted as needed.

[1397] Step 9:

[1398] The server again transmits the updated exercise plan to the terminal.

[1399] The device notifies the user of the updated exercise plan and displays it as feedback, and the user continues exercising again based on the new plan.

[1400] Example 1

[1401] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1402] Conventional exercise plan generation systems only acquire a user's physiological information and generate an exercise plan based on that information, making it difficult to provide a plan optimized for the user's self-set goals. In particular, they lack real-time feedback and adaptive updating of exercise plans, and efficient exercise guidance based on the user's progress is required.

[1403] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1404] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for utilizing a generative AI model to generate an exercise plan based on the user's self-set goals, means for presenting the generated exercise plan to the user, and means for monitoring the user's progress and adaptively updating the exercise plan as needed, thereby providing an exercise plan optimized for each individual user and enabling real-time feedback and adaptive plan changes.

[1405] "User's physiological information" is data that indicates the user's physical activity and health condition, such as heart rate, number of steps, and distance traveled.

[1406] The "means for calculating calories burned" is a device or system that calculates the amount of energy burned by the user through exercise based on the acquired physiological information.

[1407] A "generative AI model" is an artificial intelligence algorithm used to generate an optimized exercise plan based on a user's goals and current exercise data.

[1408] "Image recognition technology" is a technology for analyzing photographs and image data and identifying the objects and content contained within them.

[1409] An "exercise plan" is a specific exercise content and schedule that is planned based on the goal that the user wants to achieve.

[1410] "Monitoring" is the process of continuously observing and collecting data about a user's progress and exercise performance.

[1411] An "adaptive updating means" is a device or system for dynamically changing or adjusting an exercise plan based on the user's latest progress and feedback.

[1412] A "wearable device" is a small electronic device that can be worn by a user and is used to collect physiological information in real time.

[1413] System configuration

[1414] This system is designed to help users maintain their exercise habits and is primarily composed of a smartphone app and wearable device used by the user, as well as a server that analyzes data and generates exercise plans.

[1415] Data collection details

[1416] The user takes a photo of their meal using a smartphone app, and the device (smartphone) sends the photo to a server. The wearable device collects physiological information such as the user's heart rate, number of steps, and distance traveled in real time, and sends the data to the server via the smartphone. This data collection uses the smartphone's camera function and common communication methods such as Bluetooth and Wi-Fi.

[1417] Data analysis details

[1418] The server analyzes the received food photos using AI algorithms (e.g., Google Cloud Vision API) to identify the meal contents and extract information on each nutrient. It also uses Python and TensorFlow to analyze physiological data sent from the wearable device and calculate the type and intensity of exercise and calories burned.

[1419] Generate exercise plans

[1420] The server generates an exercise plan based on the user's set goals (e.g., weight loss or muscle gain). This process uses a generative AI model (e.g., OpenAI GPT-4). The generated exercise plan includes specific exercises and schedules, and is customized to the user's daily life and fitness level.

[1421] Feedback and Monitoring

[1422] The server sends the generated exercise plan to the device, which then presents it to the user. As the user exercises according to the plan, the data is collected again and sent to the server. The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed.

[1423] Specific examples

[1424] Examples of data collection

[1425] When a user eats an omelet for breakfast, they take a photo of it with their smartphone and send it to the server via the app.

[1426] The wearable device on the terminal measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1427] Specific examples of data analysis

[1428] The server analyzes the breakfast photo and extracts nutritional information (e.g., calories, protein, fat, and carbohydrate content). An example of a specific prompt is, "Please identify the nutrients contained in this meal."

[1429] The server analyzes the jogging data acquired from the wearable device and calculates the calorie consumption of the user. An example of a specific prompt sentence is "Please calculate the calorie consumption from this jogging data."

[1430] Example of generating an exercise plan

[1431] The server creates a weekly exercise plan based on the user's weight loss goal. Specific exercise content is suggested using a generative AI model. An example prompt is, "Please generate an exercise plan that will help the user achieve their weight goal."

[1432] The device notifies the user of the generated exercise plan and displays feedback as a dashboard.

[1433] Examples of feedback and monitoring

[1434] The user goes for a 30-minute jog on Monday, and the data is collected by the wearable device and sent to the server via the smartphone.

[1435] The server analyzes the acquired jogging data, evaluates the user's progress based on their heart rate and calories burned, and adjusts the exercise plan for the next week and sends it back to the device.

[1436] Through these processes, the system comprehensively manages the user's behavior and physiological information, supporting continuous exercise and health management.

[1437] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1438] Step 1: Data collection

[1439] A user takes a photo of a meal using a smartphone app. The input is the photo of the meal, and the output is the photo data.

[1440] The device (smartphone) previews the captured photo within the app and asks the user for confirmation. Once the user has finished checking, they press the "Send" button to send the photo data to the server. Specifically, data is sent using the HTTP protocol.

[1441] When a user starts exercising, the terminal (wearable device) collects physiological information such as heart rate, number of steps, and distance traveled in real time. The input is the user's physiological information, and the output is the collected physiological data.

[1442] The device (smartphone) sends the collected physiological data in bulk to a server. The data is synchronized to the smartphone using Bluetooth or Wi-Fi, and then transmitted to the server using the HTTP protocol.

[1443] Step 2: Data analysis

[1444] The server analyzes the received meal photos using image recognition technology (e.g., Google Cloud Vision API). The input is the photo data of the meal, and the output is the analysis results, such as the types and amounts of ingredients. For example, the prompt might be, "Please identify the nutrients contained in this meal."

[1445] The server extracts nutrient information (calories, protein, fat, carbohydrates) based on the image recognition results. Specifically, it identifies the nutrient information by referencing a food database.

[1446] The server analyzes physiological data (heart rate, steps, distance traveled) sent from the wearable device. The input is physiological data, and the output is analysis results such as the type and intensity of exercise, and calories burned. Python and TensorFlow are used to clean and preprocess the data, and a machine learning model (e.g., regression model) is used to calculate calories burned. The prompt is "Please calculate calories burned from this jogging data."

[1447] Step 3: Generate an exercise plan

[1448] The server generates an exercise plan based on the user's self-defined goals (e.g., weight loss or muscle gain). The input is the user's goals and analyzed physiological and nutritional data, and the output is a customized exercise plan.

[1449] A generative AI model (e.g., OpenAI GPT-4) is used to create detailed exercise plans. The prompt is, "Generate an exercise plan that will help the user achieve their goal weight."

[1450] The server generates a plan that takes into account the type of exercise (e.g., jogging, strength training), time, and frequency, and sends it to the device in JSON format.

[1451] Step 4: Feedback and monitoring

[1452] The terminal notifies the user of the exercise plan received from the server. The input is the generated exercise plan, and the output is the notification of the exercise plan presented to the user.

[1453] The user exercises based on the presented plan. After the exercise is completed, the wearable device collects data again and sends it to the server via the smartphone. The input is the exercise data performed, and the output is an updated database in the server.

[1454] The server analyzes the user's progress and dynamically updates the exercise plan using the generative AI model as needed. The prompt is, "Please adjust next week's exercise plan based on your exercise progress." The updated exercise plan is then notified to the user via their device.

[1455] (Application example 1)

[1456] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1457] In today's busy lifestyles, it is difficult for users to find and consistently follow an exercise plan that is optimal for them. Furthermore, balancing proper nutritional management and exercise is not easy, and users require specialized knowledge to achieve effective results. Therefore, there is a need for a system that provides individually customized exercise plans and nutritional management and effectively supports users in achieving their goals.

[1458] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1459] In this invention, the server includes means for acquiring physiological information of a user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as needed, means for analyzing photo data of meals to extract nutrient information, and means for providing training feedback in real time. This allows users to receive an optimal exercise plan and appropriate nutritional management even without specialized knowledge, and to be effectively supported continuously toward achieving their goals.

[1460] "User's physiological information" refers to data related to the user's physical activity, such as heart rate, number of steps taken, and distance traveled.

[1461] "Calories burned" refers to the amount of energy consumed by a user through exercise and daily activities.

[1462] An "exercise plan" is a program that plans specific exercise content and schedules based on goals set by the user.

[1463] "Nutrient information" refers to information on ingredients such as calories, protein, fat, and carbohydrates extracted from dietary data.

[1464] "Real-time" refers to a process that instantly collects data about the user's ongoing activities and provides analysis results.

[1465] "Feedback" refers to evaluations and advice returned based on a user's activities, providing information that will be useful for the next action.

[1466] "Adaptive updating" refers to dynamically changing and adjusting the exercise plan according to the user's progress and condition.

[1467] A "wearable device" is a device that can be worn by a user and can measure physiological information such as heart rate and number of steps.

[1468] A "smartphone" is an evolved form of a mobile phone, a multi-function device that can connect to the Internet and use applications.

[1469] "Smart glasses" are devices that have the shape of glasses, are equipped with a display and sensors, and can present information to the user in real time.

[1470] A "server" is a computer that stores, manages, and processes data over a network, and serves as the central hub of the entire system.

[1471] Overall system overview

[1472] The system is designed to encourage users to continue their exercise habits and provide individually customized exercise plans and nutritional management. The system operates using the user's smartphone, wearable device, server, smart glasses, or head-mounted display.

[1473] Hardware and software used

[1474] Hardware

[1475] 1. Smartphone: Used for data collection and as a user interface.

[1476] 2. Wearable devices: Collect the user's physiological information (heart rate, steps, distance, etc.) in real time.

[1477] 3. Smart glasses or head-mounted displays: display real-time training feedback and provide exercise instructions to the user.

[1478] 4. Server: Analyzes data, generates exercise plans, and monitors user progress.

[1479] software

[1480] 1. Flask: Used as a server-side framework.

[1481] 2. Keras: Uses deep learning models (generative AI models) for image recognition and data analysis.

[1482] 3. OpenCV: Used as an image processing library.

[1483] System processing flow

[1484] Data collection

[1485] Users use a smartphone app to take photos of their meals and send them to a server. The wearable device also collects physiological information such as heart rate and number of steps taken while the user is exercising, and sends the data to the server via the smartphone.

[1486] Data analysis

[1487] The server receives photo data of meals using Flask and performs image analysis using a Keras model. Specifically, it identifies the contents of the meal and extracts information about each nutrient. Exercise data is processed using OpenCV to calculate the user's calories burned and exercise intensity.

[1488] Generate exercise plans

[1489] The server generates a personalized exercise plan based on the user's set goals (e.g., weight loss, muscle gain), detailing the type, duration, and frequency of exercise, and dynamically updates the plan to fit the user's daily routine.

[1490] Feedback and Monitoring

[1491] The server presents the generated exercise plan to the user via a smartphone or smart glasses. The user's progress is continuously monitored via the wearable device, and the exercise plan is adaptively updated as needed.

[1492] Specific examples

[1493] 1. Specific examples of data collection

[1494] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1495] The wearable device measures the user's heart rate and distance traveled while jogging and transmits the data to a server.

[1496] 2. Specific examples of data analysis

[1497] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[1498] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1499] 3. Example of motor plan generation

[1500] The server generates an exercise plan based on the user's weight loss goal, such as "30 minutes of jogging on Monday and 45 minutes of strength training on Tuesday."

[1501] The smartphone notifies the user of this exercise plan and displays it as feedback.

[1502] 4. Examples of feedback and monitoring

[1503] The user follows the proposed plan and jogs for 30 minutes on Monday.

[1504] The wearable device transmits data about the jogging that has been performed to a server.

[1505] The server analyzes the user's heart rate and distance data, calculates calories burned, evaluates progress, and, if necessary, adjusts the exercise plan for the next week and sends it back to the smartphone.

[1506] Examples of prompt statements

[1507] Analyze the user's breakfast photo data and extract the nutrients (calories, protein, fat, carbohydrates, etc.) contained in it. Also, analyze the previous day's jogging data and calculate the calories burned. Generate a new exercise plan based on the user's weight loss goal, including "30 minutes of jogging on Monday" and "45 minutes of strength training on Tuesday."

[1508] In this way, a system is constructed by combining specific hardware and software to provide optimal exercise and nutrition management for users to achieve their goals.

[1509] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1510] Step 1:

[1511] Data collection

[1512] Input: A user opens a smartphone app and takes a photo of their breakfast, while a wearable device monitors their heart rate, steps taken, and distance traveled in real time.

[1513] Data processing / calculation: The smartphone sends the photograph of the meal to the server via the app. The wearable device sends the measurement data to the server via the smartphone.

[1514] Output: The server receives the food photo data and physiological information data.

[1515] Specific operation: The user operates the app and takes a photo. The wearable device automatically records the measurement data and sends it to a smartphone.

[1516] Step 2:

[1517] Data Analysis - Dietary Data

[1518] Input: The server receives a photo of a meal sent by the user.

[1519] Data processing / calculation: The server uses a Keras model to analyze the photo data and identify the food contents. It then uses OpenCV to perform image processing and extract nutritional information such as calories, protein, fat, and carbohydrates.

[1520] Output: Extracted nutrient information is obtained.

[1521] Specific operation: The server uses an AI model to recognize foods in an image and output specific nutritional data.

[1522] Step 3:

[1523] Data Analysis - Exercise Data

[1524] Input: The server receives the wearable device movement data sent by the user.

[1525] Data processing / calculation: The server calculates the user's calories burned and exercise intensity based on the received physiological information.

[1526] Output: Calculated calorie consumption and exercise intensity data are obtained.

[1527] Specific operation: The server analyzes the received data and stores the calculation results.

[1528] Step 4:

[1529] Exercise plan generation

[1530] Input: The server receives the analyzed nutrition information, exercise data, and user-defined goals.

[1531] Data processing / calculation: The server uses this data to generate an exercise plan that is optimal for the user's goals. The generative AI model dynamically adjusts the exercise content and schedule.

[1532] Output: A customized exercise plan is generated.

[1533] Specific operation: The server uses the AI ​​model to combine the data analysis results, calculate and output the optimal plan.

[1534] Step 5:

[1535] Plan presentation

[1536] Input: Server-generated exercise plan.

[1537] Data processing / calculation: The exercise plan is sent from the server to the device (smartphone, smart glasses, etc.).

[1538] Output: The exercise plan is displayed on the user's device.

[1539] Specific operation: The server sends the plan data along with the notification message to the terminal, which displays it through the user interface.

[1540] Step 6:

[1541] Feedback and progress monitoring

[1542] Input: User progress data (exercise performance, heart rate variability, etc.) is collected from a wearable device.

[1543] Data processing / calculation: The server compares the planned exercise content with the actual activity based on the progress data and adjusts the exercise plan if necessary.

[1544] Output: A new, adjusted exercise plan.

[1545] How it works: The server analyzes the progress data, adjusts the plan using an AI model, and sends the new plan back to the device.

[1546] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1547] Overall overview

[1548] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect their exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[1549] Data collection

[1550] Users take photos of their meals using a smartphone app. They also use a wearable device to collect daily exercise data. The smartphone app then uses the camera and voice input to recognize the user's emotions. The device then sends this data to a server, which then collects detailed user data.

[1551] Data analysis

[1552] The server uses an AI algorithm to analyze the received food photos and extract nutritional information from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state.

[1553] Generate exercise plans

[1554] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if stress levels are high, the server may recommend exercise that has a relaxing effect. The generated plan includes detailed exercise content, schedule, and frequency.

[1555] Feedback and Monitoring

[1556] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[1557] Specific examples

[1558] Examples of data collection

[1559] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1560] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1561] Users can report their emotional state after exercise using a smartphone app, or the app can automatically collect data by analyzing emotions through facial recognition and voice tone.

[1562] Specific examples of data analysis

[1563] The server analyzes the breakfast photo and extracts nutritional information (e.g., calorie, protein, fat, and carbohydrate content).

[1564] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1565] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[1566] Example of generating an exercise plan

[1567] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, the server might generate a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[1568] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[1569] Examples of feedback and monitoring

[1570] The user follows the plan and does 30 minutes of yoga on Mondays.

[1571] The terminal transmits data on the yoga session performed and subsequent emotional data (for example, stress reduction effect) to the server.

[1572] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[1573] In this way, the system manages the user's behavior, physiological information, and emotional information in an integrated manner, supporting continuous exercise and health management.

[1574] The processing flow will be explained below.

[1575] Step 1:

[1576] Users open the app on their smartphone and take a photo of their meal.

[1577] The device (smartphone) automatically sends the photograph of the meal to the server.

[1578] Step 2:

[1579] When the user starts exercising, the terminal (wearable device) automatically collects data such as heart rate, number of steps, and distance traveled.

[1580] The collected data is sent to the server in real time via the terminal.

[1581] Step 3:

[1582] Users can report their emotional state after exercise in the app, or the app will automatically analyze emotions through facial recognition and voice tone.

[1583] When a user taps an emotion button, the emotion information is also sent to the server.

[1584] Step 4:

[1585] The server uses an AI algorithm to analyze the received photos of the meal and identify the contents of the meal.

[1586] It also extracts information about each nutrient (e.g., amount of calories, protein, fat, carbohydrates).

[1587] Step 5:

[1588] The server analyzes the exercise data sent from the wearable device.

[1589] This includes calculating the type of exercise (e.g., running, walking), the intensity of the exercise, and the calories burned.

[1590] Step 6:

[1591] The server analyzes the received emotion data using an emotion engine.

[1592] Identify and assess your current emotional state (e.g., stress level, satisfaction, fatigue, etc.).

[1593] Step 7:

[1594] The server generates an optimal exercise plan for the user based on the user's self-set goals (e.g., weight loss, muscle gain).

[1595] This exercise plan is customized taking into account your emotional state as well as your physical data.

[1596] Step 8:

[1597] The server transmits the generated exercise plan to the terminal.

[1598] The device notifies the user of the exercise plan and displays detailed exercise content and schedule as feedback.

[1599] Step 9:

[1600] The user follows the notified exercise plan and performs the exercise.

[1601] The terminal (wearable device) continuously collects exercise data performed by the user and transmits it to a server.

[1602] Step 10:

[1603] The server periodically monitors the user's progress.

[1604] This includes analyzing changes in body composition and exercise outcomes (e.g., weight loss, muscle mass gain).

[1605] Step 11:

[1606] The server uses an emotion engine to analyze the user's emotion data and also monitor the user's psychological state.

[1607] Adaptively update your exercise plan and make real-time adjustments as needed.

[1608] Step 12:

[1609] The server then transmits the updated exercise plan to the terminal again.

[1610] The device notifies the user of the updated exercise plan and presents detailed exercise content and schedule again as feedback.

[1611] Step 13:

[1612] The user then exercises again based on the new exercise plan and continues to receive feedback and monitoring.

[1613] By linking the device and the server, the user's physical and emotional data are managed in an integrated manner, allowing optimal exercise instruction to be provided.

[1614] Example 2

[1615] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1616] Conventional systems primarily generate exercise plans based on the user's physiological information and exercise data, but because they do not take the user's emotional state into account, the exercise plan is not optimized for the user's individual psychological state. This can reduce the user's motivation to continue exercising, potentially reducing the effectiveness of health management.

[1617] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the calories burned by the user based on the acquired physiological information, means for generating an exercise plan based on a goal set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for acquiring and analyzing emotional information of the user, and means for adjusting the exercise plan based on the analyzed emotional information. This makes it possible to provide an optimal exercise plan according to the user's individual psychological state.

[1618] 1. "User" refers to a person who uses this system to provide exercise, diet, and emotional data and receives feedback based on that data.

[1619] 2. "Physiological information" refers to the user's physical data, such as heart rate, calories burned, and type and intensity of exercise.

[1620] 3. "Exercise Plan" refers to a personalized exercise schedule or program generated based on a user's physiological information, goals, and emotional information.

[1621] 4. "Emotional information" refers to data that indicates a user's emotional state (e.g., stress, satisfaction, etc.), and is primarily collected through facial recognition, voice tone analysis, or user self-reporting.

[1622] 5. "Server" refers to a computer system that receives and analyzes data sent from users and devices, and provides the generated exercise plan to users.

[1623] 6. "Device" refers to an electronic device, such as a smartphone or wearable device, that is used to collect and transmit user data to a server.

[1624] 7. "Means for acquiring" refers to methods and devices for collecting physiological and emotional information of a user, such as sensors, cameras, microphones, etc.

[1625] 8. "Analytical means" refers to the algorithms and software used to process the collected data and evaluate and analyze the user's condition.

[1626] 9. "Means for generating" refers to the method or algorithm for generating an exercise plan based on the analysis results.

[1627] 10. "Presentation means" refers to a method or device for notifying the user of the generated exercise plan, such as a smartphone notification function or app.

[1628] 11. "Monitoring means" refers to the methods and functions that continuously observe and collect user progress and data, and adjust exercise plans as needed.

[1629] Overall overview

[1630] This system is designed to help users maintain their exercise habits, and by combining it with an emotion engine, it further optimizes exercise plans based on the user's emotional information. Users use a smartphone app and wearable device to collect exercise, diet, and emotional data, which is then analyzed by a server to provide individually optimized exercise plans. The emotion engine also takes the user's psychological state into account, providing more personalized feedback.

[1631] Data collection

[1632] Users use a smartphone app to take photos of their meals, which are then analyzed to record the details of their meals. The user also uses a wearable device to collect daily exercise data, including heart rate, distance traveled, and calories burned. Furthermore, the smartphone app recognizes the user's emotions through camera and voice input. All of this data is sent from the device to a server, where detailed user data is collected.

[1633] Data analysis

[1634] The server analyzes the received food photos using AI algorithms (e.g., TensorFlow or PyTorch) and extracts nutritional information from the meal contents. At the same time, it analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. It uses an emotion engine to analyze the user's emotional data and identify their current psychological state. This allows the server to comprehensively evaluate the user's physiological information, exercise data, and emotional data.

[1635] Generate exercise plans

[1636] The server generates an exercise plan based on the user's self-set goals (e.g., target weight, exercise time) and emotional data. This exercise plan is customized taking into account not only the user's physical data but also their emotional state. For example, if the user is in a high-stress state, it will recommend exercise that has a relaxing effect (e.g., yoga or relaxing walking). The generated exercise plan includes detailed exercise content, schedule, and frequency.

[1637] Feedback and Monitoring

[1638] The server sends the generated exercise plan to the device, which then notifies the user. The user then exercises according to the plan. The user's progress is continuously monitored, and the exercise plan is adaptively updated as needed. Emotional data is also periodically collected and reflected in the feedback.

[1639] Specific examples

[1640] Examples of data collection

[1641] Users take a photo of their breakfast with their smartphone and send it to the server via the app.

[1642] The terminal (wearable device) measures the user's heart rate and distance traveled during their morning jog and sends the data to a server.

[1643] Users report their emotional state after exercise using a smartphone app, which can automatically collect emotions using facial recognition and voice tone analysis.

[1644] Specific examples of data analysis

[1645] The server analyzes photos of breakfast and uses AI algorithms to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate amounts).

[1646] The server analyzes the jogging data obtained from the wearable device and calculates the user's calorie expenditure.

[1647] The server uses an emotion engine to analyze the stress and satisfaction felt by the user after exercise.

[1648] Example of generating an exercise plan

[1649] The server generates an exercise plan based on the user's weight loss goal and stress level. For example, if the user is in a high stress state, it generates a plan such as "30 minutes of light yoga on Monday and relaxing walking on Tuesday."

[1650] The terminal notifies the user of the generated exercise plan and displays it as feedback.

[1651] Examples of feedback and monitoring

[1652] The user follows the plan and does 30 minutes of yoga on Mondays.

[1653] The terminal transmits data on the yoga performed and subsequent emotional data (e.g., stress reduction effect) to the server.

[1654] The server analyzes the user's exercise data and emotional data, adjusts the exercise plan for the next week as necessary, and resends it to the device.

[1655] This system uses a generative AI model to integrate and analyze the user's physiological and emotional information, generate an individually optimized exercise plan, and provide feedback to support continuous exercise and health management.

[1656] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1657] Step 1:

[1658] A user takes a photo of their meal using a smartphone app.

[1659] Input: Food photo

[1660] Output: Food photo data sent to a smartphone app

[1661] How it works: A user opens a smartphone app, takes a photo of a meal, and sends the data through the app.

[1662] Step 2:

[1663] A user wears a wearable device to collect exercise data.

[1664] Input: Physiological data during exercise (heart rate, distance traveled, etc.)

[1665] Output: Exercise data recorded on a wearable device

[1666] How it works: A user puts on the wearable device and starts exercising. The device continuously records exercise data.

[1667] Step 3:

[1668] A smartphone app recognizes the user's emotional state.

[1669] Input: User facial expressions and tone of voice

[1670] Output: Emotional state data

[1671] How it works: A smartphone app uses the camera and microphone to analyze the user's facial expressions and voice to determine their emotional state.

[1672] Step 4:

[1673] The device sends the collected data (meal photos, exercise data, emotional data) to the server.

[1674] Input: Food photo data, exercise data, emotion data

[1675] Output: Detailed user data sent to the server

[1676] How it works: Your device sends data over the internet to a server.

[1677] Step 5:

[1678] The server analyzes the received food photos and extracts nutritional information.

[1679] Input: Food photo data

[1680] Output: Nutrition information (calories, protein, fat, carbohydrates, etc.)

[1681] How it works: The server uses AI algorithms to analyze the image, identify the contents of the meal, and extract nutritional information.

[1682] Step 6:

[1683] The server analyzes the received exercise data and calculates the calories burned.

[1684] Input: Exercise data (heart rate, distance traveled, etc.)

[1685] Output: Calories burned, exercise type, intensity, etc.

[1686] Operation: The server analyzes the exercise data and performs various data calculations to calculate detailed exercise information and calories burned.

[1687] Step 7:

[1688] The server uses an emotion engine to analyze the user's emotion data.

[1689] Input: Emotional state data

[1690] Output: Current psychological state (e.g., stress level, satisfaction, etc.)

[1691] How it works: The server uses the emotion engine to analyze the user's emotion data and identify their psychological state.

[1692] Step 8:

[1693] The server generates an exercise plan based on the user's goals and the analyzed data.

[1694] Input: Goal data, Nutrition information, Exercise data, Emotion data

[1695] Output: Individually optimized exercise plans

[1696] How it works: The server comprehensively analyzes all the data it collects and creates the optimal exercise plan for the user.

[1697] Step 9:

[1698] The server sends the generated exercise plan to the terminal.

[1699] Input: Personalized exercise plan

[1700] Output: Exercise plan sent to device

[1701] Operation: The server sends the generated exercise plan to the device and notifies the user.

[1702] Step 10:

[1703] The user exercises according to the exercise plan presented to them.

[1704] Input: Exercise plan

[1705] Output: Exercise data performed

[1706] Action: The user performs the specified exercise according to the exercise plan received from the device.

[1707] Step 11:

[1708] The terminal transmits the exercise data and emotion data that have been performed to the server again.

[1709] Input: Exercise data, emotion data

[1710] Output: Data to be resent to the server

[1711] Operation: The device sends the exercise data and emotion data back to the server.

[1712] Step 12:

[1713] The server analyzes the collected data and adaptively updates the exercise plan.

[1714] Input: Exercise data, emotion data

[1715] Output: Updated exercise plan

[1716] How it works: The server analyzes the newly collected data, adaptively updates the exercise plan as needed, and resends it to the device.

[1717] The above is a specific operation of the processing steps of this system.

[1718] (Application example 2)

[1719] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1720] Maintaining health and establishing exercise habits are important issues in modern society. However, there are few systems that provide optimal exercise and meal plans that take into account the user's physiological information and emotional state. In particular, there are no systems that integrate meal suggestions and automatic ordering functions, making it difficult for users to consistently manage their health.

[1721] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring physiological information of the user, means for calculating the user's calorie expenditure based on the acquired physiological information, means for generating an exercise plan based on goals set by the user, means for presenting the generated exercise plan to the user, means for monitoring the user's progress and adaptively updating the exercise plan as necessary, means for recognizing the user's emotional state and reflecting it in the exercise plan, means for proposing an optimal meal plan, and means for automatically ordering meals based on the proposed meal plan. This integrates the user's physiological information and emotional information to provide an optimal exercise and meal plan, enabling the user to maintain a healthy lifestyle and establish exercise habits.

[1722] "User's physiological information" is data that indicates the user's health condition, such as physical activity, heart rate, calories burned, and sleep patterns.

[1723] The "means for calculating calories burned" is a system or device that calculates the amount of energy consumed by the user based on the user's exercise data and physiological information.

[1724] An "exercise plan" is a plan that shows an exercise schedule and content that is optimized based on the user's health goals and physiological information.

[1725] "User goals" are health goals set by the user, such as weight loss, muscle gain, etc.

[1726] A "progress monitoring means" is a system or device that collects a user's exercise and health data in real time or at regular intervals and checks their progress.

[1727] An "adaptive updating means" is a system or device that dynamically modifies the exercise plan based on the user's progress or new physiological information.

[1728] "Emotional state" is data that indicates the user's psychological state, including stress level, happiness, and the like.

[1729] The "means for reflecting in exercise plan" is a system or device that customizes an exercise plan taking into account the user's emotional state.

[1730] A "meal plan" is a plan that shows optimized dietary content and nutrient balance that takes into account the user's health goals, physiological information, and emotional state.

[1731] An "automated ordering means" is a system or device that automatically orders meals based on a proposed meal plan.

[1732] The present invention revolves around a system for optimizing a user's health management. The system is realized mainly using a smartphone, a wearable device, and a server. Specific embodiments for carrying out the invention are described below.

[1733] Data collection methods

[1734] Users use a smartphone app to take photos of their meals and send them to a server via the app. In addition, users use wearable devices (e.g., fitness trackers) to collect daily exercise data (e.g., steps taken, heart rate). The smartphone app also uses camera and voice input to recognize and collect data on the user's emotional state (e.g., stress level, happiness). The device then sends this data to a server, which then stores detailed user data.

[1735] Data Analysis Methods

[1736] The server uses an AI algorithm to analyze the received food photos and extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. At the same time, it analyzes exercise data sent from the wearable device to calculate the type of exercise, intensity, and calories burned. It also uses an emotion engine to analyze the user's emotional data and identify their current psychological state. The software used includes Python programs and machine learning models (e.g., TensorFlow, PyTorch).

[1737] Movement plan generation means

[1738] The server generates an exercise plan based on the user's self-set goals and emotional data. The generated exercise plan is customized by taking into account not only the user's physical data (e.g., daily calorie consumption, exercise history) but also their emotional state. For example, if stress levels are high, relaxing exercises (e.g., yoga or relaxing walking) will be recommended. The generated plan includes detailed exercise content, schedule, and frequency.

[1739] Meal planning and automated ordering tools

[1740] The server proposes an optimal meal plan (e.g., grilled chicken salad) based on the user's exercise data, emotional data, and nutritional analysis results. The server then provides the user with a one-click meal ordering function. Based on the proposed meal plan, the server automatically orders the meal.

[1741] Feedback and monitoring measures

[1742] The user exercises according to the presented exercise plan and eats according to the meal plan. Afterwards, they record their emotional state and exercise progress using a wearable device or smartphone app, and the data is sent to the server. The server analyzes this data and reflects it in their next exercise and meal plan.

[1743] Examples and prompts

[1744] For example, if the user jogs 5,000 steps and their stress level is 3, they will be suggested to order a "grilled chicken salad." An example prompt is as follows:

[1745] Consider today's exercise data and emotional state to suggest the optimal meal plan. The user's exercise data is 5000 steps of jogging, and their emotional state is stress level 3.

[1746] In this way, the system of the present invention can comprehensively manage a user's behavior, physiological information, and emotional information, and support continuous health management.

[1747] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1748] Step 1:

[1749] A user takes a photo of a meal using a smartphone app and sends it to the server through the app. Specifically, the user takes a photo of the meal and taps the send button. The input is the photo data of the meal, and the output is that this data is sent to the server.

[1750] Step 2:

[1751] The terminal (wearable device) acquires daily exercise data (e.g., number of steps, heart rate) and sends it to a server. Specifically, the wearable device records the user's exercise information using sensors. The input is the user's exercise data, and the output is that this data is sent to the server.

[1752] Step 3:

[1753] The smartphone app uses a camera or voice input to recognize the user's emotional state and sends the data to a server. Specifically, the app recognizes the user's face and analyzes their voice to extract emotional data. The input is the user's face image or voice data, and the output is data indicating the user's emotional state, which is sent to the server.

[1754] Step 4:

[1755] The server analyzes the received food photos using an AI algorithm to extract nutritional information (e.g., calorie, protein, fat, and carbohydrate content) from the meal. Specifically, the AI ​​model analyzes the food photos and retrieves nutritional information from a database. The input is food photo data, and the output is nutritional information.

[1756] Step 5:

[1757] The server analyzes the exercise data sent from the wearable device and calculates the type of exercise, intensity, and calories burned. Specifically, it processes the exercise data using a data analysis tool and calculates calories burned. The input is the exercise data, and the output is the exercise analysis results, such as calories burned.

[1758] Step 6:

[1759] The server uses an emotion engine to analyze the user's emotion data and identify their current psychological state. Specifically, it applies an emotion analysis algorithm to classify the emotional state. The input is emotion data, and the output is data indicating the user's psychological state.

[1760] Step 7:

[1761] The server generates an exercise plan based on the user's self-defined goals, emotional data, and movement data. Specifically, a custom algorithm integrates various data to generate an optimal exercise plan. The inputs are goal data, movement data, and emotional data, and the output is the exercise plan.

[1762] Step 8:

[1763] The server proposes an optimal meal plan based on the user's exercise data, emotional data, and nutrient analysis results. Specifically, it generates a meal plan using a nutrition calculation algorithm. The inputs are exercise data, emotional data, and nutrient data, and the output is a meal plan.

[1764] Step 9:

[1765] It provides an interface for users to order meals with one click, and the server automatically orders meals based on the proposed meal plan. Specifically, it calls the ordering API to confirm the meal order. The input is the meal plan, and the output is a notification that the order has been completed.

[1766] Step 10:

[1767] The user exercises according to the presented exercise plan and records their emotional state and exercise progress using a wearable device or smartphone app. The data is then sent to a server. Specifically, the user exercises, and the wearable device then records the data. The input is exercise data and emotional data, and the output is the data sent to the server.

[1768] Step 11:

[1769] The server analyzes the user's exercise and emotional data and dynamically updates the exercise and meal plans as needed. Specifically, it runs a reanalysis algorithm to generate new plans. The input is the latest exercise and emotional data, and the output is the updated exercise and meal plans.

[1770] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1771] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1772] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1773] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1774] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1775] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1776] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1777] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1778] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1779] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1780] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1781] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1782] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1783] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1784] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1785] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1786] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1787] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1788] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1789] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1790] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1791] The following is further disclosed regarding the above embodiment.

[1792] (Claim 1)

[1793] A means for acquiring physiological information of a user;

[1794] means for calculating the calorie consumption of the user based on the acquired physiological information;

[1795] means for generating an exercise plan based on a user's self-defined goals;

[1796] means for presenting the generated exercise plan to a user;

[1797] The system includes a means for monitoring a user's progress and adaptively updating the exercise plan as needed.

[1798] (Claim 2)

[1799] A means for analyzing the user's dietary content;

[1800] and means for incorporating nutritional information of the analyzed dietary content into an exercise plan.

[1801] (Claim 3)

[1802] A means for acquiring physiological information of a user in real time using a wearable device;

[1803] and means for adjusting exercise intensity based on the acquired physiological information.

[1804] "Example 1"

[1805] (Claim 1)

[1806] A means for acquiring physiological information of a user;

[1807] means for calculating the calorie consumption of the user based on the acquired physiological information;

[1808] a means for utilizing the generative AI model to generate an exercise plan based on a user's self-defined goals;

[1809] means for presenting the generated exercise plan to a user;

[1810] The system includes a means for monitoring a user's progress and adaptively updating the exercise plan as needed.

[1811] (Claim 2)

[1812] A means for analyzing the user's dietary content;

[1813] and means for using image recognition technology to incorporate the nutritional information of the analyzed dietary content into an exercise plan.

[1814] (Claim 3)

[1815] A means for acquiring physiological information of a user in real time using a wearable device;

[1816] and means for adjusting exercise intensity based on the acquired physiological information.

[1817] "Application Example 1"

[1818] (Claim 1)

[1819] A means for acquiring physiological information of a user;

[1820] means for calculating the calorie consumption of the user based on the acquired physiological information;

[1821] means for generating an exercise plan based on a user's self-defined goals;

[1822] means for presenting the generated exercise plan to a user;

[1823] means for monitoring a user's progress and adaptively updating the exercise plan as needed;

[1824] A means for analyzing food photo data and extracting nutrient information;

[1825] a means of providing real-time training feedback;

[1826] A system including:

[1827] (Claim 2)

[1828] A means for analyzing the user's dietary content;

[1829] A means for reflecting the analyzed nutritional information of the dietary content in an exercise plan;

[1830] and means for adjusting the training plan based on the exercise data in real time.

[1831] (Claim 3)

[1832] A means for acquiring physiological information of a user in real time using a wearable device;

[1833] a means for adjusting exercise intensity based on the acquired physiological information;

[1834] and means for presenting the generated exercise plan to the user via a smartphone or smart glasses.

[1835] "Example 2: Combining Emotion Engines"

[1836] (Claim 1)

[1837] A means for acquiring physiological information of a user;

[1838] means for calculating the calorie consumption of the user based on the acquired physiological information;

[1839] means for generating an exercise plan based on a user's self-defined goals;

[1840] means for presenting the generated exercise plan to a user;

[1841] means for monitoring a user's progress and adaptively updating the exercise plan as needed;

[1842] A means for acquiring and analyzing user emotion information;

[1843] The system includes means for adjusting the movement plan based on the analyzed emotional information.

[1844] (Claim 2)

[1845] A means for analyzing the user's dietary content;

[1846] and means for incorporating nutritional information of the analyzed dietary content into an exercise plan.

[1847] (Claim 3)

[1848] A means for acquiring physiological information of a user in real time using a wearable device;

[1849] and means for adjusting exercise intensity based on the acquired physiological information.

[1850] "Application example 2 when combining emotion engines"

[1851] (Claim 1)

[1852] A means for acquiring physiological information of a user;

[1853] means for calculating the calorie consumption of the user based on the acquired physiological information;

[1854] means for generating an exercise plan based on a user's self-defined goals;

[1855] means for presenting the generated exercise plan to a user;

[1856] means for monitoring a user's progress and adaptively updating the exercise plan as needed;

[1857] a means for recognizing the user's emotional state and incorporating it into the exercise plan;

[1858] A way to suggest the best meal plan,

[1859] The system includes a means for automatically ordering meals based on the proposed meal plan.

[1860] (Claim 2)

[1861] A means for analyzing the user's dietary content;

[1862] A means for reflecting the analyzed nutritional information of the dietary content in an exercise plan;

[1863] and means for generating a meal plan and automatically ordering meals based on the user's exercise data and emotion data.

[1864] (Claim 3)

[1865] A means for acquiring physiological information and emotional information of a user in real time using a wearable device;

[1866] and means for adjusting exercise intensity based on the acquired physiological information and for suggesting a meal plan based on the emotional information. [Explanation of symbols]

[1867] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring physiological information of a user; means for calculating the calorie consumption of the user based on the acquired physiological information; means for generating an exercise plan based on a user's self-defined goals; means for presenting the generated exercise plan to a user; The system includes a means for monitoring a user's progress and adaptively updating the exercise plan as needed.

2. A means for analyzing the user's dietary content; The system according to claim 1, further comprising: means for reflecting the nutritional information of the analyzed meal contents in an exercise plan.

3. A means for acquiring physiological information of a user in real time using a wearable device; and means for adjusting exercise intensity based on the acquired physiological information.

Images