System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input personal data such as height, weight, lifestyle, and target weight; means for uploading a picture of a meal; means for analyzing the picture of the meal and calculating calories; means for suggesting an exercise method based on the personal data and the calorie information; means for providing the suggested exercise method to the user; and means for recording and analyzing exercise performance data of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people find it difficult to successfully diet healthily. The main reasons for this include a lack of specialized knowledge about dietary management and exercise methods, and the high cost of services such as personal training. If these issues are ignored, users may be unable to effectively manage their weight, which could lead to health problems. Therefore, the present invention aims to solve these problems. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a diet support system that allows users to input personal data such as height, weight, lifestyle habits, and target weight, and upload photos of meals. The system includes the following means.

[0006] 1. Means for entering the user's personal data.

[0007] 2. A way to upload photos of your meals.

[0008] 3. A way to analyze photos of food and calculate calories.

[0009] 4. A means of providing personalized exercise recommendations based on personal data and calorie information.

[0010] 5. A means of providing suggested exercise methods to the user.

[0011] 6. A means for recording and analyzing user exercise performance data.

[0012] Furthermore, by including a means for displaying the analyzed calorie information and suggested exercise methods on the user interface, and a means for accumulating collected data and continuously improving the generative model, it becomes possible to provide each user with an optimal diet plan and support the user's health management.

[0013] "User" refers to an individual who uses the system to receive diet support.

[0014] "Personal data" refers to basic information necessary for the system to provide diet support, such as the user's height, weight, lifestyle habits, and target weight.

[0015] "Upload" refers to the action of a user sending a photo of a meal they have taken to the system.

[0016] "Analysis" refers to the process by which the system automatically recognizes the photos of meals received and calculates calories and nutritional information.

[0017] A "calorie" is a unit that indicates the amount of energy contained in a meal, and is an important indicator in a user's dietary management.

[0018] "Exercise Method" refers to the specific exercise and physical training plan the system suggests to help the user achieve their target weight.

[0019] "Suggestion" refers to the system providing appropriate dietary management and exercise plans based on the user's individual data and analysis results.

[0020] "User interface" refers to the screen or operation panel that allows the user to interact with the system, and is the part where information is displayed and input is performed.

[0021] "Recording" refers to the system saving data such as the exercise and dietary habits of the user.

[0022] "Analyzed calorie information" refers to detailed calorie information obtained by the system analyzing photos of meals.

[0023] "Generative models" refer to AI algorithms that are constantly improved based on collected data and are used to provide users with optimal diet plans. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] This invention is a diet support system that manages diet and suggests exercise based on personal data and food photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation.

[0046] System Overview

[0047] user:

[0048] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[0049] Device:

[0050] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results and exercise suggestions from the server to the user, allowing the user to watch exercise videos.

[0051] server:

[0052] The server receives the user's data, analyzes photos of meals using a generative AI model, and calculates calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. In addition, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0053] Program processing

[0054] 1. User data input:

[0055] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0056] 2. Upload your meal photos:

[0057] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[0058] 3. Photo analysis and calorie calculation:

[0059] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0060] 4. Exercise suggestions:

[0061] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[0062] 5. Exercise execution and recording:

[0063] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[0064] 6. Data Collection and Analysis:

[0065] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[0066] Specific examples

[0067] Let's take the example of User A using this system.

[0068] Initial Settings:

[0069] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[0070] Upload your meal photos and calculate calories:

[0071] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[0072] Exercise suggestions:

[0073] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[0074] Exercise log and next suggestion:

[0075] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[0076] As described above, the present invention is a system that effectively supports users in their healthy dieting by using a generative AI model to provide dietary management and exercise suggestions.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] user

[0080] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[0081] Step 2:

[0082] Terminal

[0083] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[0084] Step 3:

[0085] user

[0086] Users take photos of their meals and upload them to the application.

[0087] Step 4:

[0088] Terminal

[0089] The device sends the uploaded photos of the meal to the server.

[0090] Step 5:

[0091] server

[0092] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[0093] Step 6:

[0094] server

[0095] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[0096] Step 7:

[0097] Terminal

[0098] The terminal displays the calorie information and advice received from the server on a user interface.

[0099] Step 8:

[0100] user

[0101] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[0102] Step 9:

[0103] server

[0104] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[0105] Step 10:

[0106] Terminal

[0107] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[0108] Step 11:

[0109] user

[0110] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[0111] Step 12:

[0112] Terminal

[0113] The terminal transmits the recorded exercise data to a server.

[0114] Step 13:

[0115] server

[0116] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[0117] Example 1

[0118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0119] While conventional diet support systems use data and food photos entered by users to calculate calories and suggest exercises, they suffer from insufficient accuracy and individuality. Furthermore, they lack a means to effectively provide feedback based on the analysis of the data obtained and improve the overall performance of the system, making it difficult to provide optimal support to users.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes means for the user to input personal data such as height, weight, age, sex, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories using a generative AI model, means for suggesting an exercise method based on the personal data and the calorie information, means for providing the suggested exercise method to the user, means for recording the user's exercise implementation data and transmitting it to the server, and means for continuously improving the generative AI model based on the recorded data. This enables highly accurate diet support tailored to the individual user and continuous improvement of system performance.

[0122] A "user" is a user of the system who is responsible for inputting personal data such as their height, weight, age, gender, lifestyle habits, and target weight, and uploading photos of their meals.

[0123] A "terminal" is a device used by a user that provides an interface for inputting, reviewing, and sending personal data and photos of meals.

[0124] The "server" is a central system that receives and analyzes data sent by users and uses a generative AI model to calculate calories and make exercise suggestions.

[0125] "Personal data" refers to information such as height, weight, age, gender, lifestyle habits, and target weight that users enter into the system.

[0126] A "generative AI model" is an algorithm or program that uses machine learning technology to analyze data and calculate calories and suggest exercises.

[0127] "Calorie Calculation" is a method of analyzing photos of meals uploaded by users and calculating calories burned based on the type and quantity of food.

[0128] "Exercise Suggestion" is the process of providing instructions and advice on the optimal type and frequency of exercise based on the user's personal data and calorie information.

[0129] A "database" is a storage device that the server uses to store and manage users' personal data, analysis results, exercise execution data, etc.

[0130] "Continuous improvement" is the process of training the generative AI model based on accumulated data to improve the accuracy of analysis and proposals.

[0131] The present invention is a diet support system that manages dietary management and suggests exercise based on personal data and meal photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation. The detailed configuration is described below.

[0132] System Overview

[0133] user:

[0134] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[0135] Device:

[0136] The device is a smartphone, tablet, or other device that transmits data entered by the user and uploaded meal photos to the server through a user interface. The device also displays analysis results and exercise suggestions from the server to the user, and allows the user to watch exercise videos.

[0137] server:

[0138] The server uses cloud-based computing resources to receive user data, analyzes food photos using a generative AI model, calculates calories, and generates optimal exercise and diet plans for the user based on the analyzed data. Additionally, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0139] Program processing

[0140] User data input:

[0141] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0142] Upload your meal photo:

[0143] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[0144] Photo analysis and calorie counting:

[0145] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0146] Exercise suggestions:

[0147] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[0148] Exercise execution and recording:

[0149] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[0150] Data collection and analysis:

[0151] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[0152] Specific examples

[0153] Let's take the example of User A using this system.

[0154] Initial Settings:

[0155] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[0156] Upload your meal photos and calculate calories:

[0157] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[0158] Exercise suggestions:

[0159] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[0160] Exercise log and next suggestion:

[0161] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[0162] Example prompt sentence:

[0163] "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is recommended."

[0164] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0165] Step 1:

[0166] User Data Entry

[0167] The user launches the application and enters personal data (height, weight, age, gender, lifestyle habits, and target weight).

[0168] The device receives this entered personal data and checks that the data is in the correct format (e.g., height and weight are numeric).

[0169] The terminal transmits the checked data to the server.

[0170] The server stores the received personal data in a database.

[0171] Input: Personal data (height, weight, age, gender, lifestyle habits, goal weight)

[0172] Output: Personal data stored in a database

[0173] Step 2:

[0174] Upload a photo of your meal

[0175] Users take photos of their daily meals and upload them through the application.

[0176] The device receives the photo and checks the resolution and format.

[0177] The device sends the confirmed photo to the server.

[0178] The server receives the photo data and stores it in a database.

[0179] Input: Food photo

[0180] Output: Photo data stored in a database

[0181] Step 3:

[0182] Photo analysis and calorie calculation

[0183] The server retrieves the photo data from the database and begins analyzing it using the generative AI model.

[0184] A generative AI model (e.g., using TensorFlow or PyTorch) is fed photos of food to identify the type and quantity of food.

[0185] The server calculates the calories for each identified food item.

[0186] The server stores the calculation results (calories of each food item and total calories) in a database and sends the results to the terminal.

[0187] The terminal receives the analysis results from the server and notifies the user.

[0188] Input: Food photo data

[0189] Output: Calorie calculation result (notification to user)

[0190] Step 4:

[0191] Exercise suggestions

[0192] The server generates the optimal exercise method based on the user's basic data (personal data) and calorie information.

[0193] A prompt sentence is input into the generative AI model to generate exercise suggestions appropriate for the user (e.g., "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is appropriate").

[0194] The server stores the generated exercise suggestions in a database and transmits them to the terminal.

[0195] The device displays the received exercise suggestions to the user and, in some cases, provides links to exercise videos.

[0196] Input: Basic data, calorie information

[0197] Output: Exercise suggestions and exercise video links (notification to user)

[0198] Step 5:

[0199] Exercise execution and recording

[0200] The user performs the suggested exercise (e.g., a 30-minute walk) and records the results in the application.

[0201] The device receives the exercise record (type of exercise performed, time, intensity, etc.) and sends it to the server.

[0202] The server stores the received exercise data in a database and reflects it in the next proposal.

[0203] Input: Exercise record data

[0204] Output: Exercise data saved in the database (reflected in the next proposal)

[0205] Step 6:

[0206] Data accumulation and analysis

[0207] The server periodically re-analyzes the accumulated user data (personal data, calorie information, exercise data).

[0208] Using a generative AI model, the model is trained based on new data, improving analysis accuracy.

[0209] The server uses the improved generative AI model to further customize your next exercise suggestion.

[0210] Input: Accumulated user data

[0211] Output: An improved generative AI model and motion suggestions based on it

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] While existing diet support systems allow users to input personal data, upload photos of their meals, and calculate calories, they lack specificity and individuality in their exercise recommendations. They also offer limited means for effectively managing diet and exercise progress. Furthermore, technological solutions are needed to enhance convenience in online and virtual environments.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for a user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for providing the suggested exercise methods to the user and for viewing exercise videos, means for recording and analyzing the user's exercise implementation data, and means for the user to log in to a virtual store to record meals and receive exercise guidance. This allows the user to receive personalized exercise suggestions in a virtual environment, enabling effective management of diet and exercise progress.

[0217] "Personal data" refers to data entered by the user, such as height, weight, lifestyle habits, and target weight.

[0218] "Meal photos" refer to image data taken by users of their own meals.

[0219] "Calorie counting" refers to the process of analyzing a photo of a meal to calculate the total calories in that meal.

[0220] "Exercise Method" refers to a specific exercise plan suggested based on the user's personal data and calorie information.

[0221] "Exercise videos" refer to video content that allows users to exercise while watching.

[0222] A "virtual store" refers to a virtual store environment that users can access online to record their meals and receive exercise guidance.

[0223] "Exercise data" refers to data such as the type of exercise the user actually performed, the duration, and the calories burned.

[0224] "User interface" refers to the display and input devices that allow a user to interact with a system.

[0225] "Generative AI model" refers to an artificial intelligence model that analyzes data collected from users and generates personalized suggestions.

[0226] The present invention is a diet support system that includes input of personal data, analysis of food photos, calorie calculation, exercise method suggestions, exercise recording and analysis, and use in a virtual store.

[0227] The server receives personal data entered by the user, such as height, weight, lifestyle habits, and target weight, and stores the data in a database. The user enters this personal data through a device such as a smartphone or PC. The device then transmits the personal data entered via the user interface to the server.

[0228] Next, users take photos of their daily meals and upload them from their devices. The server analyzes the uploaded photos using a generative AI model to calculate the calories of each meal. The analyzed calorie information is stored in a database and provided to the user. This information can be viewed through the user interface.

[0229] The server then proposes a personalized exercise plan based on the user's personal data and calorie information obtained from their diet. This includes specific exercise types and frequency, as well as links to exercise videos. The user then performs the exercises based on the proposed plan and records the results on their device. The recorded exercise data is then sent back to the server and stored in a database.

[0230] Additionally, users can log in to the virtual store to record their meals and receive exercise guidance, where they can track their progress and interact with other users for added motivation.

[0231] As a concrete example, let's consider the case of User A using this system. User A enters his / her height of 170 cm, weight of 75 kg, lifestyle habits (desk work), and target weight of 65 kg, and uploads photos of his / her daily meals. For example, if he / she uploads a photo of sushi taken for lunch, the server analyzes it and calculates the calories. Next, the system suggests that he / she walk 30 minutes every day this week and watch yoga videos three times a week. When User A records the results of his / her exercise, the results are reflected in the next exercise suggestions.

[0232] An example of a prompt sentence is, "Please analyze the photo of sushi taken by user A and calculate the calories." In this way, users can receive personalized exercise suggestions and effectively manage their diet and exercise progress.

[0233] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0234] Step 1:

[0235] The user uses the device to enter personal data such as height, weight, lifestyle habits, and target weight. The entered data is sent from the device to the server, which receives it and stores it in a database. Specifically, when the user enters the required information into the form on the screen and presses the submit button, the device sends this information in JSON format to the server. The server then analyzes the received data and stores it in the appropriate database fields.

[0236] Input: Personal data such as height, weight, lifestyle habits, and target weight

[0237] Output: Personal data stored in a database

[0238] Step 2:

[0239] Users take photos of their daily meals on their device and upload them through the application. The device then sends the uploaded photos to the server. The server receives the photos and inputs the photo data into a generative AI model for analysis. Specifically, users press the upload button to submit the photos they have taken, and the device then sends the image file to the server. The server then runs the received image file through its analysis engine.

[0240] Input: Photo of food (image data)

[0241] Output: Parsed calorie information

[0242] Step 3:

[0243] The server uses the generative AI model to analyze the uploaded photo of the meal and calculate the calories of each food item in the meal. This calorie information is stored in a database. Specifically, after the server receives the photo of the meal, it inputs the image into the generative AI model to identify the food and predict its calories. The results of this processing are stored in a database.

[0244] Input: Food photo

[0245] Output: Calorie information for each food

[0246] Step 4:

[0247] The server then proposes a personalized exercise plan based on the user's personal data and analyzed calorie information. The exercise plan includes specific exercise types and frequency, as well as links to exercise videos. Specifically, the server obtains the user's personal data and calorie information and uses a generative AI model to generate an exercise plan. The generated exercise plan is stored in a database and sent to the device.

[0248] Input: Personal data, calorie information

[0249] Output: Proposed exercise method

[0250] Step 5:

[0251] The user exercises based on the proposed exercise plan and records the results on the device. The device then sends this exercise data to the server. The server stores the received data in a database and uses it to provide future exercise suggestions. Specifically, the user enters the exercise status within the application and sends the data from the device to the server. The server then stores the received exercise data in a database.

[0252] Input: Exercise execution data

[0253] Output: Exercise execution data stored in a database

[0254] Step 6:

[0255] Users log in to the virtual store and record their meals and receive exercise guidance. The server supports activities within the virtual store based on the user's data. Specifically, users access the virtual store through a web browser or a dedicated app, and the server provides various data (meal records, exercise guidance, etc.).

[0256] Input: Login information

[0257] Output: Meal records and exercise guidance data in the virtual store

[0258] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0259] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[0260] System Overview

[0261] user:

[0262] Users install the system's application and create an account. After creating an account, users enter personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan. Users can also record their daily emotional state through photos and text input.

[0263] Device:

[0264] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results, exercise suggestions, and emotion-based messages to the user, and allows the user to watch exercise videos.

[0265] server:

[0266] The server receives the user's data and uses a generative AI model to analyze photos of meals and calculate calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. It also uses an emotion engine to analyze the user's emotional state and generates emotion-based feedback and encouraging messages. It also records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0267] Emotion Engine:

[0268] The emotion engine recognizes the user's emotional state by analyzing data such as text, voice, and photos entered by the user, classifying the emotional state, and generating appropriate feedback and encouraging messages for the user based on this.

[0269] Program processing

[0270] 1. User data input:

[0271] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0272] 2. Upload your meal photos:

[0273] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0274] 3. Photo analysis and calorie calculation:

[0275] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0276] 4. Exercise suggestions:

[0277] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[0278] 5. Emotion Recognition with Emotion Engine:

[0279] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[0280] 6. Emotion-based feedback:

[0281] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0282] 7. Exercise execution and recording:

[0283] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[0284] 8. Data Collection and Analysis:

[0285] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[0286] Specific examples

[0287] Let's take the example of User B using this system.

[0288] Initial Settings:

[0289] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[0290] Upload your meal photos and calculate calories:

[0291] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[0292] Exercise suggestions:

[0293] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[0294] Emotion Recognition and Feedback:

[0295] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the message "Great job! Keep it up!" is displayed.

[0296] Exercise log and next suggestion:

[0297] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[0298] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[0299] The processing flow will be explained below.

[0300] Step 1:

[0301] user

[0302] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[0303] Step 2:

[0304] Terminal

[0305] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[0306] Step 3:

[0307] user

[0308] Users take photos of their meals and upload them to the application.

[0309] Step 4:

[0310] Terminal

[0311] The device sends the uploaded photos of the meal to the server.

[0312] Step 5:

[0313] server

[0314] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[0315] Step 6:

[0316] server

[0317] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[0318] Step 7:

[0319] Terminal

[0320] The terminal displays the calorie information and advice received from the server on a user interface.

[0321] Step 8:

[0322] user

[0323] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[0324] Step 9:

[0325] server

[0326] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[0327] Step 10:

[0328] Terminal

[0329] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[0330] Step 11:

[0331] user

[0332] Users record their emotional state using text, photos, and audio, and the device transmits this emotional data to a server.

[0333] Step 12:

[0334] server

[0335] The server's emotion engine analyzes the received emotion data and classifies the user's emotional state, for example into categories such as "positive" or "negative."

[0336] Step 13:

[0337] server

[0338] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0339] Step 14:

[0340] Terminal

[0341] The device displays the feedback and messages received from the server in the user interface, allowing the user to check and decide on their next action.

[0342] Step 15:

[0343] user

[0344] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[0345] Step 16:

[0346] Terminal

[0347] The terminal transmits the recorded exercise data to a server.

[0348] Step 17:

[0349] server

[0350] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[0351] Example 2

[0352] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0353] Current diet support systems only provide basic functions for users' diet management and exercise suggestions, and do not take into account the user's emotional state, making it difficult to maintain user motivation.In addition, data accumulation and analysis to improve the accuracy of the system's suggestions are insufficient, making it difficult to provide the optimal plan for each user.

[0354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0355] In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for inputting the user's emotional data and analyzing the input emotional data, means for providing feedback to the user based on the analyzed emotional data, and means for recording and analyzing the user's exercise implementation data. This makes it possible to maintain motivation taking into account the user's emotional state, and to provide an individually optimized diet plan through data accumulation and analysis.

[0356] A "user" is an individual who uses the system and inputs personal data such as height, weight, lifestyle habits, and target weight.

[0357] A "terminal" is a device that a user uses to connect to the system and perform tasks such as entering data, uploading photos, and displaying feedback.

[0358] The "server" is a computer system that receives, analyzes, and stores data sent by users, and is the central processing unit that suggests exercise methods and performs emotion analysis.

[0359] "Personal data" refers to basic information required for the system to generate a diet plan, such as the user's height, weight, lifestyle habits, and target weight.

[0360] "Meal photos" refer to image files that users take and upload to the system to record the contents of their meals.

[0361] "Calories" is a unit that indicates the amount of energy in food, and is a value calculated by the system by analyzing photos of meals.

[0362] "Exercise method" refers to the plan and type of physical activity that the server suggests based on the user's personal data and calorie information.

[0363] "Emotional Data" refers to information such as text, photos, and audio that users enter to record their emotional state.

[0364] An "emotion engine" refers to an algorithm or software component that analyzes input emotional data and classifies the user's emotional state.

[0365] "Feedback" refers to messages of encouragement or advice provided to users based on emotional data analyzed by the emotion engine.

[0366] "Exercise execution data" is data that records the details and results of the exercise performed by the user (e.g., walking distance, time, calories burned), and is sent to the server.

[0367] "Database" refers to the storage system for storing and managing personal data, meal photos, calorie information, exercise data, and emotional data collected by the System from Users.

[0368] A "generative AI model" is an artificial intelligence algorithm that performs data analysis and pattern recognition, and is used to analyze food photos, suggest exercise methods, analyze emotional data, and more.

[0369] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[0370] Hardware and Software Used

[0371] This system uses the following hardware and software:

[0372] User's device: a mobile device such as a smartphone or tablet

[0373] Server: High-performance data processing and analysis server

[0374] Generative AI models: Artificial intelligence models for food photo analysis, calorie calculation, exercise suggestions, and emotion recognition

[0375] Emotion engine: A software component that analyzes input emotion data

[0376] Processing flow

[0377] 1. User data entry

[0378] Users install the system's application and enter their personal data, which is then sent to the server, which stores it in a database.

[0379] 2. Upload a photo of your meal

[0380] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0381] 3. Analysis of food photos and calorie calculation

[0382] The server uses a generative AI model to analyze the received photos and calculate the calorie content of each meal, which is then stored in a database and provided to the user.

[0383] 4. Exercise suggestions

[0384] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[0385] 5. Emotion Recognition by Emotion Engine

[0386] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[0387] 6. Emotional Feedback

[0388] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0389] 7. Exercise execution and recording

[0390] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[0391] 8. Data accumulation and analysis

[0392] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[0393] Specific examples

[0394] Let's take the example of User B using this system.

[0395] Initial Settings:

[0396] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[0397] Upload your meal photos and calculate calories:

[0398] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[0399] Exercise suggestions:

[0400] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[0401] Emotion Recognition and Feedback:

[0402] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the server provides feedback such as, "Great job! Keep it up!"

[0403] Exercise log and next suggestion:

[0404] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[0405] Prompt Sentence Examples

[0406] Below are some examples of specific prompt sentences.

[0407] Food photo analysis prompt: "Analyze the provided food photo and calculate the calorie information. Display the name and calories of the food contained in the photo as a result."

[0408] Sentiment analysis prompt: "Classify the user's emotion from the following text. Text: 'Today I felt tired but accomplished.'"

[0409] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[0410] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0411] Step 1:

[0412] User Data Entry

[0413] The user installs the system application and enters personal data (e.g., height, weight, age, gender, lifestyle habits, target weight, etc.). The device sends this data to the server, which then stores the received data in a database.

[0414] Input: Personal data entered by the user into the application

[0415] Output: Personal data stored on the server

[0416] Step 2:

[0417] Upload a photo of your meal

[0418] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0419] Input: Food photos taken and uploaded by users

[0420] Output: Meal photos stored on the server

[0421] Step 3:

[0422] Analyze food photos and calculate calories

[0423] The server inputs the received meal photo into the generative AI model and begins its analysis. Specifically, it uses image recognition algorithms to identify the type and quantity of ingredients. It then compares the results with an existing database to calculate the calories of each ingredient. The results are stored in a database and provided to the user.

[0424] Input: Food photos stored on the server

[0425] Data processing: Image recognition and calorie calculation with generative AI models

[0426] Output: Analysis results including calorie information

[0427] Step 4:

[0428] Exercise suggestions

[0429] The server uses a generative AI model to calculate the optimal exercise method based on the user's basic data (e.g., height, weight, lifestyle, and target weight) and calorie information. Specifically, this information includes the type and frequency of exercise that corresponds to the user's lifestyle and goals, as well as links to appropriate exercise videos. This information is sent to the device and displayed to the user.

[0430] Input: Personal data, calorie information

[0431] Data Computing: Optimizing Movement Strategies with Generative AI Models

[0432] Output: Exercise suggestions

[0433] Step 5:

[0434] Entering emotion data

[0435] Users record their emotional state through text, photos, and audio, and the device transmits this emotional data to a server.

[0436] Input: Emotion data entered by the user (text, photo, voice)

[0437] Output: Emotion data stored on the server

[0438] Step 6:

[0439] Emotion analysis using an emotion engine

[0440] The server's emotion engine analyzes the received emotion data. Specifically, it analyzes text data using natural language processing (NLP), performs tone analysis on audio data, and performs facial expression recognition on photo data. This allows it to classify the user's emotional state as "positive," "negative," or "neutral."

[0441] Input: Emotion data

[0442] Data processing: natural language processing, voice tone analysis, facial expression recognition

[0443] Output: Classified emotional state

[0444] Step 7:

[0445] Emotion-based feedback

[0446] The server generates appropriate feedback and encouraging messages for the user based on the emotional state analyzed by the emotion engine. For example, if the emotion is classified as "negative," an encouraging message is generated. The generated message is sent to the device and displayed to the user.

[0447] Input: Classified emotional state

[0448] Data processing: feedback message generation

[0449] Output: Feedback message

[0450] Step 8:

[0451] Exercise execution and recording

[0452] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent from the device to the server and stored in a database.

[0453] Input: User recorded exercise results

[0454] Output: Exercise execution data stored on the server

[0455] Step 9:

[0456] Accumulating data and making proposals for the next time

[0457] The server analyzes the accumulated user data to improve the accuracy of the generated AI model. For example, it uses past exercise results and diet data to optimize the next exercise method. The analysis results are reflected in the diet plan suggestions for the next time and are sent to the device.

[0458] Input: Accumulated user data

[0459] Data Computation: Analysis and Accuracy Improvement with Generative AI Models

[0460] Output: Optimized next exercise suggestions

[0461] (Application example 2)

[0462] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0463] In recent years, with the rise in health awareness, there is a demand for systems that allow individuals to manage their own health status and diet effectively. However, many existing diet support systems do not take into account the user's emotional state, which can lead to users losing motivation midway through the process and making it difficult to continue using the system. For this reason, there is a demand for a system that can analyze the user's emotional data and provide appropriate feedback and exercise suggestions based on that data to improve the user's motivation and provide effective diet support.

[0464] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting an exercise method based on the personal data and the calorie information, means for inputting emotional data, means for analyzing the input emotional data and classifying the emotional state, means for providing the suggested exercise method to the user, means for generating a feedback message based on the emotional state, means for providing the feedback message to the user, and means for recording and analyzing the user's exercise implementation data. This makes it possible to provide feedback and exercise suggestions based on the user's emotional state.

[0465] A "user" is an individual who uses the system and is the entity that inputs personal data and provides food photos and emotional data.

[0466] "Personal Data" refers to information about a user's health and lifestyle, such as height, weight, lifestyle habits, and target weight, entered by the user.

[0467] "Meal photos" are image data that users take and upload of their daily meals, and are used for calorie calculation and nutritional analysis.

[0468] "Calorie information" refers to the calorie value of each meal obtained by analyzing the photograph of the meal.

[0469] "Exercise method" refers to specific exercise menus and exercises suggested to users based on personal data and calorie information.

[0470] "Emotional data" refers to data that indicates the user's emotional state and is entered into the system, and is recorded in the form of text, photos, audio, etc.

[0471] "Emotional state" refers to the user's current mental state and mood, which can be obtained by analyzing emotional data.

[0472] "Feedback messages" are messages such as encouragement or suggestions that the emotion engine generates based on the user's emotional state.

[0473] A "user interface" refers to the screen or operating means that allows a user to interact with a system and input and output data.

[0474] A "generative model" is an AI model that analyzes data collected from users to perform tasks such as calorie calculations, emotional classification, and exercise suggestions.

[0475] The system that realizes this application example operates in cooperation with the elements of the user terminal, server, and emotion engine.

[0476] 1. User Device

[0477] The user device refers to a smartphone, and applications can be developed using Flutter. Through the application, users enter personal data (height, weight, lifestyle habits, target weight, etc.) and upload photos of meals and emotional data. This data entered by the user is sent to a server via the Internet.

[0478] 2. Server

[0479] The server is responsible for several functions, using TensorFlow and Firebase for these processing. The main functions of the server are:

[0480] 1. Storage and management of personal data

[0481] The personal data entered by users is stored in a real-time Firebase database, which is used for calorie calculations, exercise suggestions, and sentiment analysis.

[0482] 2. Analysis of food photos and calorie calculation

[0483] The server receives photos of meals uploaded by users and performs image analysis using TensorFlow, calculating the calorie value of each meal as a result of the analysis and storing it in a database.

[0484] 3. Emotion Data Analysis

[0485] Emotional data input from users (text, photos, voice, etc.) is analyzed using TensorFlow to classify the emotional state, and the analysis results are passed to the emotion engine, which generates appropriate feedback messages.

[0486] 4. Exercise suggestions

[0487] The server generates an optimal exercise plan based on the user's personal data and calorie information, including personalized exercise menus and links to exercise videos.

[0488] 5. Generating and Displaying Feedback Messages

[0489] The server generates a feedback message based on the analyzed emotional state using the emotion engine, and the generated message is provided to the user through a user interface.

[0490] 6. Accumulating data and continuously improving the model

[0491] The server accumulates data collected from users and continuously refines the generative AI model, which improves the accuracy of future dietary analysis and exercise recommendations.

[0492] 3. Emotion Engine

[0493] The emotion engine is a system that recognizes and analyzes the user's emotional state. It uses text data and photo data entered by the user to classify the emotional state and reflects the analysis results in a feedback message.

[0494] Adding specific examples

[0495] Here are some examples of specific prompts:

[0496] Initialization prompt:

[0497] User B, height 160cm, weight 60kg, age 28, target weight 55kg. His lifestyle is mostly desk work and his hobby is jogging. Enter the initial setup information.

[0498] Dietary analysis prompt:

[0499] Analyze a photo of lunch and calculate the calories. The menu is salad and soup.

[0500] Sentiment Analysis Prompt:

[0501] A user has entered the following sentence: Today I felt tired but accomplished. Analyze this and classify the emotional state.

[0502] This makes it possible to build a system that can provide feedback and suggest exercises based on the user's emotional state.

[0503] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0504] Step 1: Entering User Data

[0505] Users enter personal data such as height, weight, lifestyle habits, and target weight into the application form. This personal data is sent from the user's device via the Internet to the server, which then stores the received data in the Firebase real-time database.

[0506] Input: Personal data (height, weight, lifestyle habits, target weight, etc.)

[0507] Output: Personal data stored in a database

[0508] Step 2: Upload a photo of your meal

[0509] Users take photos of their daily meals and upload them to the application, which then sends the photo data from the user's device to a server.

[0510] Input: Food photo data

[0511] Output: Photo data sent to the server

[0512] Step 3: Analyze photos and calculate calories

[0513] The server uses TensorFlow to analyze the uploaded photos of meals and calculate the calorie content of each meal. The resulting calorie information is stored in a database and provided to the user through a user interface.

[0514] Input: Food photo data

[0515] Output: Parsed calorie information

[0516] Step 4: Exercise suggestions

[0517] Based on the user's personal data and calorie information, the server will suggest the best way to exercise, including type, frequency, or links to specific exercise videos. The suggestions will be stored in a database and provided to the user.

[0518] Input: Personal data, calorie information

[0519] Output: Exercise suggestions

[0520] Step 5: Enter emotion data

[0521] Users record their emotional state after exercise or daily using text, photos, or audio, and this emotional data is sent from the user's device to a server.

[0522] Input: Emotion data (text, photo, voice)

[0523] Output: Emotion data sent to the server

[0524] Step 6: Sentiment Analysis

[0525] The server uses an emotion engine to analyze the transmitted emotion data and classify the emotion state, which is then stored in a database.

[0526] Input: Emotion data

[0527] Output: Classified emotional state

[0528] Step 7: Generate feedback messages

[0529] The server generates an appropriate feedback message using the generative AI model based on the classified emotional state, and the generated feedback message is provided to the user through a user interface.

[0530] Input: Classified emotional state

[0531] Output: Feedback message

[0532] Step 8: Recording and analyzing user exercise data

[0533] The user performs the suggested exercise and records the results in the application. The exercise data is sent from the user's device to the server and stored in a database. The server analyzes this data and reflects it in future exercise suggestions.

[0534] Input: Exercise execution data

[0535] Output: Analysis data, improved exercise suggestions

[0536] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0537] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0538] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0539] [Second embodiment]

[0540] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0541] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0542] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0543] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0544] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0545] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0546] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0547] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0548] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0549] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0550] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0551] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0552] This invention is a diet support system that manages diet and suggests exercise based on personal data and food photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation.

[0553] System Overview

[0554] user:

[0555] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[0556] Device:

[0557] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results and exercise suggestions from the server to the user, allowing the user to watch exercise videos.

[0558] server:

[0559] The server receives the user's data, analyzes photos of meals using a generative AI model, and calculates calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. In addition, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0560] Program processing

[0561] 1. User data input:

[0562] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0563] 2. Upload your meal photos:

[0564] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[0565] 3. Photo analysis and calorie calculation:

[0566] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0567] 4. Exercise suggestions:

[0568] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[0569] 5. Exercise execution and recording:

[0570] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[0571] 6. Data Collection and Analysis:

[0572] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[0573] Specific examples

[0574] Let's take the example of User A using this system.

[0575] Initial Settings:

[0576] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[0577] Upload your meal photos and calculate calories:

[0578] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[0579] Exercise suggestions:

[0580] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[0581] Exercise log and next suggestion:

[0582] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[0583] As described above, the present invention is a system that effectively supports users in their healthy dieting by using a generative AI model to provide dietary management and exercise suggestions.

[0584] The processing flow will be explained below.

[0585] Step 1:

[0586] user

[0587] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[0588] Step 2:

[0589] Terminal

[0590] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[0591] Step 3:

[0592] user

[0593] Users take photos of their meals and upload them to the application.

[0594] Step 4:

[0595] Terminal

[0596] The device sends the uploaded photos of the meal to the server.

[0597] Step 5:

[0598] server

[0599] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[0600] Step 6:

[0601] server

[0602] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[0603] Step 7:

[0604] Terminal

[0605] The terminal displays the calorie information and advice received from the server on a user interface.

[0606] Step 8:

[0607] user

[0608] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[0609] Step 9:

[0610] server

[0611] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[0612] Step 10:

[0613] Terminal

[0614] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[0615] Step 11:

[0616] user

[0617] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[0618] Step 12:

[0619] Terminal

[0620] The terminal transmits the recorded exercise data to a server.

[0621] Step 13:

[0622] server

[0623] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[0624] Example 1

[0625] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0626] While conventional diet support systems use data and food photos entered by users to calculate calories and suggest exercises, they suffer from insufficient accuracy and individuality. Furthermore, they lack a means to effectively provide feedback based on the analysis of the data obtained and improve the overall performance of the system, making it difficult to provide optimal support to users.

[0627] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0628] In this invention, the server includes means for the user to input personal data such as height, weight, age, sex, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories using a generative AI model, means for suggesting an exercise method based on the personal data and the calorie information, means for providing the suggested exercise method to the user, means for recording the user's exercise implementation data and transmitting it to the server, and means for continuously improving the generative AI model based on the recorded data. This enables highly accurate diet support tailored to the individual user and continuous improvement of system performance.

[0629] A "user" is a user of the system who is responsible for inputting personal data such as their height, weight, age, gender, lifestyle habits, and target weight, and uploading photos of their meals.

[0630] A "terminal" is a device used by a user that provides an interface for inputting, reviewing, and sending personal data and photos of meals.

[0631] The "server" is a central system that receives and analyzes data sent by users and uses a generative AI model to calculate calories and make exercise suggestions.

[0632] "Personal data" refers to information such as height, weight, age, gender, lifestyle habits, and target weight that users enter into the system.

[0633] A "generative AI model" is an algorithm or program that uses machine learning technology to analyze data and calculate calories and suggest exercises.

[0634] "Calorie Calculation" is a method of analyzing photos of meals uploaded by users and calculating calories burned based on the type and quantity of food.

[0635] "Exercise Suggestion" is the process of providing instructions and advice on the optimal type and frequency of exercise based on the user's personal data and calorie information.

[0636] A "database" is a storage device that the server uses to store and manage users' personal data, analysis results, exercise execution data, etc.

[0637] "Continuous improvement" is the process of training the generative AI model based on accumulated data to improve the accuracy of analysis and proposals.

[0638] The present invention is a diet support system that manages dietary management and suggests exercise based on personal data and meal photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation. The detailed configuration is described below.

[0639] System Overview

[0640] user:

[0641] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[0642] Device:

[0643] The device is a smartphone, tablet, or other device that transmits data entered by the user and uploaded meal photos to the server through a user interface. The device also displays analysis results and exercise suggestions from the server to the user, and allows the user to watch exercise videos.

[0644] server:

[0645] The server uses cloud-based computing resources to receive user data, analyzes food photos using a generative AI model, calculates calories, and generates optimal exercise and diet plans for the user based on the analyzed data. Additionally, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0646] Program processing

[0647] User data input:

[0648] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0649] Upload your meal photo:

[0650] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[0651] Photo analysis and calorie counting:

[0652] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0653] Exercise suggestions:

[0654] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[0655] Exercise execution and recording:

[0656] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[0657] Data collection and analysis:

[0658] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[0659] Specific examples

[0660] Let's take the example of User A using this system.

[0661] Initial Settings:

[0662] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[0663] Upload your meal photos and calculate calories:

[0664] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[0665] Exercise suggestions:

[0666] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[0667] Exercise log and next suggestion:

[0668] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[0669] Example prompt sentence:

[0670] "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is recommended."

[0671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] User Data Entry

[0674] The user launches the application and enters personal data (height, weight, age, gender, lifestyle habits, and target weight).

[0675] The device receives this entered personal data and checks that the data is in the correct format (e.g., height and weight are numeric).

[0676] The terminal transmits the checked data to the server.

[0677] The server stores the received personal data in a database.

[0678] Input: Personal data (height, weight, age, gender, lifestyle habits, goal weight)

[0679] Output: Personal data stored in a database

[0680] Step 2:

[0681] Upload a photo of your meal

[0682] Users take photos of their daily meals and upload them through the application.

[0683] The device receives the photo and checks the resolution and format.

[0684] The device sends the confirmed photo to the server.

[0685] The server receives the photo data and stores it in a database.

[0686] Input: Food photo

[0687] Output: Photo data stored in a database

[0688] Step 3:

[0689] Photo analysis and calorie calculation

[0690] The server retrieves the photo data from the database and begins analyzing it using the generative AI model.

[0691] A generative AI model (e.g., using TensorFlow or PyTorch) is fed photos of food to identify the type and quantity of food.

[0692] The server calculates the calories for each identified food item.

[0693] The server stores the calculation results (calories of each food item and total calories) in a database and sends the results to the terminal.

[0694] The terminal receives the analysis results from the server and notifies the user.

[0695] Input: Food photo data

[0696] Output: Calorie calculation result (notification to user)

[0697] Step 4:

[0698] Exercise suggestions

[0699] The server generates the optimal exercise method based on the user's basic data (personal data) and calorie information.

[0700] A prompt sentence is input into the generative AI model to generate exercise suggestions appropriate for the user (e.g., "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is appropriate").

[0701] The server stores the generated exercise suggestions in a database and transmits them to the terminal.

[0702] The device displays the received exercise suggestions to the user and, in some cases, provides links to exercise videos.

[0703] Input: Basic data, calorie information

[0704] Output: Exercise suggestions and exercise video links (notification to user)

[0705] Step 5:

[0706] Exercise execution and recording

[0707] The user performs the suggested exercise (e.g., a 30-minute walk) and records the results in the application.

[0708] The device receives the exercise record (type of exercise performed, time, intensity, etc.) and sends it to the server.

[0709] The server stores the received exercise data in a database and reflects it in the next proposal.

[0710] Input: Exercise record data

[0711] Output: Exercise data saved in the database (reflected in the next proposal)

[0712] Step 6:

[0713] Data accumulation and analysis

[0714] The server periodically re-analyzes the accumulated user data (personal data, calorie information, exercise data).

[0715] Using a generative AI model, the model is trained based on new data, improving analysis accuracy.

[0716] The server uses the improved generative AI model to further customize your next exercise suggestion.

[0717] Input: Accumulated user data

[0718] Output: An improved generative AI model and motion suggestions based on it

[0719] (Application example 1)

[0720] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0721] While existing diet support systems allow users to input personal data, upload photos of their meals, and calculate calories, they lack specificity and individuality in their exercise recommendations. They also offer limited means for effectively managing diet and exercise progress. Furthermore, technological solutions are needed to enhance convenience in online and virtual environments.

[0722] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0723] In this invention, the server includes means for a user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for providing the suggested exercise methods to the user and for viewing exercise videos, means for recording and analyzing the user's exercise implementation data, and means for the user to log in to a virtual store to record meals and receive exercise guidance. This allows the user to receive personalized exercise suggestions in a virtual environment, enabling effective management of diet and exercise progress.

[0724] "Personal data" refers to data entered by the user, such as height, weight, lifestyle habits, and target weight.

[0725] "Meal photos" refer to image data taken by users of their own meals.

[0726] "Calorie counting" refers to the process of analyzing a photo of a meal to calculate the total calories in that meal.

[0727] "Exercise Method" refers to a specific exercise plan suggested based on the user's personal data and calorie information.

[0728] "Exercise videos" refer to video content that allows users to exercise while watching.

[0729] A "virtual store" refers to a virtual store environment that users can access online to record their meals and receive exercise guidance.

[0730] "Exercise data" refers to data such as the type of exercise the user actually performed, the duration, and the calories burned.

[0731] "User interface" refers to the display and input devices that allow a user to interact with a system.

[0732] "Generative AI model" refers to an artificial intelligence model that analyzes data collected from users and generates personalized suggestions.

[0733] The present invention is a diet support system that includes input of personal data, analysis of food photos, calorie calculation, exercise method suggestions, exercise recording and analysis, and use in a virtual store.

[0734] The server receives personal data entered by the user, such as height, weight, lifestyle habits, and target weight, and stores the data in a database. The user enters this personal data through a device such as a smartphone or PC. The device then transmits the personal data entered via the user interface to the server.

[0735] Next, users take photos of their daily meals and upload them from their devices. The server analyzes the uploaded photos using a generative AI model to calculate the calories of each meal. The analyzed calorie information is stored in a database and provided to the user. This information can be viewed through the user interface.

[0736] The server then proposes a personalized exercise plan based on the user's personal data and calorie information obtained from their diet. This includes specific exercise types and frequency, as well as links to exercise videos. The user then performs the exercises based on the proposed plan and records the results on their device. The recorded exercise data is then sent back to the server and stored in a database.

[0737] Additionally, users can log in to the virtual store to record their meals and receive exercise guidance, where they can track their progress and interact with other users for added motivation.

[0738] As a concrete example, let's consider the case of User A using this system. User A enters his / her height of 170 cm, weight of 75 kg, lifestyle habits (desk work), and target weight of 65 kg, and uploads photos of his / her daily meals. For example, if he / she uploads a photo of sushi taken for lunch, the server analyzes it and calculates the calories. Next, the system suggests that he / she walk 30 minutes every day this week and watch yoga videos three times a week. When User A records the results of his / her exercise, the results are reflected in the next exercise suggestions.

[0739] An example of a prompt sentence is, "Please analyze the photo of sushi taken by user A and calculate the calories." In this way, users can receive personalized exercise suggestions and effectively manage their diet and exercise progress.

[0740] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0741] Step 1:

[0742] The user uses the device to enter personal data such as height, weight, lifestyle habits, and target weight. The entered data is sent from the device to the server, which receives it and stores it in a database. Specifically, when the user enters the required information into the form on the screen and presses the submit button, the device sends this information in JSON format to the server. The server then analyzes the received data and stores it in the appropriate database fields.

[0743] Input: Personal data such as height, weight, lifestyle habits, and target weight

[0744] Output: Personal data stored in a database

[0745] Step 2:

[0746] Users take photos of their daily meals on their device and upload them through the application. The device then sends the uploaded photos to the server. The server receives the photos and inputs the photo data into a generative AI model for analysis. Specifically, users press the upload button to submit the photos they have taken, and the device then sends the image file to the server. The server then runs the received image file through its analysis engine.

[0747] Input: Photo of food (image data)

[0748] Output: Parsed calorie information

[0749] Step 3:

[0750] The server uses the generative AI model to analyze the uploaded photo of the meal and calculate the calories of each food item in the meal. This calorie information is stored in a database. Specifically, after the server receives the photo of the meal, it inputs the image into the generative AI model to identify the food and predict its calories. The results of this processing are stored in a database.

[0751] Input: Food photo

[0752] Output: Calorie information for each food

[0753] Step 4:

[0754] The server then proposes a personalized exercise plan based on the user's personal data and analyzed calorie information. The exercise plan includes specific exercise types and frequency, as well as links to exercise videos. Specifically, the server obtains the user's personal data and calorie information and uses a generative AI model to generate an exercise plan. The generated exercise plan is stored in a database and sent to the device.

[0755] Input: Personal data, calorie information

[0756] Output: Proposed exercise method

[0757] Step 5:

[0758] The user exercises based on the proposed exercise plan and records the results on the device. The device then sends this exercise data to the server. The server stores the received data in a database and uses it to provide future exercise suggestions. Specifically, the user enters the exercise status within the application and sends the data from the device to the server. The server then stores the received exercise data in a database.

[0759] Input: Exercise execution data

[0760] Output: Exercise execution data stored in a database

[0761] Step 6:

[0762] Users log in to the virtual store and record their meals and receive exercise guidance. The server supports activities within the virtual store based on the user's data. Specifically, users access the virtual store through a web browser or a dedicated app, and the server provides various data (meal records, exercise guidance, etc.).

[0763] Input: Login information

[0764] Output: Meal records and exercise guidance data in the virtual store

[0765] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0766] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[0767] System Overview

[0768] user:

[0769] Users install the system's application and create an account. After creating an account, users enter personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan. Users can also record their daily emotional state through photos and text input.

[0770] Device:

[0771] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results, exercise suggestions, and emotion-based messages to the user, and allows the user to watch exercise videos.

[0772] server:

[0773] The server receives the user's data and uses a generative AI model to analyze photos of meals and calculate calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. It also uses an emotion engine to analyze the user's emotional state and generates emotion-based feedback and encouraging messages. It also records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[0774] Emotion Engine:

[0775] The emotion engine recognizes the user's emotional state by analyzing data such as text, voice, and photos entered by the user, classifying the emotional state, and generating appropriate feedback and encouraging messages for the user based on this.

[0776] Program processing

[0777] 1. User data input:

[0778] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[0779] 2. Upload your meal photos:

[0780] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0781] 3. Photo analysis and calorie calculation:

[0782] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[0783] 4. Exercise suggestions:

[0784] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[0785] 5. Emotion Recognition with Emotion Engine:

[0786] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[0787] 6. Emotion-based feedback:

[0788] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0789] 7. Exercise execution and recording:

[0790] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[0791] 8. Data Collection and Analysis:

[0792] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[0793] Specific examples

[0794] Let's take the example of User B using this system.

[0795] Initial Settings:

[0796] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[0797] Upload your meal photos and calculate calories:

[0798] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[0799] Exercise suggestions:

[0800] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[0801] Emotion Recognition and Feedback:

[0802] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the message "Great job! Keep it up!" is displayed.

[0803] Exercise log and next suggestion:

[0804] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[0805] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[0806] The processing flow will be explained below.

[0807] Step 1:

[0808] user

[0809] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[0810] Step 2:

[0811] Terminal

[0812] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[0813] Step 3:

[0814] user

[0815] Users take photos of their meals and upload them to the application.

[0816] Step 4:

[0817] Terminal

[0818] The device sends the uploaded photos of the meal to the server.

[0819] Step 5:

[0820] server

[0821] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[0822] Step 6:

[0823] server

[0824] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[0825] Step 7:

[0826] Terminal

[0827] The terminal displays the calorie information and advice received from the server on a user interface.

[0828] Step 8:

[0829] user

[0830] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[0831] Step 9:

[0832] server

[0833] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[0834] Step 10:

[0835] Terminal

[0836] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[0837] Step 11:

[0838] user

[0839] Users record their emotional state using text, photos, and audio, and the device transmits this emotional data to a server.

[0840] Step 12:

[0841] server

[0842] The server's emotion engine analyzes the received emotion data and classifies the user's emotional state, for example into categories such as "positive" or "negative."

[0843] Step 13:

[0844] server

[0845] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0846] Step 14:

[0847] Terminal

[0848] The device displays the feedback and messages received from the server in the user interface, allowing the user to check and decide on their next action.

[0849] Step 15:

[0850] user

[0851] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[0852] Step 16:

[0853] Terminal

[0854] The terminal transmits the recorded exercise data to a server.

[0855] Step 17:

[0856] server

[0857] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[0858] Example 2

[0859] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0860] Current diet support systems only provide basic functions for users' diet management and exercise suggestions, and do not take into account the user's emotional state, making it difficult to maintain user motivation.In addition, data accumulation and analysis to improve the accuracy of the system's suggestions are insufficient, making it difficult to provide the optimal plan for each user.

[0861] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0862] In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for inputting the user's emotional data and analyzing the input emotional data, means for providing feedback to the user based on the analyzed emotional data, and means for recording and analyzing the user's exercise implementation data. This makes it possible to maintain motivation taking into account the user's emotional state, and to provide an individually optimized diet plan through data accumulation and analysis.

[0863] A "user" is an individual who uses the system and inputs personal data such as height, weight, lifestyle habits, and target weight.

[0864] A "terminal" is a device that a user uses to connect to the system and perform tasks such as entering data, uploading photos, and displaying feedback.

[0865] The "server" is a computer system that receives, analyzes, and stores data sent by users, and is the central processing unit that suggests exercise methods and performs emotion analysis.

[0866] "Personal data" refers to basic information required for the system to generate a diet plan, such as the user's height, weight, lifestyle habits, and target weight.

[0867] "Meal photos" refer to image files that users take and upload to the system to record the contents of their meals.

[0868] "Calories" is a unit that indicates the amount of energy in food, and is a value calculated by the system by analyzing photos of meals.

[0869] "Exercise method" refers to the plan and type of physical activity that the server suggests based on the user's personal data and calorie information.

[0870] "Emotional Data" refers to information such as text, photos, and audio that users enter to record their emotional state.

[0871] An "emotion engine" refers to an algorithm or software component that analyzes input emotional data and classifies the user's emotional state.

[0872] "Feedback" refers to messages of encouragement or advice provided to users based on emotional data analyzed by the emotion engine.

[0873] "Exercise execution data" is data that records the details and results of the exercise performed by the user (e.g., walking distance, time, calories burned), and is sent to the server.

[0874] "Database" refers to the storage system for storing and managing personal data, meal photos, calorie information, exercise data, and emotional data collected by the System from Users.

[0875] A "generative AI model" is an artificial intelligence algorithm that performs data analysis and pattern recognition, and is used to analyze food photos, suggest exercise methods, analyze emotional data, and more.

[0876] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[0877] Hardware and Software Used

[0878] This system uses the following hardware and software:

[0879] User's device: a mobile device such as a smartphone or tablet

[0880] Server: High-performance data processing and analysis server

[0881] Generative AI models: Artificial intelligence models for food photo analysis, calorie calculation, exercise suggestions, and emotion recognition

[0882] Emotion engine: A software component that analyzes input emotion data

[0883] Processing flow

[0884] 1. User data entry

[0885] Users install the system's application and enter their personal data, which is then sent to the server, which stores it in a database.

[0886] 2. Upload a photo of your meal

[0887] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0888] 3. Analysis of food photos and calorie calculation

[0889] The server uses a generative AI model to analyze the received photos and calculate the calorie content of each meal, which is then stored in a database and provided to the user.

[0890] 4. Exercise suggestions

[0891] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[0892] 5. Emotion Recognition by Emotion Engine

[0893] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[0894] 6. Emotional Feedback

[0895] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[0896] 7. Exercise execution and recording

[0897] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[0898] 8. Data accumulation and analysis

[0899] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[0900] Specific examples

[0901] Let's take the example of User B using this system.

[0902] Initial Settings:

[0903] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[0904] Upload your meal photos and calculate calories:

[0905] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[0906] Exercise suggestions:

[0907] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[0908] Emotion Recognition and Feedback:

[0909] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the server provides feedback such as, "Great job! Keep it up!"

[0910] Exercise log and next suggestion:

[0911] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[0912] Prompt Sentence Examples

[0913] Below are some examples of specific prompt sentences.

[0914] Food photo analysis prompt: "Analyze the provided food photo and calculate the calorie information. Display the name and calories of the food contained in the photo as a result."

[0915] Sentiment analysis prompt: "Classify the user's emotion from the following text. Text: 'Today I felt tired but accomplished.'"

[0916] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[0917] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0918] Step 1:

[0919] User Data Entry

[0920] The user installs the system application and enters personal data (e.g., height, weight, age, gender, lifestyle habits, target weight, etc.). The device sends this data to the server, which then stores the received data in a database.

[0921] Input: Personal data entered by the user into the application

[0922] Output: Personal data stored on the server

[0923] Step 2:

[0924] Upload a photo of your meal

[0925] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[0926] Input: Food photos taken and uploaded by users

[0927] Output: Meal photos stored on the server

[0928] Step 3:

[0929] Analyze food photos and calculate calories

[0930] The server inputs the received meal photo into the generative AI model and begins its analysis. Specifically, it uses image recognition algorithms to identify the type and quantity of ingredients. It then compares the results with an existing database to calculate the calories of each ingredient. The results are stored in a database and provided to the user.

[0931] Input: Food photos stored on the server

[0932] Data processing: Image recognition and calorie calculation with generative AI models

[0933] Output: Analysis results including calorie information

[0934] Step 4:

[0935] Exercise suggestions

[0936] The server uses a generative AI model to calculate the optimal exercise method based on the user's basic data (e.g., height, weight, lifestyle, and target weight) and calorie information. Specifically, this information includes the type and frequency of exercise that corresponds to the user's lifestyle and goals, as well as links to appropriate exercise videos. This information is sent to the device and displayed to the user.

[0937] Input: Personal data, calorie information

[0938] Data Computing: Optimizing Movement Strategies with Generative AI Models

[0939] Output: Exercise suggestions

[0940] Step 5:

[0941] Entering emotion data

[0942] Users record their emotional state through text, photos, and audio, and the device transmits this emotional data to a server.

[0943] Input: Emotion data entered by the user (text, photo, voice)

[0944] Output: Emotion data stored on the server

[0945] Step 6:

[0946] Emotion analysis using an emotion engine

[0947] The server's emotion engine analyzes the received emotion data. Specifically, it analyzes text data using natural language processing (NLP), performs tone analysis on audio data, and performs facial expression recognition on photo data. This allows it to classify the user's emotional state as "positive," "negative," or "neutral."

[0948] Input: Emotion data

[0949] Data processing: natural language processing, voice tone analysis, facial expression recognition

[0950] Output: Classified emotional state

[0951] Step 7:

[0952] Emotion-based feedback

[0953] The server generates appropriate feedback and encouraging messages for the user based on the emotional state analyzed by the emotion engine. For example, if the emotion is classified as "negative," an encouraging message is generated. The generated message is sent to the device and displayed to the user.

[0954] Input: Classified emotional state

[0955] Data processing: feedback message generation

[0956] Output: Feedback message

[0957] Step 8:

[0958] Exercise execution and recording

[0959] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent from the device to the server and stored in a database.

[0960] Input: User recorded exercise results

[0961] Output: Exercise execution data stored on the server

[0962] Step 9:

[0963] Accumulating data and making proposals for the next time

[0964] The server analyzes the accumulated user data to improve the accuracy of the generated AI model. For example, it uses past exercise results and diet data to optimize the next exercise method. The analysis results are reflected in the diet plan suggestions for the next time and are sent to the device.

[0965] Input: Accumulated user data

[0966] Data Computation: Analysis and Accuracy Improvement with Generative AI Models

[0967] Output: Optimized next exercise suggestions

[0968] (Application example 2)

[0969] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0970] In recent years, with the rise in health awareness, there is a demand for systems that allow individuals to manage their own health status and diet effectively. However, many existing diet support systems do not take into account the user's emotional state, which can lead to users losing motivation midway through the process and making it difficult to continue using the system. For this reason, there is a demand for a system that can analyze the user's emotional data and provide appropriate feedback and exercise suggestions based on that data to improve the user's motivation and provide effective diet support.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting an exercise method based on the personal data and the calorie information, means for inputting emotional data, means for analyzing the input emotional data and classifying the emotional state, means for providing the suggested exercise method to the user, means for generating a feedback message based on the emotional state, means for providing the feedback message to the user, and means for recording and analyzing the user's exercise implementation data. This makes it possible to provide feedback and exercise suggestions based on the user's emotional state.

[0972] A "user" is an individual who uses the system and is the entity that inputs personal data and provides food photos and emotional data.

[0973] "Personal Data" refers to information about a user's health and lifestyle, such as height, weight, lifestyle habits, and target weight, entered by the user.

[0974] "Meal photos" are image data that users take and upload of their daily meals, and are used for calorie calculation and nutritional analysis.

[0975] "Calorie information" refers to the calorie value of each meal obtained by analyzing the photograph of the meal.

[0976] "Exercise method" refers to specific exercise menus and exercises suggested to users based on personal data and calorie information.

[0977] "Emotional data" refers to data that indicates the user's emotional state and is entered into the system, and is recorded in the form of text, photos, audio, etc.

[0978] "Emotional state" refers to the user's current mental state and mood, which can be obtained by analyzing emotional data.

[0979] "Feedback messages" are messages such as encouragement or suggestions that the emotion engine generates based on the user's emotional state.

[0980] A "user interface" refers to the screen or operating means that allows a user to interact with a system and input and output data.

[0981] A "generative model" is an AI model that analyzes data collected from users to perform tasks such as calorie calculations, emotional classification, and exercise suggestions.

[0982] The system that realizes this application example operates in cooperation with the elements of the user terminal, server, and emotion engine.

[0983] 1. User Device

[0984] The user device refers to a smartphone, and applications can be developed using Flutter. Through the application, users enter personal data (height, weight, lifestyle habits, target weight, etc.) and upload photos of meals and emotional data. This data entered by the user is sent to a server via the Internet.

[0985] 2. Server

[0986] The server is responsible for several functions, using TensorFlow and Firebase for these processing. The main functions of the server are:

[0987] 1. Storage and management of personal data

[0988] The personal data entered by users is stored in a real-time Firebase database, which is used for calorie calculations, exercise suggestions, and sentiment analysis.

[0989] 2. Analysis of food photos and calorie calculation

[0990] The server receives photos of meals uploaded by users and performs image analysis using TensorFlow, calculating the calorie value of each meal as a result of the analysis and storing it in a database.

[0991] 3. Emotion Data Analysis

[0992] Emotional data input from users (text, photos, voice, etc.) is analyzed using TensorFlow to classify the emotional state, and the analysis results are passed to the emotion engine, which generates appropriate feedback messages.

[0993] 4. Exercise suggestions

[0994] The server generates an optimal exercise plan based on the user's personal data and calorie information, including personalized exercise menus and links to exercise videos.

[0995] 5. Generating and Displaying Feedback Messages

[0996] The server generates a feedback message based on the analyzed emotional state using the emotion engine, and the generated message is provided to the user through a user interface.

[0997] 6. Accumulating data and continuously improving the model

[0998] The server accumulates data collected from users and continuously refines the generative AI model, which improves the accuracy of future dietary analysis and exercise recommendations.

[0999] 3. Emotion Engine

[1000] The emotion engine is a system that recognizes and analyzes the user's emotional state. It uses text data and photo data entered by the user to classify the emotional state and reflects the analysis results in a feedback message.

[1001] Adding specific examples

[1002] Here are some examples of specific prompts:

[1003] Initialization prompt:

[1004] User B, height 160cm, weight 60kg, age 28, target weight 55kg. His lifestyle is mostly desk work and his hobby is jogging. Enter the initial setup information.

[1005] Dietary analysis prompt:

[1006] Analyze a photo of lunch and calculate the calories. The menu is salad and soup.

[1007] Sentiment Analysis Prompt:

[1008] A user has entered the following sentence: Today I felt tired but accomplished. Analyze this and classify the emotional state.

[1009] This makes it possible to build a system that can provide feedback and suggest exercises based on the user's emotional state.

[1010] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1011] Step 1: Entering User Data

[1012] Users enter personal data such as height, weight, lifestyle habits, and target weight into the application form. This personal data is sent from the user's device via the Internet to the server, which then stores the received data in the Firebase real-time database.

[1013] Input: Personal data (height, weight, lifestyle habits, target weight, etc.)

[1014] Output: Personal data stored in a database

[1015] Step 2: Upload a photo of your meal

[1016] Users take photos of their daily meals and upload them to the application, which then sends the photo data from the user's device to a server.

[1017] Input: Food photo data

[1018] Output: Photo data sent to the server

[1019] Step 3: Analyze photos and calculate calories

[1020] The server uses TensorFlow to analyze the uploaded photos of meals and calculate the calorie content of each meal. The resulting calorie information is stored in a database and provided to the user through a user interface.

[1021] Input: Food photo data

[1022] Output: Parsed calorie information

[1023] Step 4: Exercise suggestions

[1024] Based on the user's personal data and calorie information, the server will suggest the best way to exercise, including type, frequency, or links to specific exercise videos. The suggestions will be stored in a database and provided to the user.

[1025] Input: Personal data, calorie information

[1026] Output: Exercise suggestions

[1027] Step 5: Enter emotion data

[1028] Users record their emotional state after exercise or daily using text, photos, or audio, and this emotional data is sent from the user's device to a server.

[1029] Input: Emotion data (text, photo, voice)

[1030] Output: Emotion data sent to the server

[1031] Step 6: Sentiment Analysis

[1032] The server uses an emotion engine to analyze the transmitted emotion data and classify the emotion state, which is then stored in a database.

[1033] Input: Emotion data

[1034] Output: Classified emotional state

[1035] Step 7: Generate feedback messages

[1036] The server generates an appropriate feedback message using the generative AI model based on the classified emotional state, and the generated feedback message is provided to the user through a user interface.

[1037] Input: Classified emotional state

[1038] Output: Feedback message

[1039] Step 8: Recording and analyzing user exercise data

[1040] The user performs the suggested exercise and records the results in the application. The exercise data is sent from the user's device to the server and stored in a database. The server analyzes this data and reflects it in future exercise suggestions.

[1041] Input: Exercise execution data

[1042] Output: Analysis data, improved exercise suggestions

[1043] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1044] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1045] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1046] [Third embodiment]

[1047] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1048] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1049] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1050] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1051] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1052] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1053] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1054] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1055] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1056] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1057] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1058] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1059] This invention is a diet support system that manages diet and suggests exercise based on personal data and food photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation.

[1060] System Overview

[1061] user:

[1062] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[1063] Device:

[1064] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results and exercise suggestions from the server to the user, allowing the user to watch exercise videos.

[1065] server:

[1066] The server receives the user's data, analyzes photos of meals using a generative AI model, and calculates calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. In addition, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1067] Program processing

[1068] 1. User data input:

[1069] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1070] 2. Upload your meal photos:

[1071] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[1072] 3. Photo analysis and calorie calculation:

[1073] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1074] 4. Exercise suggestions:

[1075] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[1076] 5. Exercise execution and recording:

[1077] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[1078] 6. Data Collection and Analysis:

[1079] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[1080] Specific examples

[1081] Let's take the example of User A using this system.

[1082] Initial Settings:

[1083] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[1084] Upload your meal photos and calculate calories:

[1085] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[1086] Exercise suggestions:

[1087] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[1088] Exercise log and next suggestion:

[1089] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[1090] As described above, the present invention is a system that effectively supports users in their healthy dieting by using a generative AI model to provide dietary management and exercise suggestions.

[1091] The processing flow will be explained below.

[1092] Step 1:

[1093] user

[1094] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[1095] Step 2:

[1096] Terminal

[1097] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[1098] Step 3:

[1099] user

[1100] Users take photos of their meals and upload them to the application.

[1101] Step 4:

[1102] Terminal

[1103] The device sends the uploaded photos of the meal to the server.

[1104] Step 5:

[1105] server

[1106] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[1107] Step 6:

[1108] server

[1109] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[1110] Step 7:

[1111] Terminal

[1112] The terminal displays the calorie information and advice received from the server on a user interface.

[1113] Step 8:

[1114] user

[1115] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[1116] Step 9:

[1117] server

[1118] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[1119] Step 10:

[1120] Terminal

[1121] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[1122] Step 11:

[1123] user

[1124] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[1125] Step 12:

[1126] Terminal

[1127] The terminal transmits the recorded exercise data to a server.

[1128] Step 13:

[1129] server

[1130] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[1131] Example 1

[1132] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] While conventional diet support systems use data and food photos entered by users to calculate calories and suggest exercises, they suffer from insufficient accuracy and individuality. Furthermore, they lack a means to effectively provide feedback based on the analysis of the data obtained and improve the overall performance of the system, making it difficult to provide optimal support to users.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1135] In this invention, the server includes means for the user to input personal data such as height, weight, age, sex, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories using a generative AI model, means for suggesting an exercise method based on the personal data and the calorie information, means for providing the suggested exercise method to the user, means for recording the user's exercise implementation data and transmitting it to the server, and means for continuously improving the generative AI model based on the recorded data. This enables highly accurate diet support tailored to the individual user and continuous improvement of system performance.

[1136] A "user" is a user of the system who is responsible for inputting personal data such as their height, weight, age, gender, lifestyle habits, and target weight, and uploading photos of their meals.

[1137] A "terminal" is a device used by a user that provides an interface for inputting, reviewing, and sending personal data and photos of meals.

[1138] The "server" is a central system that receives and analyzes data sent by users and uses a generative AI model to calculate calories and make exercise suggestions.

[1139] "Personal data" refers to information such as height, weight, age, gender, lifestyle habits, and target weight that users enter into the system.

[1140] A "generative AI model" is an algorithm or program that uses machine learning technology to analyze data and calculate calories and suggest exercises.

[1141] "Calorie Calculation" is a method of analyzing photos of meals uploaded by users and calculating calories burned based on the type and quantity of food.

[1142] "Exercise Suggestion" is the process of providing instructions and advice on the optimal type and frequency of exercise based on the user's personal data and calorie information.

[1143] A "database" is a storage device that the server uses to store and manage users' personal data, analysis results, exercise execution data, etc.

[1144] "Continuous improvement" is the process of training the generative AI model based on accumulated data to improve the accuracy of analysis and proposals.

[1145] The present invention is a diet support system that manages dietary management and suggests exercise based on personal data and meal photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation. The detailed configuration is described below.

[1146] System Overview

[1147] user:

[1148] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[1149] Device:

[1150] The device is a smartphone, tablet, or other device that transmits data entered by the user and uploaded meal photos to the server through a user interface. The device also displays analysis results and exercise suggestions from the server to the user, and allows the user to watch exercise videos.

[1151] server:

[1152] The server uses cloud-based computing resources to receive user data, analyzes food photos using a generative AI model, calculates calories, and generates optimal exercise and diet plans for the user based on the analyzed data. Additionally, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1153] Program processing

[1154] User data input:

[1155] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1156] Upload your meal photo:

[1157] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[1158] Photo analysis and calorie counting:

[1159] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1160] Exercise suggestions:

[1161] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[1162] Exercise execution and recording:

[1163] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[1164] Data collection and analysis:

[1165] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[1166] Specific examples

[1167] Let's take the example of User A using this system.

[1168] Initial Settings:

[1169] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[1170] Upload your meal photos and calculate calories:

[1171] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[1172] Exercise suggestions:

[1173] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[1174] Exercise log and next suggestion:

[1175] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[1176] Example prompt sentence:

[1177] "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is recommended."

[1178] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1179] Step 1:

[1180] User Data Entry

[1181] The user launches the application and enters personal data (height, weight, age, gender, lifestyle habits, and target weight).

[1182] The device receives this entered personal data and checks that the data is in the correct format (e.g., height and weight are numeric).

[1183] The terminal transmits the checked data to the server.

[1184] The server stores the received personal data in a database.

[1185] Input: Personal data (height, weight, age, gender, lifestyle habits, goal weight)

[1186] Output: Personal data stored in a database

[1187] Step 2:

[1188] Upload a photo of your meal

[1189] Users take photos of their daily meals and upload them through the application.

[1190] The device receives the photo and checks the resolution and format.

[1191] The device sends the confirmed photo to the server.

[1192] The server receives the photo data and stores it in a database.

[1193] Input: Food photo

[1194] Output: Photo data stored in a database

[1195] Step 3:

[1196] Photo analysis and calorie calculation

[1197] The server retrieves the photo data from the database and begins analyzing it using the generative AI model.

[1198] A generative AI model (e.g., using TensorFlow or PyTorch) is fed photos of food to identify the type and quantity of food.

[1199] The server calculates the calories for each identified food item.

[1200] The server stores the calculation results (calories of each food item and total calories) in a database and sends the results to the terminal.

[1201] The terminal receives the analysis results from the server and notifies the user.

[1202] Input: Food photo data

[1203] Output: Calorie calculation result (notification to user)

[1204] Step 4:

[1205] Exercise suggestions

[1206] The server generates the optimal exercise method based on the user's basic data (personal data) and calorie information.

[1207] A prompt sentence is input into the generative AI model to generate exercise suggestions appropriate for the user (e.g., "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is appropriate").

[1208] The server stores the generated exercise suggestions in a database and transmits them to the terminal.

[1209] The device displays the received exercise suggestions to the user and, in some cases, provides links to exercise videos.

[1210] Input: Basic data, calorie information

[1211] Output: Exercise suggestions and exercise video links (notification to user)

[1212] Step 5:

[1213] Exercise execution and recording

[1214] The user performs the suggested exercise (e.g., a 30-minute walk) and records the results in the application.

[1215] The device receives the exercise record (type of exercise performed, time, intensity, etc.) and sends it to the server.

[1216] The server stores the received exercise data in a database and reflects it in the next proposal.

[1217] Input: Exercise record data

[1218] Output: Exercise data saved in the database (reflected in the next proposal)

[1219] Step 6:

[1220] Data accumulation and analysis

[1221] The server periodically re-analyzes the accumulated user data (personal data, calorie information, exercise data).

[1222] Using a generative AI model, the model is trained based on new data, improving analysis accuracy.

[1223] The server uses the improved generative AI model to further customize your next exercise suggestion.

[1224] Input: Accumulated user data

[1225] Output: An improved generative AI model and motion suggestions based on it

[1226] (Application example 1)

[1227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1228] While existing diet support systems allow users to input personal data, upload photos of their meals, and calculate calories, they lack specificity and individuality in their exercise recommendations. They also offer limited means for effectively managing diet and exercise progress. Furthermore, technological solutions are needed to enhance convenience in online and virtual environments.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1230] In this invention, the server includes means for a user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for providing the suggested exercise methods to the user and for viewing exercise videos, means for recording and analyzing the user's exercise implementation data, and means for the user to log in to a virtual store to record meals and receive exercise guidance. This allows the user to receive personalized exercise suggestions in a virtual environment, enabling effective management of diet and exercise progress.

[1231] "Personal data" refers to data entered by the user, such as height, weight, lifestyle habits, and target weight.

[1232] "Meal photos" refer to image data taken by users of their own meals.

[1233] "Calorie counting" refers to the process of analyzing a photo of a meal to calculate the total calories in that meal.

[1234] "Exercise Method" refers to a specific exercise plan suggested based on the user's personal data and calorie information.

[1235] "Exercise videos" refer to video content that allows users to exercise while watching.

[1236] A "virtual store" refers to a virtual store environment that users can access online to record their meals and receive exercise guidance.

[1237] "Exercise data" refers to data such as the type of exercise the user actually performed, the duration, and the calories burned.

[1238] "User interface" refers to the display and input devices that allow a user to interact with a system.

[1239] "Generative AI model" refers to an artificial intelligence model that analyzes data collected from users and generates personalized suggestions.

[1240] The present invention is a diet support system that includes input of personal data, analysis of food photos, calorie calculation, exercise method suggestions, exercise recording and analysis, and use in a virtual store.

[1241] The server receives personal data entered by the user, such as height, weight, lifestyle habits, and target weight, and stores the data in a database. The user enters this personal data through a device such as a smartphone or PC. The device then transmits the personal data entered via the user interface to the server.

[1242] Next, users take photos of their daily meals and upload them from their devices. The server analyzes the uploaded photos using a generative AI model to calculate the calories of each meal. The analyzed calorie information is stored in a database and provided to the user. This information can be viewed through the user interface.

[1243] The server then proposes a personalized exercise plan based on the user's personal data and calorie information obtained from their diet. This includes specific exercise types and frequency, as well as links to exercise videos. The user then performs the exercises based on the proposed plan and records the results on their device. The recorded exercise data is then sent back to the server and stored in a database.

[1244] Additionally, users can log in to the virtual store to record their meals and receive exercise guidance, where they can track their progress and interact with other users for added motivation.

[1245] As a concrete example, let's consider the case of User A using this system. User A enters his / her height of 170 cm, weight of 75 kg, lifestyle habits (desk work), and target weight of 65 kg, and uploads photos of his / her daily meals. For example, if he / she uploads a photo of sushi taken for lunch, the server analyzes it and calculates the calories. Next, the system suggests that he / she walk 30 minutes every day this week and watch yoga videos three times a week. When User A records the results of his / her exercise, the results are reflected in the next exercise suggestions.

[1246] An example of a prompt sentence is, "Please analyze the photo of sushi taken by user A and calculate the calories." In this way, users can receive personalized exercise suggestions and effectively manage their diet and exercise progress.

[1247] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1248] Step 1:

[1249] The user uses the device to enter personal data such as height, weight, lifestyle habits, and target weight. The entered data is sent from the device to the server, which receives it and stores it in a database. Specifically, when the user enters the required information into the form on the screen and presses the submit button, the device sends this information in JSON format to the server. The server then analyzes the received data and stores it in the appropriate database fields.

[1250] Input: Personal data such as height, weight, lifestyle habits, and target weight

[1251] Output: Personal data stored in a database

[1252] Step 2:

[1253] Users take photos of their daily meals on their device and upload them through the application. The device then sends the uploaded photos to the server. The server receives the photos and inputs the photo data into a generative AI model for analysis. Specifically, users press the upload button to submit the photos they have taken, and the device then sends the image file to the server. The server then runs the received image file through its analysis engine.

[1254] Input: Photo of food (image data)

[1255] Output: Parsed calorie information

[1256] Step 3:

[1257] The server uses the generative AI model to analyze the uploaded photo of the meal and calculate the calories of each food item in the meal. This calorie information is stored in a database. Specifically, after the server receives the photo of the meal, it inputs the image into the generative AI model to identify the food and predict its calories. The results of this processing are stored in a database.

[1258] Input: Food photo

[1259] Output: Calorie information for each food

[1260] Step 4:

[1261] The server then proposes a personalized exercise plan based on the user's personal data and analyzed calorie information. The exercise plan includes specific exercise types and frequency, as well as links to exercise videos. Specifically, the server obtains the user's personal data and calorie information and uses a generative AI model to generate an exercise plan. The generated exercise plan is stored in a database and sent to the device.

[1262] Input: Personal data, calorie information

[1263] Output: Proposed exercise method

[1264] Step 5:

[1265] The user exercises based on the proposed exercise plan and records the results on the device. The device then sends this exercise data to the server. The server stores the received data in a database and uses it to provide future exercise suggestions. Specifically, the user enters the exercise status within the application and sends the data from the device to the server. The server then stores the received exercise data in a database.

[1266] Input: Exercise execution data

[1267] Output: Exercise execution data stored in a database

[1268] Step 6:

[1269] Users log in to the virtual store and record their meals and receive exercise guidance. The server supports activities within the virtual store based on the user's data. Specifically, users access the virtual store through a web browser or a dedicated app, and the server provides various data (meal records, exercise guidance, etc.).

[1270] Input: Login information

[1271] Output: Meal records and exercise guidance data in the virtual store

[1272] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1273] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[1274] System Overview

[1275] user:

[1276] Users install the system's application and create an account. After creating an account, users enter personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan. Users can also record their daily emotional state through photos and text input.

[1277] Device:

[1278] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results, exercise suggestions, and emotion-based messages to the user, and allows the user to watch exercise videos.

[1279] server:

[1280] The server receives the user's data and uses a generative AI model to analyze photos of meals and calculate calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. It also uses an emotion engine to analyze the user's emotional state and generates emotion-based feedback and encouraging messages. It also records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1281] Emotion Engine:

[1282] The emotion engine recognizes the user's emotional state by analyzing data such as text, voice, and photos entered by the user, classifying the emotional state, and generating appropriate feedback and encouraging messages for the user based on this.

[1283] Program processing

[1284] 1. User data input:

[1285] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1286] 2. Upload your meal photos:

[1287] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1288] 3. Photo analysis and calorie calculation:

[1289] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1290] 4. Exercise suggestions:

[1291] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[1292] 5. Emotion Recognition with Emotion Engine:

[1293] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[1294] 6. Emotion-based feedback:

[1295] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1296] 7. Exercise execution and recording:

[1297] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[1298] 8. Data Collection and Analysis:

[1299] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[1300] Specific examples

[1301] Let's take the example of User B using this system.

[1302] Initial Settings:

[1303] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[1304] Upload your meal photos and calculate calories:

[1305] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[1306] Exercise suggestions:

[1307] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[1308] Emotion Recognition and Feedback:

[1309] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the message "Great job! Keep it up!" is displayed.

[1310] Exercise log and next suggestion:

[1311] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[1312] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[1313] The processing flow will be explained below.

[1314] Step 1:

[1315] user

[1316] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[1317] Step 2:

[1318] Terminal

[1319] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[1320] Step 3:

[1321] user

[1322] Users take photos of their meals and upload them to the application.

[1323] Step 4:

[1324] Terminal

[1325] The device sends the uploaded photos of the meal to the server.

[1326] Step 5:

[1327] server

[1328] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[1329] Step 6:

[1330] server

[1331] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[1332] Step 7:

[1333] Terminal

[1334] The terminal displays the calorie information and advice received from the server on a user interface.

[1335] Step 8:

[1336] user

[1337] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[1338] Step 9:

[1339] server

[1340] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[1341] Step 10:

[1342] Terminal

[1343] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[1344] Step 11:

[1345] user

[1346] Users record their emotional state using text, photos, and audio, and the device transmits this emotional data to a server.

[1347] Step 12:

[1348] server

[1349] The server's emotion engine analyzes the received emotion data and classifies the user's emotional state, for example into categories such as "positive" or "negative."

[1350] Step 13:

[1351] server

[1352] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1353] Step 14:

[1354] Terminal

[1355] The device displays the feedback and messages received from the server in the user interface, allowing the user to check and decide on their next action.

[1356] Step 15:

[1357] user

[1358] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[1359] Step 16:

[1360] Terminal

[1361] The terminal transmits the recorded exercise data to a server.

[1362] Step 17:

[1363] server

[1364] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[1365] Example 2

[1366] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1367] Current diet support systems only provide basic functions for users' diet management and exercise suggestions, and do not take into account the user's emotional state, making it difficult to maintain user motivation.In addition, data accumulation and analysis to improve the accuracy of the system's suggestions are insufficient, making it difficult to provide the optimal plan for each user.

[1368] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1369] In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for inputting the user's emotional data and analyzing the input emotional data, means for providing feedback to the user based on the analyzed emotional data, and means for recording and analyzing the user's exercise implementation data. This makes it possible to maintain motivation taking into account the user's emotional state, and to provide an individually optimized diet plan through data accumulation and analysis.

[1370] A "user" is an individual who uses the system and inputs personal data such as height, weight, lifestyle habits, and target weight.

[1371] A "terminal" is a device that a user uses to connect to the system and perform tasks such as entering data, uploading photos, and displaying feedback.

[1372] The "server" is a computer system that receives, analyzes, and stores data sent by users, and is the central processing unit that suggests exercise methods and performs emotion analysis.

[1373] "Personal data" refers to basic information required for the system to generate a diet plan, such as the user's height, weight, lifestyle habits, and target weight.

[1374] "Meal photos" refer to image files that users take and upload to the system to record the contents of their meals.

[1375] "Calories" is a unit that indicates the amount of energy in food, and is a value calculated by the system by analyzing photos of meals.

[1376] "Exercise method" refers to the plan and type of physical activity that the server suggests based on the user's personal data and calorie information.

[1377] "Emotional Data" refers to information such as text, photos, and audio that users enter to record their emotional state.

[1378] An "emotion engine" refers to an algorithm or software component that analyzes input emotional data and classifies the user's emotional state.

[1379] "Feedback" refers to messages of encouragement or advice provided to users based on emotional data analyzed by the emotion engine.

[1380] "Exercise execution data" is data that records the details and results of the exercise performed by the user (e.g., walking distance, time, calories burned), and is sent to the server.

[1381] "Database" refers to the storage system for storing and managing personal data, meal photos, calorie information, exercise data, and emotional data collected by the System from Users.

[1382] A "generative AI model" is an artificial intelligence algorithm that performs data analysis and pattern recognition, and is used to analyze food photos, suggest exercise methods, analyze emotional data, and more.

[1383] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[1384] Hardware and Software Used

[1385] This system uses the following hardware and software:

[1386] User's device: a mobile device such as a smartphone or tablet

[1387] Server: High-performance data processing and analysis server

[1388] Generative AI models: Artificial intelligence models for food photo analysis, calorie calculation, exercise suggestions, and emotion recognition

[1389] Emotion engine: A software component that analyzes input emotion data

[1390] Processing flow

[1391] 1. User data entry

[1392] Users install the system's application and enter their personal data, which is then sent to the server, which stores it in a database.

[1393] 2. Upload a photo of your meal

[1394] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1395] 3. Analysis of food photos and calorie calculation

[1396] The server uses a generative AI model to analyze the received photos and calculate the calorie content of each meal, which is then stored in a database and provided to the user.

[1397] 4. Exercise suggestions

[1398] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[1399] 5. Emotion Recognition by Emotion Engine

[1400] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[1401] 6. Emotional Feedback

[1402] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1403] 7. Exercise execution and recording

[1404] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[1405] 8. Data accumulation and analysis

[1406] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[1407] Specific examples

[1408] Let's take the example of User B using this system.

[1409] Initial Settings:

[1410] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[1411] Upload your meal photos and calculate calories:

[1412] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[1413] Exercise suggestions:

[1414] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[1415] Emotion Recognition and Feedback:

[1416] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the server provides feedback such as, "Great job! Keep it up!"

[1417] Exercise log and next suggestion:

[1418] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[1419] Prompt Sentence Examples

[1420] Below are some examples of specific prompt sentences.

[1421] Food photo analysis prompt: "Analyze the provided food photo and calculate the calorie information. Display the name and calories of the food contained in the photo as a result."

[1422] Sentiment analysis prompt: "Classify the user's emotion from the following text. Text: 'Today I felt tired but accomplished.'"

[1423] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[1424] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1425] Step 1:

[1426] User Data Entry

[1427] The user installs the system application and enters personal data (e.g., height, weight, age, gender, lifestyle habits, target weight, etc.). The device sends this data to the server, which then stores the received data in a database.

[1428] Input: Personal data entered by the user into the application

[1429] Output: Personal data stored on the server

[1430] Step 2:

[1431] Upload a photo of your meal

[1432] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1433] Input: Food photos taken and uploaded by users

[1434] Output: Meal photos stored on the server

[1435] Step 3:

[1436] Analyze food photos and calculate calories

[1437] The server inputs the received meal photo into the generative AI model and begins its analysis. Specifically, it uses image recognition algorithms to identify the type and quantity of ingredients. It then compares the results with an existing database to calculate the calories of each ingredient. The results are stored in a database and provided to the user.

[1438] Input: Food photos stored on the server

[1439] Data processing: Image recognition and calorie calculation with generative AI models

[1440] Output: Analysis results including calorie information

[1441] Step 4:

[1442] Exercise suggestions

[1443] The server uses a generative AI model to calculate the optimal exercise method based on the user's basic data (e.g., height, weight, lifestyle, and target weight) and calorie information. Specifically, this information includes the type and frequency of exercise that corresponds to the user's lifestyle and goals, as well as links to appropriate exercise videos. This information is sent to the device and displayed to the user.

[1444] Input: Personal data, calorie information

[1445] Data Computing: Optimizing Movement Strategies with Generative AI Models

[1446] Output: Exercise suggestions

[1447] Step 5:

[1448] Entering emotion data

[1449] Users record their emotional state through text, photos, and audio, and the device transmits this emotional data to a server.

[1450] Input: Emotion data entered by the user (text, photo, voice)

[1451] Output: Emotion data stored on the server

[1452] Step 6:

[1453] Emotion analysis using an emotion engine

[1454] The server's emotion engine analyzes the received emotion data. Specifically, it analyzes text data using natural language processing (NLP), performs tone analysis on audio data, and performs facial expression recognition on photo data. This allows it to classify the user's emotional state as "positive," "negative," or "neutral."

[1455] Input: Emotion data

[1456] Data processing: natural language processing, voice tone analysis, facial expression recognition

[1457] Output: Classified emotional state

[1458] Step 7:

[1459] Emotion-based feedback

[1460] The server generates appropriate feedback and encouraging messages for the user based on the emotional state analyzed by the emotion engine. For example, if the emotion is classified as "negative," an encouraging message is generated. The generated message is sent to the device and displayed to the user.

[1461] Input: Classified emotional state

[1462] Data processing: feedback message generation

[1463] Output: Feedback message

[1464] Step 8:

[1465] Exercise execution and recording

[1466] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent from the device to the server and stored in a database.

[1467] Input: User recorded exercise results

[1468] Output: Exercise execution data stored on the server

[1469] Step 9:

[1470] Accumulating data and making proposals for the next time

[1471] The server analyzes the accumulated user data to improve the accuracy of the generated AI model. For example, it uses past exercise results and diet data to optimize the next exercise method. The analysis results are reflected in the diet plan suggestions for the next time and are sent to the device.

[1472] Input: Accumulated user data

[1473] Data Computation: Analysis and Accuracy Improvement with Generative AI Models

[1474] Output: Optimized next exercise suggestions

[1475] (Application example 2)

[1476] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1477] In recent years, with the rise in health awareness, there is a demand for systems that allow individuals to manage their own health status and diet effectively. However, many existing diet support systems do not take into account the user's emotional state, which can lead to users losing motivation midway through the process and making it difficult to continue using the system. For this reason, there is a demand for a system that can analyze the user's emotional data and provide appropriate feedback and exercise suggestions based on that data to improve the user's motivation and provide effective diet support.

[1478] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting an exercise method based on the personal data and the calorie information, means for inputting emotional data, means for analyzing the input emotional data and classifying the emotional state, means for providing the suggested exercise method to the user, means for generating a feedback message based on the emotional state, means for providing the feedback message to the user, and means for recording and analyzing the user's exercise implementation data. This makes it possible to provide feedback and exercise suggestions based on the user's emotional state.

[1479] A "user" is an individual who uses the system and is the entity that inputs personal data and provides food photos and emotional data.

[1480] "Personal Data" refers to information about a user's health and lifestyle, such as height, weight, lifestyle habits, and target weight, entered by the user.

[1481] "Meal photos" are image data that users take and upload of their daily meals, and are used for calorie calculation and nutritional analysis.

[1482] "Calorie information" refers to the calorie value of each meal obtained by analyzing the photograph of the meal.

[1483] "Exercise method" refers to specific exercise menus and exercises suggested to users based on personal data and calorie information.

[1484] "Emotional data" refers to data that indicates the user's emotional state and is entered into the system, and is recorded in the form of text, photos, audio, etc.

[1485] "Emotional state" refers to the user's current mental state and mood, which can be obtained by analyzing emotional data.

[1486] "Feedback messages" are messages such as encouragement or suggestions that the emotion engine generates based on the user's emotional state.

[1487] A "user interface" refers to the screen or operating means that allows a user to interact with a system and input and output data.

[1488] A "generative model" is an AI model that analyzes data collected from users to perform tasks such as calorie calculations, emotional classification, and exercise suggestions.

[1489] The system that realizes this application example operates in cooperation with the elements of the user terminal, server, and emotion engine.

[1490] 1. User Device

[1491] The user device refers to a smartphone, and applications can be developed using Flutter. Through the application, users enter personal data (height, weight, lifestyle habits, target weight, etc.) and upload photos of meals and emotional data. This data entered by the user is sent to a server via the Internet.

[1492] 2. Server

[1493] The server is responsible for several functions, using TensorFlow and Firebase for these processing. The main functions of the server are:

[1494] 1. Storage and management of personal data

[1495] The personal data entered by users is stored in a real-time Firebase database, which is used for calorie calculations, exercise suggestions, and sentiment analysis.

[1496] 2. Analysis of food photos and calorie calculation

[1497] The server receives photos of meals uploaded by users and performs image analysis using TensorFlow, calculating the calorie value of each meal as a result of the analysis and storing it in a database.

[1498] 3. Emotion Data Analysis

[1499] Emotional data input from users (text, photos, voice, etc.) is analyzed using TensorFlow to classify the emotional state, and the analysis results are passed to the emotion engine, which generates appropriate feedback messages.

[1500] 4. Exercise suggestions

[1501] The server generates an optimal exercise plan based on the user's personal data and calorie information, including personalized exercise menus and links to exercise videos.

[1502] 5. Generating and Displaying Feedback Messages

[1503] The server generates a feedback message based on the analyzed emotional state using the emotion engine, and the generated message is provided to the user through a user interface.

[1504] 6. Accumulating data and continuously improving the model

[1505] The server accumulates data collected from users and continuously refines the generative AI model, which improves the accuracy of future dietary analysis and exercise recommendations.

[1506] 3. Emotion Engine

[1507] The emotion engine is a system that recognizes and analyzes the user's emotional state. It uses text data and photo data entered by the user to classify the emotional state and reflects the analysis results in a feedback message.

[1508] Adding specific examples

[1509] Here are some examples of specific prompts:

[1510] Initialization prompt:

[1511] User B, height 160cm, weight 60kg, age 28, target weight 55kg. His lifestyle is mostly desk work and his hobby is jogging. Enter the initial setup information.

[1512] Dietary analysis prompt:

[1513] Analyze a photo of lunch and calculate the calories. The menu is salad and soup.

[1514] Sentiment Analysis Prompt:

[1515] A user has entered the following sentence: Today I felt tired but accomplished. Analyze this and classify the emotional state.

[1516] This makes it possible to build a system that can provide feedback and suggest exercises based on the user's emotional state.

[1517] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1518] Step 1: Entering User Data

[1519] Users enter personal data such as height, weight, lifestyle habits, and target weight into the application form. This personal data is sent from the user's device via the Internet to the server, which then stores the received data in the Firebase real-time database.

[1520] Input: Personal data (height, weight, lifestyle habits, target weight, etc.)

[1521] Output: Personal data stored in a database

[1522] Step 2: Upload a photo of your meal

[1523] Users take photos of their daily meals and upload them to the application, which then sends the photo data from the user's device to a server.

[1524] Input: Food photo data

[1525] Output: Photo data sent to the server

[1526] Step 3: Analyze photos and calculate calories

[1527] The server uses TensorFlow to analyze the uploaded photos of meals and calculate the calorie content of each meal. The resulting calorie information is stored in a database and provided to the user through a user interface.

[1528] Input: Food photo data

[1529] Output: Parsed calorie information

[1530] Step 4: Exercise suggestions

[1531] Based on the user's personal data and calorie information, the server will suggest the best way to exercise, including type, frequency, or links to specific exercise videos. The suggestions will be stored in a database and provided to the user.

[1532] Input: Personal data, calorie information

[1533] Output: Exercise suggestions

[1534] Step 5: Enter emotion data

[1535] Users record their emotional state after exercise or daily using text, photos, or audio, and this emotional data is sent from the user's device to a server.

[1536] Input: Emotion data (text, photo, voice)

[1537] Output: Emotion data sent to the server

[1538] Step 6: Sentiment Analysis

[1539] The server uses an emotion engine to analyze the transmitted emotion data and classify the emotion state, which is then stored in a database.

[1540] Input: Emotion data

[1541] Output: Classified emotional state

[1542] Step 7: Generate feedback messages

[1543] The server generates an appropriate feedback message using the generative AI model based on the classified emotional state, and the generated feedback message is provided to the user through a user interface.

[1544] Input: Classified emotional state

[1545] Output: Feedback message

[1546] Step 8: Recording and analyzing user exercise data

[1547] The user performs the suggested exercise and records the results in the application. The exercise data is sent from the user's device to the server and stored in a database. The server analyzes this data and reflects it in future exercise suggestions.

[1548] Input: Exercise execution data

[1549] Output: Analysis data, improved exercise suggestions

[1550] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1551] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1552] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1553] [Fourth embodiment]

[1554] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1555] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1556] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1557] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1558] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1560] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1561] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1562] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1563] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1564] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1565] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1566] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1567] This invention is a diet support system that manages diet and suggests exercise based on personal data and food photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation.

[1568] System Overview

[1569] user:

[1570] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[1571] Device:

[1572] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results and exercise suggestions from the server to the user, allowing the user to watch exercise videos.

[1573] server:

[1574] The server receives the user's data, analyzes photos of meals using a generative AI model, and calculates calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. In addition, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1575] Program processing

[1576] 1. User data input:

[1577] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1578] 2. Upload your meal photos:

[1579] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[1580] 3. Photo analysis and calorie calculation:

[1581] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1582] 4. Exercise suggestions:

[1583] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[1584] 5. Exercise execution and recording:

[1585] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[1586] 6. Data Collection and Analysis:

[1587] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[1588] Specific examples

[1589] Let's take the example of User A using this system.

[1590] Initial Settings:

[1591] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[1592] Upload your meal photos and calculate calories:

[1593] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[1594] Exercise suggestions:

[1595] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[1596] Exercise log and next suggestion:

[1597] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[1598] As described above, the present invention is a system that effectively supports users in their healthy dieting by using a generative AI model to provide dietary management and exercise suggestions.

[1599] The processing flow will be explained below.

[1600] Step 1:

[1601] user

[1602] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[1603] Step 2:

[1604] Terminal

[1605] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[1606] Step 3:

[1607] user

[1608] Users take photos of their meals and upload them to the application.

[1609] Step 4:

[1610] Terminal

[1611] The device sends the uploaded photos of the meal to the server.

[1612] Step 5:

[1613] server

[1614] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[1615] Step 6:

[1616] server

[1617] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[1618] Step 7:

[1619] Terminal

[1620] The terminal displays the calorie information and advice received from the server on a user interface.

[1621] Step 8:

[1622] user

[1623] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[1624] Step 9:

[1625] server

[1626] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[1627] Step 10:

[1628] Terminal

[1629] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[1630] Step 11:

[1631] user

[1632] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[1633] Step 12:

[1634] Terminal

[1635] The terminal transmits the recorded exercise data to a server.

[1636] Step 13:

[1637] server

[1638] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[1639] Example 1

[1640] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1641] While conventional diet support systems use data and food photos entered by users to calculate calories and suggest exercises, they suffer from insufficient accuracy and individuality. Furthermore, they lack a means to effectively provide feedback based on the analysis of the data obtained and improve the overall performance of the system, making it difficult to provide optimal support to users.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1643] In this invention, the server includes means for the user to input personal data such as height, weight, age, sex, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories using a generative AI model, means for suggesting an exercise method based on the personal data and the calorie information, means for providing the suggested exercise method to the user, means for recording the user's exercise implementation data and transmitting it to the server, and means for continuously improving the generative AI model based on the recorded data. This enables highly accurate diet support tailored to the individual user and continuous improvement of system performance.

[1644] A "user" is a user of the system who is responsible for inputting personal data such as their height, weight, age, gender, lifestyle habits, and target weight, and uploading photos of their meals.

[1645] A "terminal" is a device used by a user that provides an interface for inputting, reviewing, and sending personal data and photos of meals.

[1646] The "server" is a central system that receives and analyzes data sent by users and uses a generative AI model to calculate calories and make exercise suggestions.

[1647] "Personal data" refers to information such as height, weight, age, gender, lifestyle habits, and target weight that users enter into the system.

[1648] A "generative AI model" is an algorithm or program that uses machine learning technology to analyze data and calculate calories and suggest exercises.

[1649] "Calorie Calculation" is a method of analyzing photos of meals uploaded by users and calculating calories burned based on the type and quantity of food.

[1650] "Exercise Suggestion" is the process of providing instructions and advice on the optimal type and frequency of exercise based on the user's personal data and calorie information.

[1651] A "database" is a storage device that the server uses to store and manage users' personal data, analysis results, exercise execution data, etc.

[1652] "Continuous improvement" is the process of training the generative AI model based on accumulated data to improve the accuracy of analysis and proposals.

[1653] The present invention is a diet support system that manages dietary management and suggests exercise based on personal data and meal photos entered by the user. This system is realized by the actions of the user, terminal, and server, and their cooperation. The detailed configuration is described below.

[1654] System Overview

[1655] user:

[1656] The user installs the system's application and creates an account. After creating the account, the user enters personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan.

[1657] Device:

[1658] The device is a smartphone, tablet, or other device that transmits data entered by the user and uploaded meal photos to the server through a user interface. The device also displays analysis results and exercise suggestions from the server to the user, and allows the user to watch exercise videos.

[1659] server:

[1660] The server uses cloud-based computing resources to receive user data, analyzes food photos using a generative AI model, calculates calories, and generates optimal exercise and diet plans for the user based on the analyzed data. Additionally, the server records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1661] Program processing

[1662] User data input:

[1663] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1664] Upload your meal photo:

[1665] Users take photos of their daily meals and upload them to their device, which then sends the photos to a server.

[1666] Photo analysis and calorie counting:

[1667] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1668] Exercise suggestions:

[1669] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including specific exercise types and frequency, as well as links to exercise videos.

[1670] Exercise execution and recording:

[1671] The user performs the suggested exercise and records the results in the application via their device. The recorded data is sent to a server and stored in a database.

[1672] Data collection and analysis:

[1673] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and customizes the next proposal.

[1674] Specific examples

[1675] Let's take the example of User A using this system.

[1676] Initial Settings:

[1677] User A inputs his height (170cm), weight (75kg), age (30), target weight (65kg), and lifestyle habits (he does a lot of desk work and does not exercise much). This data is sent to the server and saved.

[1678] Upload your meal photos and calculate calories:

[1679] User A takes a photo of their lunch (e.g., a hamburger and fries) and uploads it to the app. The server analyzes the photo and determines that the hamburger is 500 calories and the fries are 300 calories. The results are reported to User A.

[1680] Exercise suggestions:

[1681] Based on User A's information, the server suggests walking 30 minutes every day this week and advises him to exercise by watching a specific stretching video three times a week.

[1682] Exercise log and next suggestion:

[1683] User A walks and records the results in the app. The data is stored on the server and reflected in future exercise suggestions.

[1684] Example prompt sentence:

[1685] "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is recommended."

[1686] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1687] Step 1:

[1688] User Data Entry

[1689] The user launches the application and enters personal data (height, weight, age, gender, lifestyle habits, and target weight).

[1690] The device receives this entered personal data and checks that the data is in the correct format (e.g., height and weight are numeric).

[1691] The terminal transmits the checked data to the server.

[1692] The server stores the received personal data in a database.

[1693] Input: Personal data (height, weight, age, gender, lifestyle habits, goal weight)

[1694] Output: Personal data stored in a database

[1695] Step 2:

[1696] Upload a photo of your meal

[1697] Users take photos of their daily meals and upload them through the application.

[1698] The device receives the photo and checks the resolution and format.

[1699] The device sends the confirmed photo to the server.

[1700] The server receives the photo data and stores it in a database.

[1701] Input: Food photo

[1702] Output: Photo data stored in a database

[1703] Step 3:

[1704] Photo analysis and calorie calculation

[1705] The server retrieves the photo data from the database and begins analyzing it using the generative AI model.

[1706] A generative AI model (e.g., using TensorFlow or PyTorch) is fed photos of food to identify the type and quantity of food.

[1707] The server calculates the calories for each identified food item.

[1708] The server stores the calculation results (calories of each food item and total calories) in a database and sends the results to the terminal.

[1709] The terminal receives the analysis results from the server and notifies the user.

[1710] Input: Food photo data

[1711] Output: Calorie calculation result (notification to user)

[1712] Step 4:

[1713] Exercise suggestions

[1714] The server generates the optimal exercise method based on the user's basic data (personal data) and calorie information.

[1715] A prompt sentence is input into the generative AI model to generate exercise suggestions appropriate for the user (e.g., "For a 30-year-old male with a sedentary lifestyle, 30 minutes of walking is appropriate").

[1716] The server stores the generated exercise suggestions in a database and transmits them to the terminal.

[1717] The device displays the received exercise suggestions to the user and, in some cases, provides links to exercise videos.

[1718] Input: Basic data, calorie information

[1719] Output: Exercise suggestions and exercise video links (notification to user)

[1720] Step 5:

[1721] Exercise execution and recording

[1722] The user performs the suggested exercise (e.g., a 30-minute walk) and records the results in the application.

[1723] The device receives the exercise record (type of exercise performed, time, intensity, etc.) and sends it to the server.

[1724] The server stores the received exercise data in a database and reflects it in the next proposal.

[1725] Input: Exercise record data

[1726] Output: Exercise data saved in the database (reflected in the next proposal)

[1727] Step 6:

[1728] Data accumulation and analysis

[1729] The server periodically re-analyzes the accumulated user data (personal data, calorie information, exercise data).

[1730] Using a generative AI model, the model is trained based on new data, improving analysis accuracy.

[1731] The server uses the improved generative AI model to further customize your next exercise suggestion.

[1732] Input: Accumulated user data

[1733] Output: An improved generative AI model and motion suggestions based on it

[1734] (Application example 1)

[1735] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1736] While existing diet support systems allow users to input personal data, upload photos of their meals, and calculate calories, they lack specificity and individuality in their exercise recommendations. They also offer limited means for effectively managing diet and exercise progress. Furthermore, technological solutions are needed to enhance convenience in online and virtual environments.

[1737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1738] In this invention, the server includes means for a user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for providing the suggested exercise methods to the user and for viewing exercise videos, means for recording and analyzing the user's exercise implementation data, and means for the user to log in to a virtual store to record meals and receive exercise guidance. This allows the user to receive personalized exercise suggestions in a virtual environment, enabling effective management of diet and exercise progress.

[1739] "Personal data" refers to data entered by the user, such as height, weight, lifestyle habits, and target weight.

[1740] "Meal photos" refer to image data taken by users of their own meals.

[1741] "Calorie counting" refers to the process of analyzing a photo of a meal to calculate the total calories in that meal.

[1742] "Exercise Method" refers to a specific exercise plan suggested based on the user's personal data and calorie information.

[1743] "Exercise videos" refer to video content that allows users to exercise while watching.

[1744] A "virtual store" refers to a virtual store environment that users can access online to record their meals and receive exercise guidance.

[1745] "Exercise data" refers to data such as the type of exercise the user actually performed, the duration, and the calories burned.

[1746] "User interface" refers to the display and input devices that allow a user to interact with a system.

[1747] "Generative AI model" refers to an artificial intelligence model that analyzes data collected from users and generates personalized suggestions.

[1748] The present invention is a diet support system that includes input of personal data, analysis of food photos, calorie calculation, exercise method suggestions, exercise recording and analysis, and use in a virtual store.

[1749] The server receives personal data entered by the user, such as height, weight, lifestyle habits, and target weight, and stores the data in a database. The user enters this personal data through a device such as a smartphone or PC. The device then transmits the personal data entered via the user interface to the server.

[1750] Next, users take photos of their daily meals and upload them from their devices. The server analyzes the uploaded photos using a generative AI model to calculate the calories of each meal. The analyzed calorie information is stored in a database and provided to the user. This information can be viewed through the user interface.

[1751] The server then proposes a personalized exercise plan based on the user's personal data and calorie information obtained from their diet. This includes specific exercise types and frequency, as well as links to exercise videos. The user then performs the exercises based on the proposed plan and records the results on their device. The recorded exercise data is then sent back to the server and stored in a database.

[1752] Additionally, users can log in to the virtual store to record their meals and receive exercise guidance, where they can track their progress and interact with other users for added motivation.

[1753] As a concrete example, let's consider the case of User A using this system. User A enters his / her height of 170 cm, weight of 75 kg, lifestyle habits (desk work), and target weight of 65 kg, and uploads photos of his / her daily meals. For example, if he / she uploads a photo of sushi taken for lunch, the server analyzes it and calculates the calories. Next, the system suggests that he / she walk 30 minutes every day this week and watch yoga videos three times a week. When User A records the results of his / her exercise, the results are reflected in the next exercise suggestions.

[1754] An example of a prompt sentence is, "Please analyze the photo of sushi taken by user A and calculate the calories." In this way, users can receive personalized exercise suggestions and effectively manage their diet and exercise progress.

[1755] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1756] Step 1:

[1757] The user uses the device to enter personal data such as height, weight, lifestyle habits, and target weight. The entered data is sent from the device to the server, which receives it and stores it in a database. Specifically, when the user enters the required information into the form on the screen and presses the submit button, the device sends this information in JSON format to the server. The server then analyzes the received data and stores it in the appropriate database fields.

[1758] Input: Personal data such as height, weight, lifestyle habits, and target weight

[1759] Output: Personal data stored in a database

[1760] Step 2:

[1761] Users take photos of their daily meals on their device and upload them through the application. The device then sends the uploaded photos to the server. The server receives the photos and inputs the photo data into a generative AI model for analysis. Specifically, users press the upload button to submit the photos they have taken, and the device then sends the image file to the server. The server then runs the received image file through its analysis engine.

[1762] Input: Photo of food (image data)

[1763] Output: Parsed calorie information

[1764] Step 3:

[1765] The server uses the generative AI model to analyze the uploaded photo of the meal and calculate the calories of each food item in the meal. This calorie information is stored in a database. Specifically, after the server receives the photo of the meal, it inputs the image into the generative AI model to identify the food and predict its calories. The results of this processing are stored in a database.

[1766] Input: Food photo

[1767] Output: Calorie information for each food

[1768] Step 4:

[1769] The server then proposes a personalized exercise plan based on the user's personal data and analyzed calorie information. The exercise plan includes specific exercise types and frequency, as well as links to exercise videos. Specifically, the server obtains the user's personal data and calorie information and uses a generative AI model to generate an exercise plan. The generated exercise plan is stored in a database and sent to the device.

[1770] Input: Personal data, calorie information

[1771] Output: Proposed exercise method

[1772] Step 5:

[1773] The user exercises based on the proposed exercise plan and records the results on the device. The device then sends this exercise data to the server. The server stores the received data in a database and uses it to provide future exercise suggestions. Specifically, the user enters the exercise status within the application and sends the data from the device to the server. The server then stores the received exercise data in a database.

[1774] Input: Exercise execution data

[1775] Output: Exercise execution data stored in a database

[1776] Step 6:

[1777] Users log in to the virtual store and record their meals and receive exercise guidance. The server supports activities within the virtual store based on the user's data. Specifically, users access the virtual store through a web browser or a dedicated app, and the server provides various data (meal records, exercise guidance, etc.).

[1778] Input: Login information

[1779] Output: Meal records and exercise guidance data in the virtual store

[1780] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1781] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[1782] System Overview

[1783] user:

[1784] Users install the system's application and create an account. After creating an account, users enter personal data such as height, weight, age, gender, lifestyle habits, and target weight. This data is used as basic information to generate a diet plan. Users can also record their daily emotional state through photos and text input.

[1785] Device:

[1786] The device is responsible for transmitting data entered by the user and uploaded meal photos to the server through the user interface, and also displays analysis results, exercise suggestions, and emotion-based messages to the user, and allows the user to watch exercise videos.

[1787] server:

[1788] The server receives the user's data and uses a generative AI model to analyze photos of meals and calculate calories. Based on the analyzed data, it generates the optimal exercise method and diet management plan for the user. It also uses an emotion engine to analyze the user's emotional state and generates emotion-based feedback and encouraging messages. It also records the user's exercise data and stores it in a database to improve the accuracy of the generative AI model.

[1789] Emotion Engine:

[1790] The emotion engine recognizes the user's emotional state by analyzing data such as text, voice, and photos entered by the user, classifying the emotional state, and generating appropriate feedback and encouraging messages for the user based on this.

[1791] Program processing

[1792] 1. User data input:

[1793] Users enter their personal data through the application, and the device sends this information to the server, which stores it in a database.

[1794] 2. Upload your meal photos:

[1795] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1796] 3. Photo analysis and calorie calculation:

[1797] The server then analyzes the photos using a generative AI model to calculate the calories of each meal, which is then stored in a database and provided to the user.

[1798] 4. Exercise suggestions:

[1799] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[1800] 5. Emotion Recognition with Emotion Engine:

[1801] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[1802] 6. Emotion-based feedback:

[1803] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1804] 7. Exercise execution and recording:

[1805] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[1806] 8. Data Collection and Analysis:

[1807] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[1808] Specific examples

[1809] Let's take the example of User B using this system.

[1810] Initial Settings:

[1811] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[1812] Upload your meal photos and calculate calories:

[1813] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[1814] Exercise suggestions:

[1815] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[1816] Emotion Recognition and Feedback:

[1817] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the message "Great job! Keep it up!" is displayed.

[1818] Exercise log and next suggestion:

[1819] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[1820] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[1821] The processing flow will be explained below.

[1822] Step 1:

[1823] user

[1824] Users install the application and create an account, after which they enter personal data such as height, weight, age, gender, lifestyle habits, and target weight.

[1825] Step 2:

[1826] Terminal

[1827] The device sends the personal data entered by the user to the server, which then stores the user's basic information in a database.

[1828] Step 3:

[1829] user

[1830] Users take photos of their meals and upload them to the application.

[1831] Step 4:

[1832] Terminal

[1833] The device sends the uploaded photos of the meal to the server.

[1834] Step 5:

[1835] server

[1836] The generative AI model installed on the server analyzes the photos of the food it receives and calculates the calories for each meal. It uses image recognition technology to identify the dish and ingredients, and then references the calorie information for each ingredient in a database to calculate the total calories.

[1837] Step 6:

[1838] server

[1839] The server adds the calculated calorie information and meal content data to a database and provides the user with calorie information and suggestions for nutritional balance.

[1840] Step 7:

[1841] Terminal

[1842] The terminal displays the calorie information and advice received from the server on a user interface.

[1843] Step 8:

[1844] user

[1845] Users review the calorie information and advice provided and plan their next diet and exercise regimen.

[1846] Step 9:

[1847] server

[1848] Based on the user's basic data and calorie information, the server uses a generative AI model to suggest the optimal exercise method, including type of exercise, frequency, and links to suitable exercise videos.

[1849] Step 10:

[1850] Terminal

[1851] The device displays the exercise suggestions and exercise videos received from the server on a user interface, making them easily accessible to the user.

[1852] Step 11:

[1853] user

[1854] Users record their emotional state using text, photos, and audio, and the device transmits this emotional data to a server.

[1855] Step 12:

[1856] server

[1857] The server's emotion engine analyzes the received emotion data and classifies the user's emotional state, for example into categories such as "positive" or "negative."

[1858] Step 13:

[1859] server

[1860] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1861] Step 14:

[1862] Terminal

[1863] The device displays the feedback and messages received from the server in the user interface, allowing the user to check and decide on their next action.

[1864] Step 15:

[1865] user

[1866] The user follows the suggested exercise plan and records the results (e.g., walking distance, time, and calories burned) in the application.

[1867] Step 16:

[1868] Terminal

[1869] The terminal transmits the recorded exercise data to a server.

[1870] Step 17:

[1871] server

[1872] The server stores the received exercise data in a database, continuously assesses the user's progress, and uses the accumulated data to improve the accuracy of the generative AI model and refine future diet plan suggestions.

[1873] Example 2

[1874] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1875] Current diet support systems only provide basic functions for users' diet management and exercise suggestions, and do not take into account the user's emotional state, making it difficult to maintain user motivation.In addition, data accumulation and analysis to improve the accuracy of the system's suggestions are insufficient, making it difficult to provide the optimal plan for each user.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1877] In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting exercise methods based on the personal data and the calorie information, means for inputting the user's emotional data and analyzing the input emotional data, means for providing feedback to the user based on the analyzed emotional data, and means for recording and analyzing the user's exercise implementation data. This makes it possible to maintain motivation taking into account the user's emotional state, and to provide an individually optimized diet plan through data accumulation and analysis.

[1878] A "user" is an individual who uses the system and inputs personal data such as height, weight, lifestyle habits, and target weight.

[1879] A "terminal" is a device that a user uses to connect to the system and perform tasks such as entering data, uploading photos, and displaying feedback.

[1880] The "server" is a computer system that receives, analyzes, and stores data sent by users, and is the central processing unit that suggests exercise methods and performs emotion analysis.

[1881] "Personal data" refers to basic information required for the system to generate a diet plan, such as the user's height, weight, lifestyle habits, and target weight.

[1882] "Meal photos" refer to image files that users take and upload to the system to record the contents of their meals.

[1883] "Calories" is a unit that indicates the amount of energy in food, and is a value calculated by the system by analyzing photos of meals.

[1884] "Exercise method" refers to the plan and type of physical activity that the server suggests based on the user's personal data and calorie information.

[1885] "Emotional Data" refers to information such as text, photos, and audio that users enter to record their emotional state.

[1886] An "emotion engine" refers to an algorithm or software component that analyzes input emotional data and classifies the user's emotional state.

[1887] "Feedback" refers to messages of encouragement or advice provided to users based on emotional data analyzed by the emotion engine.

[1888] "Exercise execution data" is data that records the details and results of the exercise performed by the user (e.g., walking distance, time, calories burned), and is sent to the server.

[1889] "Database" refers to the storage system for storing and managing personal data, meal photos, calorie information, exercise data, and emotional data collected by the System from Users.

[1890] A "generative AI model" is an artificial intelligence algorithm that performs data analysis and pattern recognition, and is used to analyze food photos, suggest exercise methods, analyze emotional data, and more.

[1891] This invention combines a diet support system that manages dietary management and suggests exercise based on personal data and food photos entered by the user with an emotion engine that recognizes the user's emotions, thereby providing a function to improve the user's motivation. This system is realized by the cooperation of the user, terminal, server, and emotion engine.

[1892] Hardware and Software Used

[1893] This system uses the following hardware and software:

[1894] User's device: a mobile device such as a smartphone or tablet

[1895] Server: High-performance data processing and analysis server

[1896] Generative AI models: Artificial intelligence models for food photo analysis, calorie calculation, exercise suggestions, and emotion recognition

[1897] Emotion engine: A software component that analyzes input emotion data

[1898] Processing flow

[1899] 1. User data entry

[1900] Users install the system's application and enter their personal data, which is then sent to the server, which stores it in a database.

[1901] 2. Upload a photo of your meal

[1902] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1903] 3. Analysis of food photos and calorie calculation

[1904] The server uses a generative AI model to analyze the received photos and calculate the calorie content of each meal, which is then stored in a database and provided to the user.

[1905] 4. Exercise suggestions

[1906] The server then uses the user's basic data and calorie information to suggest the optimal exercise method, including the type of exercise, frequency, and links to appropriate exercise videos.

[1907] 5. Emotion Recognition by Emotion Engine

[1908] Users can record their emotional state using text, photos, and audio. The device then sends this emotional data to a server, where an emotion engine analyzes it and classifies the user's emotional state.

[1909] 6. Emotional Feedback

[1910] The server generates appropriate feedback and encouraging messages for the user based on the analyzed emotional state, thereby maintaining or increasing the user's motivation.

[1911] 7. Exercise execution and recording

[1912] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent to the server and stored in a database.

[1913] 8. Data accumulation and analysis

[1914] The server analyzes the accumulated user data, improves the accuracy of the generated AI model, and improves the diet plan suggestions for the next time and beyond.

[1915] Specific examples

[1916] Let's take the example of User B using this system.

[1917] Initial Settings:

[1918] User B enters data such as height 160 cm, weight 60 kg, age 28, target weight 55 kg, and lifestyle habits (mainly desk work, jogging as a hobby). This data is sent to the server and saved.

[1919] Upload your meal photos and calculate calories:

[1920] User B takes a photo of their lunch (e.g., salad and soup) and uploads it to the app. The server analyzes the photo and determines that the salad is 150 calories and the soup is 100 calories. The results are reported to User B.

[1921] Exercise suggestions:

[1922] Based on User B's information, the server suggests jogging for 30 minutes every morning and stretching twice a week this week, along with a link to a specific stretching video.

[1923] Emotion Recognition and Feedback:

[1924] User B records his / her thoughts after the exercise in text, entering, "I felt tired today, but I also felt a sense of accomplishment." The server's emotion engine analyzes this and classifies User B's emotional state as "positive." Based on this, the server provides feedback such as, "Great job! Keep it up!"

[1925] Exercise log and next suggestion:

[1926] User B goes jogging and records the results in the app. The data is stored on the server and is reflected in future exercise suggestions.

[1927] Prompt Sentence Examples

[1928] Below are some examples of specific prompt sentences.

[1929] Food photo analysis prompt: "Analyze the provided food photo and calculate the calorie information. Display the name and calories of the food contained in the photo as a result."

[1930] Sentiment analysis prompt: "Classify the user's emotion from the following text. Text: 'Today I felt tired but accomplished.'"

[1931] As described above, the present invention is a system that recognizes the user's emotional state and provides dietary management and exercise suggestions based on this, thereby more effectively supporting the user's healthy diet.

[1932] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1933] Step 1:

[1934] User Data Entry

[1935] The user installs the system application and enters personal data (e.g., height, weight, age, gender, lifestyle habits, target weight, etc.). The device sends this data to the server, which then stores the received data in a database.

[1936] Input: Personal data entered by the user into the application

[1937] Output: Personal data stored on the server

[1938] Step 2:

[1939] Upload a photo of your meal

[1940] Users take photos of their daily meals and upload them to the application, which then sends the photos to a server.

[1941] Input: Food photos taken and uploaded by users

[1942] Output: Meal photos stored on the server

[1943] Step 3:

[1944] Analyze food photos and calculate calories

[1945] The server inputs the received meal photo into the generative AI model and begins its analysis. Specifically, it uses image recognition algorithms to identify the type and quantity of ingredients. It then compares the results with an existing database to calculate the calories of each ingredient. The results are stored in a database and provided to the user.

[1946] Input: Food photos stored on the server

[1947] Data processing: Image recognition and calorie calculation with generative AI models

[1948] Output: Analysis results including calorie information

[1949] Step 4:

[1950] Exercise suggestions

[1951] The server uses a generative AI model to calculate the optimal exercise method based on the user's basic data (e.g., height, weight, lifestyle, and target weight) and calorie information. Specifically, this information includes the type and frequency of exercise that corresponds to the user's lifestyle and goals, as well as links to appropriate exercise videos. This information is sent to the device and displayed to the user.

[1952] Input: Personal data, calorie information

[1953] Data Computing: Optimizing Movement Strategies with Generative AI Models

[1954] Output: Exercise suggestions

[1955] Step 5:

[1956] Entering emotion data

[1957] Users record their emotional state through text, photos, and audio, and the device transmits this emotional data to a server.

[1958] Input: Emotion data entered by the user (text, photo, voice)

[1959] Output: Emotion data stored on the server

[1960] Step 6:

[1961] Emotion analysis using an emotion engine

[1962] The server's emotion engine analyzes the received emotion data. Specifically, it analyzes text data using natural language processing (NLP), performs tone analysis on audio data, and performs facial expression recognition on photo data. This allows it to classify the user's emotional state as "positive," "negative," or "neutral."

[1963] Input: Emotion data

[1964] Data processing: natural language processing, voice tone analysis, facial expression recognition

[1965] Output: Classified emotional state

[1966] Step 7:

[1967] Emotion-based feedback

[1968] The server generates appropriate feedback and encouraging messages for the user based on the emotional state analyzed by the emotion engine. For example, if the emotion is classified as "negative," an encouraging message is generated. The generated message is sent to the device and displayed to the user.

[1969] Input: Classified emotional state

[1970] Data processing: feedback message generation

[1971] Output: Feedback message

[1972] Step 8:

[1973] Exercise execution and recording

[1974] The user performs the suggested exercise and records the results (e.g., walking distance, time, and calories burned) in the application. The recorded data is sent from the device to the server and stored in a database.

[1975] Input: User recorded exercise results

[1976] Output: Exercise execution data stored on the server

[1977] Step 9:

[1978] Accumulating data and making proposals for the next time

[1979] The server analyzes the accumulated user data to improve the accuracy of the generated AI model. For example, it uses past exercise results and diet data to optimize the next exercise method. The analysis results are reflected in the diet plan suggestions for the next time and are sent to the device.

[1980] Input: Accumulated user data

[1981] Data Computation: Analysis and Accuracy Improvement with Generative AI Models

[1982] Output: Optimized next exercise suggestions

[1983] (Application example 2)

[1984] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1985] In recent years, with the rise in health awareness, there is a demand for systems that allow individuals to manage their own health status and diet effectively. However, many existing diet support systems do not take into account the user's emotional state, which can lead to users losing motivation midway through the process and making it difficult to continue using the system. For this reason, there is a demand for a system that can analyze the user's emotional data and provide appropriate feedback and exercise suggestions based on that data to improve the user's motivation and provide effective diet support.

[1986] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for the user to input personal data such as height, weight, lifestyle habits, and target weight, means for uploading photos of meals, means for analyzing the photos of meals and calculating calories, means for suggesting an exercise method based on the personal data and the calorie information, means for inputting emotional data, means for analyzing the input emotional data and classifying the emotional state, means for providing the suggested exercise method to the user, means for generating a feedback message based on the emotional state, means for providing the feedback message to the user, and means for recording and analyzing the user's exercise implementation data. This makes it possible to provide feedback and exercise suggestions based on the user's emotional state.

[1987] A "user" is an individual who uses the system and is the entity that inputs personal data and provides food photos and emotional data.

[1988] "Personal Data" refers to information about a user's health and lifestyle, such as height, weight, lifestyle habits, and target weight, entered by the user.

[1989] "Meal photos" are image data that users take and upload of their daily meals, and are used for calorie calculation and nutritional analysis.

[1990] "Calorie information" refers to the calorie value of each meal obtained by analyzing the photograph of the meal.

[1991] "Exercise method" refers to specific exercise menus and exercises suggested to users based on personal data and calorie information.

[1992] "Emotional data" refers to data that indicates the user's emotional state and is entered into the system, and is recorded in the form of text, photos, audio, etc.

[1993] "Emotional state" refers to the user's current mental state and mood, which can be obtained by analyzing emotional data.

[1994] "Feedback messages" are messages such as encouragement or suggestions that the emotion engine generates based on the user's emotional state.

[1995] A "user interface" refers to the screen or operating means that allows a user to interact with a system and input and output data.

[1996] A "generative model" is an AI model that analyzes data collected from users to perform tasks such as calorie calculations, emotional classification, and exercise suggestions.

[1997] The system that realizes this application example operates in cooperation with the elements of the user terminal, server, and emotion engine.

[1998] 1. User Device

[1999] The user device refers to a smartphone, and applications can be developed using Flutter. Through the application, users enter personal data (height, weight, lifestyle habits, target weight, etc.) and upload photos of meals and emotional data. This data entered by the user is sent to a server via the Internet.

[2000] 2. Server

[2001] The server is responsible for several functions, using TensorFlow and Firebase for these processing. The main functions of the server are:

[2002] 1. Storage and management of personal data

[2003] The personal data entered by users is stored in a real-time Firebase database, which is used for calorie calculations, exercise suggestions, and sentiment analysis.

[2004] 2. Analysis of food photos and calorie calculation

[2005] The server receives photos of meals uploaded by users and performs image analysis using TensorFlow, calculating the calorie value of each meal as a result of the analysis and storing it in a database.

[2006] 3. Emotion Data Analysis

[2007] Emotional data input from users (text, photos, voice, etc.) is analyzed using TensorFlow to classify the emotional state, and the analysis results are passed to the emotion engine, which generates appropriate feedback messages.

[2008] 4. Exercise suggestions

[2009] The server generates an optimal exercise plan based on the user's personal data and calorie information, including personalized exercise menus and links to exercise videos.

[2010] 5. Generating and Displaying Feedback Messages

[2011] The server generates a feedback message based on the analyzed emotional state using the emotion engine, and the generated message is provided to the user through a user interface.

[2012] 6. Accumulating data and continuously improving the model

[2013] The server accumulates data collected from users and continuously refines the generative AI model, which improves the accuracy of future dietary analysis and exercise recommendations.

[2014] 3. Emotion Engine

[2015] The emotion engine is a system that recognizes and analyzes the user's emotional state. It uses text data and photo data entered by the user to classify the emotional state and reflects the analysis results in a feedback message.

[2016] Adding specific examples

[2017] Here are some examples of specific prompts:

[2018] Initialization prompt:

[2019] User B, height 160cm, weight 60kg, age 28, target weight 55kg. His lifestyle is mostly desk work and his hobby is jogging. Enter the initial setup information.

[2020] Dietary analysis prompt:

[2021] Analyze a photo of lunch and calculate the calories. The menu is salad and soup.

[2022] Sentiment Analysis Prompt:

[2023] A user has entered the following sentence: Today I felt tired but accomplished. Analyze this and classify the emotional state.

[2024] This makes it possible to build a system that can provide feedback and suggest exercises based on the user's emotional state.

[2025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2026] Step 1: Entering User Data

[2027] Users enter personal data such as height, weight, lifestyle habits, and target weight into the application form. This personal data is sent from the user's device via the Internet to the server, which then stores the received data in the Firebase real-time database.

[2028] Input: Personal data (height, weight, lifestyle habits, target weight, etc.)

[2029] Output: Personal data stored in a database

[2030] Step 2: Upload a photo of your meal

[2031] Users take photos of their daily meals and upload them to the application, which then sends the photo data from the user's device to a server.

[2032] Input: Food photo data

[2033] Output: Photo data sent to the server

[2034] Step 3: Analyze photos and calculate calories

[2035] The server uses TensorFlow to analyze the uploaded photos of meals and calculate the calorie content of each meal. The resulting calorie information is stored in a database and provided to the user through a user interface.

[2036] Input: Food photo data

[2037] Output: Parsed calorie information

[2038] Step 4: Exercise suggestions

[2039] Based on the user's personal data and calorie information, the server will suggest the best way to exercise, including type, frequency, or links to specific exercise videos. The suggestions will be stored in a database and provided to the user.

[2040] Input: Personal data, calorie information

[2041] Output: Exercise suggestions

[2042] Step 5: Enter emotion data

[2043] Users record their emotional state after exercise or daily using text, photos, or audio, and this emotional data is sent from the user's device to a server.

[2044] Input: Emotion data (text, photo, voice)

[2045] Output: Emotion data sent to the server

[2046] Step 6: Sentiment Analysis

[2047] The server uses an emotion engine to analyze the transmitted emotion data and classify the emotion state, which is then stored in a database.

[2048] Input: Emotion data

[2049] Output: Classified emotional state

[2050] Step 7: Generate feedback messages

[2051] The server generates an appropriate feedback message using the generative AI model based on the classified emotional state, and the generated feedback message is provided to the user through a user interface.

[2052] Input: Classified emotional state

[2053] Output: Feedback message

[2054] Step 8: Recording and analyzing user exercise data

[2055] The user performs the suggested exercise and records the results in the application. The exercise data is sent from the user's device to the server and stored in a database. The server analyzes this data and reflects it in future exercise suggestions.

[2056] Input: Exercise execution data

[2057] Output: Analysis data, improved exercise suggestions

[2058] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2059] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2060] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2061] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2062] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2063] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2064] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2065] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2066] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2067] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2068] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2069] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2070] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2071] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2072] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2073] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2074] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2075] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2076] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2077] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2078] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2079] The following is further disclosed regarding the above embodiment.

[2080] (Claim 1)

[2081] A means for users to input personal data such as height, weight, lifestyle habits, and target weight;

[2082] A way to upload photos of your meals,

[2083] means for analyzing the photograph of the meal and calculating calories;

[2084] a means for suggesting an exercise method based on the personal data and the calorie information;

[2085] means for providing the suggested exercise method to a user;

[2086] means for recording and analyzing the user's exercise performance data;

[2087] A system including:

[2088] (Claim 2)

[2089] The system of claim 1 , further comprising: means for displaying the analyzed calorie information and suggested exercise methods in a user interface.

[2090] (Claim 3)

[2091] 10. The system of claim 1, further comprising means for accumulating data collected from the user and continuously improving the generative model.

[2092] "Example 1"

[2093] (Claim 1)

[2094] A means for users to input personal data such as height, weight, age, gender, lifestyle habits, and target weight;

[2095] A way to upload photos of your meals,

[2096] means for analyzing said photograph of the meal and calculating calories using a generative AI model;

[2097] a means for suggesting an exercise method based on the personal data and the calorie information;

[2098] means for providing the suggested exercise method to a user;

[2099] means for recording the user's exercise performance data and transmitting the data to a server;

[2100] means for continuously improving the generative AI model based on the recorded data;

[2101] A system including:

[2102] (Claim 2)

[2103] The system of claim 1 , further comprising: means for displaying the analyzed calorie information and suggested exercise methods in a user interface.

[2104] (Claim 3)

[2105] The system of claim 1 , further comprising means for aggregating data collected from the user and optimizing the generative model.

[2106] "Application Example 1"

[2107] (Claim 1)

[2108] A means for users to input personal data such as height, weight, lifestyle habits, and target weight;

[2109] A way to upload photos of your meals,

[2110] means for analyzing the photograph of the meal and calculating calories;

[2111] a means for suggesting an exercise method based on the personal data and the calorie information;

[2112] A means for providing the suggested exercise method to a user and for the user to watch an exercise video;

[2113] means for recording and analyzing the user's exercise performance data;

[2114] A way for users to log in to the virtual store and record their meals and receive exercise guidance,

[2115] A system including:

[2116] (Claim 2)

[2117] 10. The system of claim 1, further comprising means for displaying the analyzed calorie information and suggested exercise methods in a user interface and for streaming exercise videos.

[2118] (Claim 3)

[2119] 10. The system of claim 1, further comprising means for accumulating data collected from the user, continuously improving the generative AI model, and reflecting the data in subsequent exercise suggestions.

[2120] "Example 2: Combining Emotion Engines"

[2121] (Claim 1)

[2122] A means for users to input personal data such as height, weight, lifestyle habits, and target weight;

[2123] A way to upload photos of your meals,

[2124] means for analyzing the photograph of the meal and calculating calories;

[2125] a means for suggesting an exercise method based on the personal data and the calorie information;

[2126] means for providing the suggested exercise method to a user;

[2127] a means for inputting user emotion data and analyzing the input emotion data;

[2128] means for providing feedback to a user based on the analyzed emotion data;

[2129] means for recording and analyzing the user's exercise performance data;

[2130] A system including:

[2131] (Claim 2)

[2132] The system of claim 1 , further comprising: means for displaying the analyzed calorie information and suggested exercise methods in a user interface.

[2133] (Claim 3)

[2134] 10. The system of claim 1, further comprising means for accumulating data collected from the user and continuously improving the generative model.

[2135] "Application example 2 when combining emotion engines"

[2136] (Claim 1)

[2137] A means for users to input personal data such as height, weight, lifestyle habits, and target weight;

[2138] A way to upload photos of your meals,

[2139] means for analyzing the photograph of the meal and calculating calories;

[2140] a means for suggesting an exercise method based on the personal data and the calorie information;

[2141] a means for inputting emotion data;

[2142] means for analyzing the input emotion data and classifying the emotion state;

[2143] means for providing the suggested exercise method to a user;

[2144] means for generating a feedback message based on said emotional state;

[2145] means for providing said feedback message to a user;

[2146] means for recording and analyzing the user's exercise performance data;

[2147] A system including:

[2148] (Claim 2)

[2149] The system of claim 1 , further comprising: means for displaying the analyzed calorie information and suggested exercise methods in a user interface.

[2150] (Claim 3)

[2151] 10. The system of claim 1, further comprising means for accumulating data collected from the user and continuously improving the generative model. [Explanation of symbols]

[2152] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to input personal data such as height, weight, lifestyle habits, and target weight; A way to upload photos of your meals, means for analyzing the photograph of the meal and calculating calories; a means for suggesting an exercise method based on the personal data and the calorie information; means for providing the suggested exercise method to a user; means for recording and analyzing the user's exercise performance data; A system including:

2. The system of claim 1 , further comprising means for displaying the analyzed calorie information and suggested exercise methods in a user interface.

3. The system of claim 1 , further comprising means for accumulating data collected from the users and continuously improving the generative model.

Images