System

Abstract

A system is provided.SOLUTION: A system, comprising: means for registering user profile information; means for searching for a cooking recipe based on the user profile information; means for optimizing a nutritional balance of the cooking recipe; means for displaying a cooking process using image generation means; means for providing optimized information to a user device; and means for suggesting a new recipe based on past user selections.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people find it difficult to incorporate healthy diets. Maintaining a proper nutritional balance is also extremely difficult when dieting, training, or improving one's beauty. Cooking, which requires time and effort, is a major challenge, especially for novice cooks and busy people. Given this background, there is a demand for a method that allows individuals to easily incorporate nutritionally balanced meals that are tailored to their individual needs. [Means for solving the problem]

[0005] This system solves the above problems by including the following means. First, it provides a means for registering user profile information and acquires the user's dietary preferences, allergy information, and health goals. Next, it provides a means for searching for cooking recipes based on this information. It also provides a means for optimizing the nutritional balance of the searched cooking recipes. It uses an image generation means to display the cooking steps and finished images to support the user's cooking process. In addition, it provides information optimized for the user's device, and is compatible with various devices such as smartphones, smartwatches, and eyeglass displays. Finally, it includes a means for suggesting new recipes based on the user's past selections. This allows users to easily enjoy nutritionally balanced meals and receive strong support for achieving their set health goals.

[0006] "User profile information" is information that describes a user's individual characteristics and preferences, including dietary preferences, allergy information, health goals, and the like.

[0007] A "cooking recipe" is a document that lists the steps and ingredients for preparing a meal.

[0008] "Nutritional balance" is a concept that refers to the appropriate distribution of nutrients necessary to maintain health.

[0009] "Image generation means" means a system or technology that uses artificial intelligence to create and provide images.

[0010] "User device" refers to various digital devices used by users, including smartphones, smartwatches, and eyeglass displays.

[0011] A "database" is a system for systematically organizing information and efficiently storing and managing it.

[0012] "Generative AI" is an artificial intelligence technology that automatically generates new information and suggestions based on a user's preferences and past choices.

[0013] "Cooking process" refers to the specific steps and tasks required to complete a dish.

[0014] "Health goal" refers to a specific purpose or task set by a user to maintain or improve their health.

[0015] "Filtering" is the process of selecting and organizing information based on certain conditions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0038] User Registration and Login

[0039] A user accesses the system and creates an account. The system receives information input from the user, such as name, email address, password, dietary preferences, allergy information, and health goals. Based on this information, an individual user profile is generated and stored in a database. After registration, the user can log in and use services based on their individual information.

[0040] Recipe Search and Filtering

[0041] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0042] Optimizing nutritional balance

[0043] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0044] Cooking process support

[0045] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0046] Multi-device compatible

[0047] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0048] Individual proposals

[0049] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0050] Specific examples

[0051] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches for "vegan" and "gluten-free" recipes based on the user profile. When the user selects one of the recipes, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, helping the user to accurately follow the steps. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's past choices, allowing the user to always enjoy new cooking experiences.

[0052] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The user accesses the system and enters the information required to create an account (name, email address, password, dietary preferences, allergy information, health goals). Once the information is complete, the user presses the "Register" button.

[0056] Step 2:

[0057] The server receives the information sent by the user, generates a user profile based on it, and stores it in a database. Once registration is complete, the server sends a confirmation message to the user.

[0058] Step 3:

[0059] The user enters login information (email address and password) and clicks the login button to access the system.

[0060] Step 4:

[0061] The server verifies the user's login information and retrieves the corresponding user profile from the database. If authentication is successful, the user is redirected to the home screen.

[0062] Step 5:

[0063] Users enter filtering criteria such as the desired food category from the search menu and press the search button. For example, they can specify criteria such as "vegan" or "gluten-free."

[0064] Step 6:

[0065] The server searches the database for matching recipes based on the user's filtering criteria and profile information, and the search results are filtered to include only recipes that meet the user's criteria.

[0066] Step 7:

[0067] The terminal displays the recipe search results acquired from the server to the user, who then selects a specific recipe from the displayed list of recipes.

[0068] Step 8:

[0069] When a user selects a recipe, the server calculates the nutritional value along with the details of the selected recipe. Once the calculation of the nutritional value is complete, the result is displayed to the user.

[0070] Step 9:

[0071] Based on the user's health goals (e.g., weight loss), the server optimizes the nutritional balance of the selected recipe, adjusting ingredient amounts as needed and generating new suggestions.

[0072] Step 10:

[0073] The device uses an image generation means to display each step of the cooking process, specifically visually showing the process in the form of images or videos, and simultaneously providing audio advice.

[0074] Step 11:

[0075] The user starts cooking, and the device displays each step in real time and provides voice instructions to guide the user, ensuring that cooking proceeds smoothly.

[0076] Step 12:

[0077] The server detects the type of device information the user uses (smartphone, smartwatch, eyeglass display) and provides information in a display format optimized for each device.

[0078] Step 13:

[0079] After the user has finished cooking, the server will suggest new recipes and ingredients based on their previous selections, for example, new vegan recipes or untried gluten-free recipes.

[0080] Step 14:

[0081] If the user accepts the new suggestion, the process of recipe search, optimization of nutritional balance, display of cooking steps, etc. is repeated again. This cycle allows the user to continuously enjoy new cooking experiences.

[0082] Example 1

[0083] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0084] Today's consumers expect easy and convenient ways to prepare healthy meals amid their busy lifestyles. However, finding nutritionally balanced recipes that fit their individual dietary preferences, allergies, and health goals, and understanding the cooking process, can be challenging. Systems that suggest new cooking experiences based on past choices are also lacking. Providing a consistent user experience across multiple devices is also a key challenge.

[0085] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0086] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for generating new recipes based on prompt sentences using a generative AI model]; and [means for providing audio advice to the user. This allows users to easily enjoy nutritionally balanced meals and receive powerful support for achieving their set health goals.

[0087] "User profile information" refers to personal information that a user registers with the system, such as name, email address, dietary preferences, allergy information, and health goals.

[0088] A "cooking recipe" is information that includes specific steps and information on ingredients for a user to cook.

[0089] "Nutrition optimization" is the process of adjusting recipe ingredients and cooking methods to meet a user's health goals and nutritional needs.

[0090] "Image generation means" refers to technology for displaying each step of the cooking process and the finished product as images or videos.

[0091] A "user device" is a device used by a user to access the system, such as a smartphone, tablet, smartwatch, or eyeglass display.

[0092] "Past user selections" refers to historical information about recipes and ingredients selected by the user using the system.

[0093] A "generative AI model" is an artificial intelligence model that takes prompt text as input and generates new recipes and suggestions.

[0094] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[0095] "Audio advice" is audio guidance provided to help the user accurately follow the cooking process.

[0096] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0097] User Registration and Login

[0098] A user accesses the system and enters information such as name, email address, password, dietary preferences, allergy information, and health goals on a dedicated sign-up page. This generates an individual profile for the user, which the server stores in a secure database. For example, user information can be managed securely using a MySQL (registered trademark) database. Next, the user enters login information on the sign-in page to access the system. The server checks the authentication information against the database and, if there is a match, authenticates the user.

[0099] Recipe Search and Filtering

[0100] The server retrieves the logged-in user's profile information from the database. For example, it retrieves the profile data using the Django framework. Next, the server searches the database for cooking recipes based on the user's profile information. This is done efficiently using SQL queries or NoSQL databases. The server then filters the retrieved recipes based on the user's allergies and dietary preferences.

[0101] Optimizing nutritional balance

[0102] The server calculates the nutritional information for the filtered recipes by summing up the nutritional data for each ingredient and displaying it. It uses a Python library to perform the calculations and optimizes the amount of ingredients used based on the user's health goals (e.g., weight loss).

[0103] Cooking process support

[0104] The device uses image generation means to display the cooking process and a finished product. For example, a mobile app using React Native displays images and videos for each step. It also uses a speech synthesis API (e.g., Google® Text-to-Speech) to provide audio advice for each step, helping the user to follow the steps accurately.

[0105] Multi-device compatible

[0106] The server collects information about the device used by the user and provides information optimized for each device. For example, it analyzes user agent information and provides the optimal interface for each device, such as a smartphone, smartwatch, or eyeglass display.

[0107] Individual proposals

[0108] The server retrieves the user's past recipe selection data from a database and uses it to suggest new recipes and ingredients. Furthermore, it uses a generative AI model to generate new recipes based on the prompt. For example, it uses a generative AI model from OpenAI (registered trademark) to generate new recipes and suggest them to the user.

[0109] Specific examples

[0110] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches and filters "vegan" and "gluten-free" recipes based on the user profile. When the user selects one recipe from the list, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step in sequence with images and audio guidance to help the user follow the steps accurately. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's previous selections, allowing the user to enjoy constantly new cooking experiences.

[0111] Example prompts for generative AI models

[0112] "Please suggest vegan and gluten-free recipes for those looking to lose weight."

[0113] With this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their health goals.

[0114] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0115] Step 1:

[0116] A user accesses the system and enters the required information (name, email address, password, dietary preferences, allergy information, health goals, etc.) on the sign-up page. The entered information is sent to the server by clicking the submit button. (Input) User information. (Output) Creation of a user profile.

[0117] Step 2:

[0118] The server generates an individual user profile based on the received user information and stores it in a secure database. For example, a MySQL database is used. This database stores all information about registered users. (Input) User information. (Output) User profile stored in a secure database.

[0119] Step 3:

[0120] A user logs in by entering their email address and password on the sign-in page. (Input) Email address and password. (Output) Login request.

[0121] Step 4:

[0122] The server retrieves the user's information from the database and compares it with the entered password. If they match, the user is authenticated and the login is successful. (Input) Login information and database information. (Output) Authentication result.

[0123] Step 5:

[0124] The server retrieves the profile information of the logged-in user from the database. For example, it retrieves the profile data using the Django framework. (Input) User ID. (Output) Profile information.

[0125] Step 6:

[0126] The server searches the database for recipes that match the user's food preferences based on the user's profile information. The search is performed efficiently using SQL queries and NoSQL databases. (Input) User profile information. (Output) Unfiltered list of recipes.

[0127] Step 7:

[0128] The server filters the retrieved recipes based on the user's allergy information and health goals. For example, it extracts only vegan and gluten-free recipes. (Input) Allergy information and health goals. (Output) Filtered list of cooking recipes.

[0129] Step 8:

[0130] The server calculates the nutritional information for each ingredient in the filtered recipe and optimizes it to meet the user's health goals. It uses a Python library to aggregate the nutritional information and adjust the ingredient amounts as needed. (Input) Filtered cooking recipe. (Output) Optimized nutritional information and ingredient amounts.

[0131] Step 9:

[0132] Based on the recipe selected by the user, the device uses image generation means to display each cooking step and a finished image. A mobile app using React Native displays images and videos corresponding to each step. (Input) Selected cooking recipe. (Output) Display of cooking steps using images and videos.

[0133] Step 10:

[0134] The device uses a speech synthesis API (such as Google Text-to-Speech) to provide audio advice for each step. (Input) Cooking process information. (Output) Audio advice.

[0135] Step 11:

[0136] The server collects information about the device used by the user and provides information optimized for each device. It analyzes the user agent information and generates different interfaces for smartphones, smartwatches, etc. (Input) Device information. (Output) Optimized interface.

[0137] Step 12:

[0138] The server retrieves the user's past recipe selection data from the database and uses this data to suggest new recipes and ingredients. (Input) Past selection data. (Output) New recipe suggestions.

[0139] Step 13:

[0140] The server uses a generative AI model to generate a new recipe based on the prompt. The generated recipe is displayed optimized for the user's profile information. (Input) Prompt and user profile information. (Output) The generated new recipe.

[0141] For example, when the prompt "Please suggest vegan and gluten-free recipes for weight loss" is entered, the server uses a generative AI model to generate appropriate recipes and suggest them to the user.

[0142] (Application example 1)

[0143] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0144] In recent years, with the rise in health consciousness, a nutritionally balanced diet has become increasingly important. However, it is difficult to consistently practice this diet due to the time-consuming nature of busy lives. While there are many dietary management applications available, few provide recipes tailored to individual user preferences, allergies, and other factors. Furthermore, it is often difficult to find the time to actually make the suggested recipes. There is a need for a system that can solve these problems and support healthy eating habits.

[0145] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0146] In this invention, the server includes means for registering user profile information, means for searching for cooking recipes based on the user profile information, means for optimizing the nutritional balance of the cooking recipes, means for displaying cooking steps using an image generation means, means for providing the optimized information to a user device, means for suggesting new recipes based on past user selections, and means for enabling the suggested dishes to be instantly ordered via a delivery service, thereby enabling users to easily create and order nutritionally balanced meals tailored to their individual health goals.

[0147] "User profile information" refers to data that includes personal information about a user, such as dietary preferences, allergy information, and health goals.

[0148] A "cooking recipe" is information that shows the ingredients, steps, and processes required to make a certain dish.

[0149] "Nutritional balance" refers to the appropriate proportions of each nutrient (calories, protein, fat, carbohydrates, vitamins, etc.) based on specific health goals and dietary restrictions.

[0150] "Image generation means" refers to a technical means for displaying the cooking process and the finished product as images or videos.

[0151] "User device" refers to a terminal used by a user, such as a smartphone, smart glasses, or head-mounted display.

[0152] "Past user selections" refers to data on recipes and ingredients that the user has previously selected or used.

[0153] A "delivery service" is a service that delivers food ordered by a user to a specified location.

[0154] A "server" is a computer system that performs various processes such as managing user profile information, searching for recipes, and optimizing nutritional balance.

[0155] The present invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system functions in cooperation with a server, a terminal, and a user device. First, a specific embodiment will be described.

[0156] User Registration and Login

[0157] A user accesses the system and creates an account. The server receives information input from the user, registering their name, email address, password, dietary preferences, allergy information, health goals, etc. Based on this information, an individual user profile is generated and stored in the database. After registering, the user can log in and use services based on their individual information.

[0158] Recipe Search and Filtering

[0159] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0160] Optimizing nutritional balance

[0161] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0162] Cooking process support

[0163] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help the user follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0164] Delivery collaboration

[0165] The server provides a function that allows users to instantly order the suggested dishes through a delivery service, enabling users to consume healthy meals with minimal effort based on optimized recipes.

[0166] Multi-device compatible

[0167] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0168] Individual proposals

[0169] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0170] Examples and prompts

[0171] For example, if the user is "vegan" and "looking to lose weight," they might send a prompt to the generative AI like this:

[0172] User Profile:

[0173] Food preference: Vegan

[0174] Health goal: weight loss

[0175] Generate proposals:

[0176] Please suggest healthy vegan recipes that fit this profile, preferably ones that are nutritionally balanced and support weight loss.

[0177] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals, with the aim of improving the quality of their meals as part of their health management.

[0178] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0179] Step 1: User Registration

[0180] When a user creates a new account, the server receives the user's name, email address, password, dietary preferences, allergy information, health goals, etc. A user profile is generated based on this information and stored in a database. The input is the user's personal information and dietary settings, and the output is a notification that the user profile has been registered.

[0181] Step 2: Log in

[0182] A user enters an email address and password on a login screen. The server receives this and checks it against an existing user profile in its database. If there is a match, the login is successful and the user is granted access to their account. The input is the user's authentication information, and the output is a notification of login success or failure.

[0183] Step 3: Find a recipe

[0184] After logging in, the server searches the database for cooking recipes based on the user profile information. It filters recipes that match the user's dietary preferences and allergy information and presents them to the user. The input is the user profile information, and the output is the filtered list of recipes.

[0185] Step 4: Optimize your nutritional balance

[0186] The server calculates nutritional information for the selected recipe and suggests optimal nutritional balance by adjusting ingredients and portions based on the user's health goals (e.g., weight loss). The input is the selected recipe and the user's health goals, and the output is the optimized nutritional information and suggested recipe modifications.

[0187] Step 5: Show the cooking process

[0188] The device uses an image generation means to display the cooking process and the finished product. Images, videos, and even audio guidance help the user follow the steps accurately. The input is optimized recipe information, and the output is displayed images and videos of the cooking process steps.

[0189] Step 6: Order delivery

[0190] The server allows the user to order the selected recipe via a delivery service. Once the user confirms the order, the server sends the order information via the delivery service's API and processes the food for delivery. The input is the user's order information, and the output is a delivery order confirmation and tracking information.

[0191] Step 7: Multi-device compatibility

[0192] The server provides information optimized for the type of device the user is using, providing a consistent user experience across different devices such as smartphones, smartwatches, and eyeglass displays. The input is device information and user profile information, and the output is display information optimized for the device.

[0193] Step 8: Personalized recipe suggestions

[0194] The server uses a generative AI model based on past user selection data and current profile information to automatically generate new recipe and ingredient suggestions. For example, it can suggest the most popular recipes over a certain period of time or new, untried recipes to the user. The input is the user's past selection data and profile information, and the output is individually generated new recipe suggestions.

[0195] Specifically, the server sends the prompt "User profile: Food preference: Vegan, Health goal: Weight loss" to the generative AI model, which then suggests healthy vegan recipes.

[0196] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0197] This invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy meal. This system can further improve the user's dining experience by combining it with an emotion engine that recognizes the user's emotions. Detailed embodiments of the system are described below.

[0198] User Registration and Login

[0199] A user accesses the system and enters information to create an account (such as name, email address, password, dietary preferences, allergy information, and health goals). Once the information is complete, the user presses the "Register" button. The server receives the information sent by the user, generates a user profile based on that information, and stores it in a database. The user can then log in and use services based on their individual information.

[0200] Recipe Search and Filtering

[0201] The server searches a database for recipes based on the user's profile information, and the resulting recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0202] Optimizing nutritional balance

[0203] The server calculates the nutritional information for the searched recipe. If the goal is weight loss, it will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0204] Cooking process support

[0205] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0206] Multi-device compatible

[0207] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[0208] Individual proposals

[0209] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0210] Use of emotion engine

[0211] The app is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and vocal tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice based on that information.

[0212] Specifically, if the user is feeling stressed, the emotion engine will suggest recipes and steps that have a relaxing effect. For example, it will suggest recipes that are easy to make in a short time and contain ingredients that will improve the user's mood. In addition, the emotion engine will monitor the user's emotions while cooking and provide appropriate voice advice and encouraging messages.

[0213] Specific examples

[0214] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." The user logs into the system, and the server searches for vegan and gluten-free recipes based on the user profile. The user then selects a recipe, and the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, allowing the user to follow the steps accurately. The server also provides an optimized display for the user's device (e.g., a smartwatch).

[0215] Furthermore, the emotion engine can recognize the user's emotions and, for example, if the user is feeling stressed, it can suggest dishes that have a relaxing effect. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages to help the user enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also gain emotional satisfaction in the process.

[0216] The processing flow will be explained below.

[0217] Step 1:

[0218] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals to create an account. The user presses the "Register" button.

[0219] Step 2:

[0220] The server receives the information sent by the user, generates a profile based on this information, stores it in a database, and sends the user a registration confirmation message.

[0221] Step 3:

[0222] A user enters their email address and password and presses the login button to access the system. The server verifies the login information and loads the profile information.

[0223] Step 4:

[0224] The server searches the database for suitable recipes based on the user profile, filters the recipes that match the user's criteria, and sends the results to the device.

[0225] Step 5:

[0226] The device displays a filtered list of recipes, and the user selects one from the displayed list.

[0227] Step 6:

[0228] After a user selects a recipe, the server calculates the recipe's details and nutritional balance, and then makes recommendations based on the results to help with health goals such as weight loss.

[0229] Step 7:

[0230] The server sends the optimized recipe information to the device, which then displays it to the user, showing each step of the recipe with images and videos, and providing audio advice.

[0231] Step 8:

[0232] The emotion engine analyzes the user's facial expressions and voice tone to recognize the user's emotional state. The server receives the data from the emotion engine and determines the user's current emotional state.

[0233] Step 9:

[0234] The server uses an emotion engine to suggest recipes and ingredients that suit the user's emotional state. For example, if the user is feeling stressed, it suggests dishes that have a relaxing effect.

[0235] Step 10:

[0236] The device starts cooking according to the recipe selected by the user, and displays each step in real time, while the emotion engine monitors the user's emotional state.

[0237] Step 11:

[0238] Based on the data from the emotion engine, the device provides appropriate voice advice and encouraging messages to the user while cooking. The content and tone of the advice are adjusted according to the user's emotional state.

[0239] Step 12:

[0240] After the user has finished cooking, the server will suggest new recipes and ingredients based on the user's past selection data, taking into account the user's preferences and emotional state.

[0241] Step 13:

[0242] If the user accepts the new suggestion, the process of recipe search, nutritional balance optimization, process display, and support from the emotion engine is repeated again. This cycle allows users to continuously enjoy new cooking experiences and nutritionally balanced meals.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] In today's busy lifestyles, it is difficult for users to maintain a healthy diet. It is also difficult to select individual meals and manage nutritional balance, and existing systems do not adequately support recipe selection and cooking according to emotional states. Therefore, there is a need for a system that can effectively improve users' eating habits and provide meal suggestions and cooking support according to their emotions.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0247] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; and [means for recognizing the user's emotions using an emotion engine and adjusting the content of cooking advice.] This not only enables users to easily select and cook individual, nutritionally balanced meals, but also allows them to receive appropriate support according to their emotional state.

[0248] "User profile information" refers to personal information such as dietary preferences, allergy information, and health goals that a user registers in the system.

[0249] A "server" is a computer system that processes the entire system and manages data, and is a device that plays a role in executing various means.

[0250] A "cooking recipe" is information that describes the steps and ingredients for making a particular dish.

[0251] "Nutritional balance" refers to the proper distribution of nutrients such as calories, protein, fat, and carbohydrates.

[0252] "Image generation means" refers to a technology for generating images of cooking steps and finished recipes, and visually presenting them to the user.

[0253] "User Device" refers to the equipment used by a user to access the system, including smartphones, smartwatches, eyeglass displays, etc.

[0254] "Past user selections" refers to historical data of recipes and ingredients selected by the user in the past.

[0255] An "emotion engine" is a technology that analyzes a user's facial expressions and vocal tone to recognize their emotional state.

[0256] "Cooking advice" refers to information that guides or instructs the user when cooking.

[0257] MODE FOR CARRYING OUT THE INVENTION

[0258] This invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. In particular, by combining it with an emotion engine that recognizes the user's emotions, the user's dining experience can be further improved. Detailed embodiments of the system are described below.

[0259] User Registration and Login

[0260] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals. When the user presses the "Register" button, the device sends the entered information to the server. The server generates a user profile based on the received information and stores it in a database. The user can then log in and use individual services.

[0261] Recipe Search and Filtering

[0262] The server searches the database for recipes based on the user's profile information. For example, if the user is vegan and prefers gluten-free meals, it will retrieve only recipes that match those criteria.

[0263] Optimizing nutritional balance

[0264] The server calculates the nutritional information for the searched recipe, examining each nutrient component, such as calories, protein, fat, and carbohydrates. Based on the user's health goals (e.g., weight loss), it adjusts the amounts of ingredients and suggests optimal nutritional balance. For example, it can adjust the amounts of certain ingredients to reduce calorie intake.

[0265] Cooking process support

[0266] The device uses an image generation means to display the cooking process and a finished product. Images and videos are displayed for each step, and audio advice is also provided. This allows even beginners and busy users to complete the dish with confidence.

[0267] Multi-device compatible

[0268] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[0269] Individual proposals

[0270] The server uses a generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current user profile. For example, it can suggest recipes that the user has chosen most over a certain period of time, or new recipes that the user has not yet tried.

[0271] Use of emotion engine

[0272] The device is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and voice tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice content based on that information. Specifically, if the user is feeling stressed, it can suggest recipes and steps that have a relaxing effect. For example, it can suggest relaxing dishes that can be made easily and quickly, or recipes that include ingredients that will lift the user's mood. In addition, the emotion engine monitors the user's emotions while cooking and provides appropriate voice advice and encouraging messages.

[0273] Specific examples

[0274] For example, consider a user named "User123" who prefers vegan and gluten-free diets and is trying to lose weight. After logging in to the system, the server searches for vegan and gluten-free recipes based on the user profile. After the user selects a recipe, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device supports the user's cooking process by displaying images and providing audio guidance. Furthermore, the emotion engine recognizes the user's emotions and, for example, can suggest dishes with a relaxing effect if the user is feeling stressed. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages, helping the user to enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also feel emotionally fulfilled in the process.

[0275] Example prompts for generative AI models

[0276] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[0277] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0278] Step 1:

[0279] User Registration

[0280] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals.

[0281] The terminal transmits the input information to the server.

[0282] The server generates a user profile based on the received information and stores it in a database.

[0283] Input: User information (name, email address, password, food preferences, allergy information, health goals)

[0284] Output: User profile stored in the database

[0285] Step 2:

[0286] User Login

[0287] The user logs in using the email address and password they registered.

[0288] The server verifies the credentials and retrieves the user profile.

[0289] Input: Email address, Password

[0290] Output: Authentication result, user profile

[0291] Step 3:

[0292] Recipe Search and Filtering

[0293] The server searches the database for cooking recipes based on the user profile information.

[0294] Extract only recipes that match the user's dietary preferences and allergy information.

[0295] Input: User profile information

[0296] Output: Filtered recipe list

[0297] Step 4:

[0298] Optimizing nutritional balance

[0299] The server calculates the nutritional information for the retrieved recipe.

[0300] Based on the user's health goals (e.g., weight loss), it adjusts ingredient amounts and suggests optimal nutritional balance.

[0301] Input: filtered list of recipes, user's health goals

[0302] Output: Nutritionally balanced recipes

[0303] Step 5:

[0304] Cooking process support

[0305] The terminal uses an image generating means to display the cooking process and a finished image.

[0306] Each step is accompanied by images and videos of the process, and audio guidance is also provided.

[0307] Input: Nutritionally balanced recipe

[0308] Output: Recipe process images, videos, audio advice

[0309] Step 6:

[0310] Multi-device compatible

[0311] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc.

[0312] Input: User's device information

[0313] Output: Information display optimized for each device

[0314] Step 7:

[0315] Individual proposal generation

[0316] The server uses the generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current profile.

[0317] Input: Past user selection data, user profile

[0318] Output: New recipe and ingredient suggestions

[0319] Step 8:

[0320] Use of emotion engine

[0321] The emotion engine recognizes the user's emotions, for example by analyzing the user's facial expressions and tone of voice.

[0322] The server collects the user's emotional state in real time and adjusts recipes and cooking advice based on that information.

[0323] Input: User's facial expression data, voice tone data

[0324] Output: User's emotional state, recipes and advice based on the emotion

[0325] Prompt Sentence Examples

[0326] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[0327] (Application example 2)

[0328] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0329] In modern society, it is difficult for users to easily enjoy healthy meals. In particular, it is complex to consider the nutritional balance and food preferences of each individual user, and the cooking process is difficult, which places a heavy burden on beginners and busy users. Furthermore, food choices and motivation to cook are often influenced by stress and emotional fluctuations, so a system that can simultaneously solve these issues is needed.

[0330] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0331] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for providing cooking advice based on the user's emotional state using an emotion engine that recognizes the user's emotions]; and [means for providing audio advice and encouraging messages to the user in cooperation with a smartphone, smart glasses, or smart watch.] This allows users to receive personalized meal suggestions and support for cooking steps, and encouragement and advice according to their emotional state, making it easy to enjoy healthy, nutritionally balanced meals.

[0332] "User profile information" is data about an individual user, such as dietary preferences, allergy information, and health goals.

[0333] The "means for searching for recipes" is a function for extracting optimal recipes based on user profile information.

[0334] The "means for optimizing the nutritional balance of a cooking recipe" is a function for adjusting the nutritional components of a cooking recipe in accordance with the user's health goals, thereby providing an optimal nutritional balance.

[0335] The "means for displaying cooking steps using image generation means" is a function for visually displaying each cooking step and the finished product, using images and videos.

[0336] "Means for providing information optimized for user devices" refers to a function that provides information in a format that is optimal for various devices used by users (smartphones, smart glasses, smart watches, etc.).

[0337] The "means for proposing new recipes based on past user selections" is a function that analyzes the user's past selection history and appropriately suggests new cooking recipes.

[0338] An "emotion engine" is a technology for recognizing and analyzing a user's emotional state, primarily by analyzing facial expressions and vocal tone.

[0339] The "means for providing cooking advice" is a function for providing appropriate advice and guidance to the user while cooking.

[0340] "Smartphones, smart glasses, and smart watches" are devices that users use to receive information, and each can provide data in different formats.

[0341] "Voice advice" is a function that provides instructions and advice to the user through voice.

[0342] An "encouraging message" is a message intended to encourage and motivate the user.

[0343] This invention is a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system registers user profile information, searches for cooking recipes, optimizes the nutritional balance of those recipes, and displays the cooking steps to support cooking. It also provides optimized information to the user's device and uses an emotion engine to provide advice based on the user's emotional state.

[0344] The server first registers user profile information. The user enters their dietary preferences, allergy information, health goals, etc., which are then saved in a database. The server then searches for the most suitable cooking recipe based on the user profile information and provides it to the user. After a recipe is selected, the server optimizes the nutritional balance of the recipe and makes suggestions that meet the user's health goals.

[0345] Furthermore, the device displays each cooking step and a picture of the finished product using images and audio. This information is optimized for the device used by the user, such as a smartphone, smart glasses, or smart watch. At this point, the cooking steps are displayed visually in an easy-to-understand manner using an image generation means, making cooking easy even for beginners or busy users.

[0346] The server also analyzes the user's past selection history and suggests new recipes, using a generative AI model to generate prompts and suggest optimal dishes based on the user's past selection data and current profile.

[0347] The emotion engine recognizes the user's emotional state and monitors it in real time while cooking. For example, if the user is feeling stressed, the emotion engine will provide relaxing dishes or encouraging messages. The smartphone, smart glasses, and smartwatch work together to provide voice advice, helping users to enjoy cooking more.

[0348] As a specific example, let's say a user is vegan, prefers gluten-free meals, and is trying to lose weight. In this case, the user's profile information is registered, and the server searches for and optimizes vegan and gluten-free recipes. If the user is feeling stressed, the server will suggest recipes with a relaxing effect, such as "banana smoothie" or "salad," and while cooking, it will provide encouraging voice messages such as "It's okay, it's hard, but you're almost there!" This allows users to easily enjoy healthy, nutritionally balanced meals.

[0349] An example prompt is:

[0350] "User profile: Vegan, gluten-free, goal: weight loss, current emotion: stressed. Generate a meal plan that considers these preferences and goals, and includes relaxation-inducing recipes."

[0351] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0352] Step 1: Register your user profile information

[0353] The server receives profile information from the user, such as dietary preferences, allergy information, health goals, etc. Based on this input information, the server registers and saves the user's profile in a database.

[0354] Input: User's dietary preferences, allergy information, health goals

[0355] Data processing and calculation: Formatting this information so that it can be stored in a database

[0356] Output: User profile saved message

[0357] Step 2: Find a recipe

[0358] The server references the user profile information, searches the database for relevant recipes, and filters out only those that match the user's preferences and allergy information.

[0359] Input: User profile information

[0360] Data processing and calculation: Execute a database query to retrieve recipes that match the criteria

[0361] Output: A list of matching recipes

[0362] Step 3: Optimize your nutritional balance

[0363] The server calculates the nutritional information of the searched recipes and optimizes them to meet the user's health goals, suggesting optimal nutritional balance by adjusting the amounts of ingredients used, etc.

[0364] Input: list of cooking recipes, user's health goals

[0365] Data processing and calculation: Analysis and adjustment of nutritional components

[0366] Output: Optimized nutritional balance recipe

[0367] Step 4: Show the cooking process

[0368] The device displays each step of the optimized recipe received from the server using images, videos, and audio, and is optimized to display appropriately regardless of the device the user is using.

[0369] Input: Optimized nutritional balance recipe

[0370] Data processing and calculation: Image generation and audio data distribution preparation

[0371] Output: Images, videos, and audio showing the cooking process

[0372] Step 5: Recognizing user emotions

[0373] The device analyzes the user's facial expressions and voice tone and uses an emotion engine to recognize the user's emotions in real time.

[0374] Input: User's face photo, voice data

[0375] Data processing and calculation: Extraction of emotional states using emotion analysis algorithms

[0376] Output: User's emotional state (e.g., stress, joy, sadness)

[0377] Step 6: Offering emotionally-based advice

[0378] The server provides the user with appropriate cooking recipes and cooking advice based on the user's emotional state obtained from the emotion engine.

[0379] Input: User emotional state, optimized recipe, user profile information

[0380] Data processing and calculation: Analysis of past selection data and generation of prompt sentences using a generative AI model

[0381] Output: Emotion-based cooking recipes and cooking advice

[0382] Step 7: Providing audio advice and encouraging messages

[0383] The device receives voice advice and encouraging messages from the server and provides them to the user. This can be a smartphone, smart glasses, or smart watch.

[0384] Input: Audio advice and encouraging messages sent from the server

[0385] Data processing and calculation: Preparing audio data for device distribution

[0386] Output: Playback of voice advice and encouraging messages

[0387] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0388] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0389] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0390] [Second embodiment]

[0391] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0392] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0393] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0394] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0395] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0396] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0397] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0398] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0399] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0400] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0401] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0402] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0403] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0404] User Registration and Login

[0405] A user accesses the system and creates an account. The system receives information input from the user, such as name, email address, password, dietary preferences, allergy information, and health goals. Based on this information, an individual user profile is generated and stored in a database. After registration, the user can log in and use services based on their individual information.

[0406] Recipe Search and Filtering

[0407] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0408] Optimizing nutritional balance

[0409] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0410] Cooking process support

[0411] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0412] Multi-device compatible

[0413] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0414] Individual proposals

[0415] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0416] Specific examples

[0417] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches for "vegan" and "gluten-free" recipes based on the user profile. When the user selects one of the recipes, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, helping the user to accurately follow the steps. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's past choices, allowing the user to always enjoy new cooking experiences.

[0418] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals.

[0419] The processing flow will be explained below.

[0420] Step 1:

[0421] The user accesses the system and enters the information required to create an account (name, email address, password, dietary preferences, allergy information, health goals). Once the information is complete, the user presses the "Register" button.

[0422] Step 2:

[0423] The server receives the information sent by the user, generates a user profile based on it, and stores it in a database. Once registration is complete, the server sends a confirmation message to the user.

[0424] Step 3:

[0425] The user enters login information (email address and password) and clicks the login button to access the system.

[0426] Step 4:

[0427] The server verifies the user's login information and retrieves the corresponding user profile from the database. If authentication is successful, the user is redirected to the home screen.

[0428] Step 5:

[0429] Users enter filtering criteria such as the desired food category from the search menu and press the search button. For example, they can specify criteria such as "vegan" or "gluten-free."

[0430] Step 6:

[0431] The server searches the database for matching recipes based on the user's filtering criteria and profile information, and the search results are filtered to include only recipes that meet the user's criteria.

[0432] Step 7:

[0433] The terminal displays the recipe search results acquired from the server to the user, who then selects a specific recipe from the displayed list of recipes.

[0434] Step 8:

[0435] When a user selects a recipe, the server calculates the nutritional value along with the details of the selected recipe. Once the calculation of the nutritional value is complete, the result is displayed to the user.

[0436] Step 9:

[0437] Based on the user's health goals (e.g., weight loss), the server optimizes the nutritional balance of the selected recipe, adjusting ingredient amounts as needed and generating new suggestions.

[0438] Step 10:

[0439] The device uses an image generation means to display each step of the cooking process, specifically visually showing the process in the form of images or videos, and simultaneously providing audio advice.

[0440] Step 11:

[0441] The user starts cooking, and the device displays each step in real time and provides voice instructions to guide the user, ensuring that cooking proceeds smoothly.

[0442] Step 12:

[0443] The server detects the type of device information the user uses (smartphone, smartwatch, eyeglass display) and provides information in a display format optimized for each device.

[0444] Step 13:

[0445] After the user has finished cooking, the server will suggest new recipes and ingredients based on their previous selections, for example, new vegan recipes or untried gluten-free recipes.

[0446] Step 14:

[0447] If the user accepts the new suggestion, the process of recipe search, optimization of nutritional balance, display of cooking steps, etc. is repeated again. This cycle allows the user to continuously enjoy new cooking experiences.

[0448] Example 1

[0449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0450] Today's consumers expect easy and convenient ways to prepare healthy meals amid their busy lifestyles. However, finding nutritionally balanced recipes that fit their individual dietary preferences, allergies, and health goals, and understanding the cooking process, can be challenging. Systems that suggest new cooking experiences based on past choices are also lacking. Providing a consistent user experience across multiple devices is also a key challenge.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0452] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for generating new recipes based on prompt sentences using a generative AI model]; and [means for providing audio advice to the user. This allows users to easily enjoy nutritionally balanced meals and receive powerful support for achieving their set health goals.

[0453] "User profile information" refers to personal information that a user registers with the system, such as name, email address, dietary preferences, allergy information, and health goals.

[0454] A "cooking recipe" is information that includes specific steps and information on ingredients for a user to cook.

[0455] "Nutrition optimization" is the process of adjusting recipe ingredients and cooking methods to meet a user's health goals and nutritional needs.

[0456] "Image generation means" refers to technology for displaying each step of the cooking process and the finished product as images or videos.

[0457] A "user device" is a device used by a user to access the system, such as a smartphone, tablet, smartwatch, or eyeglass display.

[0458] "Past user selections" refers to historical information about recipes and ingredients selected by the user using the system.

[0459] A "generative AI model" is an artificial intelligence model that takes prompt text as input and generates new recipes and suggestions.

[0460] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[0461] "Audio advice" is audio guidance provided to help the user accurately follow the cooking process.

[0462] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0463] User Registration and Login

[0464] A user accesses the system and enters information such as their name, email address, password, dietary preferences, allergy information, and health goals on a dedicated sign-up page. This generates an individual user profile, which the server stores in a secure database. For example, a MySQL database can be used to securely manage user information. Next, the user enters their login information on the sign-in page to access the system. The server checks the authentication information against the database and, if there is a match, authenticates the user.

[0465] Recipe Search and Filtering

[0466] The server retrieves the logged-in user's profile information from the database. For example, it retrieves the profile data using the Django framework. Next, the server searches the database for cooking recipes based on the user's profile information. This is done efficiently using SQL queries or NoSQL databases. The server then filters the retrieved recipes based on the user's allergies and dietary preferences.

[0467] Optimizing nutritional balance

[0468] The server calculates the nutritional information for the filtered recipes by summing up the nutritional data for each ingredient and displaying it. It uses a Python library to perform the calculations and optimizes the amount of ingredients used based on the user's health goals (e.g., weight loss).

[0469] Cooking process support

[0470] The device uses image generation tools to display the cooking process and the finished product. For example, a mobile app using React Native displays images and videos for each step. It also uses a speech synthesis API (e.g., Google Text-to-Speech) to provide audio guidance for each step, helping the user to follow the steps accurately.

[0471] Multi-device compatible

[0472] The server collects information about the device used by the user and provides information optimized for each device. For example, it analyzes user agent information and provides the optimal interface for each device, such as a smartphone, smartwatch, or eyeglass display.

[0473] Individual proposals

[0474] The server retrieves the user's past recipe selection data from a database and uses it to suggest new recipes and ingredients. Furthermore, it uses a generative AI model to generate new recipes based on the prompt. For example, it uses OpenAI's generative AI model to generate new recipes and suggest them to the user.

[0475] Specific examples

[0476] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches and filters "vegan" and "gluten-free" recipes based on the user profile. When the user selects one recipe from the list, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step in sequence with images and audio guidance to help the user follow the steps accurately. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's previous selections, allowing the user to enjoy constantly new cooking experiences.

[0477] Example prompts for generative AI models

[0478] "Please suggest vegan and gluten-free recipes for those looking to lose weight."

[0479] With this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their health goals.

[0480] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0481] Step 1:

[0482] A user accesses the system and enters the required information (name, email address, password, dietary preferences, allergy information, health goals, etc.) on the sign-up page. The entered information is sent to the server by clicking the submit button. (Input) User information. (Output) Creation of a user profile.

[0483] Step 2:

[0484] The server generates an individual user profile based on the received user information and stores it in a secure database. For example, a MySQL database is used. This database stores all information about registered users. (Input) User information. (Output) User profile stored in a secure database.

[0485] Step 3:

[0486] A user logs in by entering their email address and password on the sign-in page. (Input) Email address and password. (Output) Login request.

[0487] Step 4:

[0488] The server retrieves the user's information from the database and compares it with the entered password. If they match, the user is authenticated and the login is successful. (Input) Login information and database information. (Output) Authentication result.

[0489] Step 5:

[0490] The server retrieves the profile information of the logged-in user from the database. For example, it retrieves the profile data using the Django framework. (Input) User ID. (Output) Profile information.

[0491] Step 6:

[0492] The server searches the database for recipes that match the user's food preferences based on the user's profile information. The search is performed efficiently using SQL queries and NoSQL databases. (Input) User profile information. (Output) Unfiltered list of recipes.

[0493] Step 7:

[0494] The server filters the retrieved recipes based on the user's allergy information and health goals. For example, it extracts only vegan and gluten-free recipes. (Input) Allergy information and health goals. (Output) Filtered list of cooking recipes.

[0495] Step 8:

[0496] The server calculates the nutritional information for each ingredient in the filtered recipe and optimizes it to meet the user's health goals. It uses a Python library to aggregate the nutritional information and adjust the ingredient amounts as needed. (Input) Filtered cooking recipe. (Output) Optimized nutritional information and ingredient amounts.

[0497] Step 9:

[0498] Based on the recipe selected by the user, the device uses image generation means to display each cooking step and a finished image. A mobile app using React Native displays images and videos corresponding to each step. (Input) Selected cooking recipe. (Output) Display of cooking steps using images and videos.

[0499] Step 10:

[0500] The device uses a speech synthesis API (such as Google Text-to-Speech) to provide audio advice for each step. (Input) Cooking process information. (Output) Audio advice.

[0501] Step 11:

[0502] The server collects information about the device used by the user and provides information optimized for each device. It analyzes the user agent information and generates different interfaces for smartphones, smartwatches, etc. (Input) Device information. (Output) Optimized interface.

[0503] Step 12:

[0504] The server retrieves the user's past recipe selection data from the database and uses this data to suggest new recipes and ingredients. (Input) Past selection data. (Output) New recipe suggestions.

[0505] Step 13:

[0506] The server uses a generative AI model to generate a new recipe based on the prompt. The generated recipe is displayed optimized for the user's profile information. (Input) Prompt and user profile information. (Output) The generated new recipe.

[0507] For example, when the prompt "Please suggest vegan and gluten-free recipes for weight loss" is entered, the server uses a generative AI model to generate appropriate recipes and suggest them to the user.

[0508] (Application example 1)

[0509] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0510] In recent years, with the rise in health consciousness, a nutritionally balanced diet has become increasingly important. However, it is difficult to consistently practice this diet due to the time-consuming nature of busy lives. While there are many dietary management applications available, few provide recipes tailored to individual user preferences, allergies, and other factors. Furthermore, it is often difficult to find the time to actually make the suggested recipes. There is a need for a system that can solve these problems and support healthy eating habits.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0512] In this invention, the server includes means for registering user profile information, means for searching for cooking recipes based on the user profile information, means for optimizing the nutritional balance of the cooking recipes, means for displaying cooking steps using an image generation means, means for providing the optimized information to a user device, means for suggesting new recipes based on past user selections, and means for enabling the suggested dishes to be instantly ordered via a delivery service, thereby enabling users to easily create and order nutritionally balanced meals tailored to their individual health goals.

[0513] "User profile information" refers to data that includes personal information about a user, such as dietary preferences, allergy information, and health goals.

[0514] A "cooking recipe" is information that shows the ingredients, steps, and processes required to make a certain dish.

[0515] "Nutritional balance" refers to the appropriate proportions of each nutrient (calories, protein, fat, carbohydrates, vitamins, etc.) based on specific health goals and dietary restrictions.

[0516] "Image generation means" refers to a technical means for displaying the cooking process and the finished product as images or videos.

[0517] "User device" refers to a terminal used by a user, such as a smartphone, smart glasses, or head-mounted display.

[0518] "Past user selections" refers to data on recipes and ingredients that the user has previously selected or used.

[0519] A "delivery service" is a service that delivers food ordered by a user to a specified location.

[0520] A "server" is a computer system that performs various processes such as managing user profile information, searching for recipes, and optimizing nutritional balance.

[0521] The present invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system functions in cooperation with a server, a terminal, and a user device. First, a specific embodiment will be described.

[0522] User Registration and Login

[0523] A user accesses the system and creates an account. The server receives information input from the user, registering their name, email address, password, dietary preferences, allergy information, health goals, etc. Based on this information, an individual user profile is generated and stored in the database. After registering, the user can log in and use services based on their individual information.

[0524] Recipe Search and Filtering

[0525] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0526] Optimizing nutritional balance

[0527] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0528] Cooking process support

[0529] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help the user follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0530] Delivery collaboration

[0531] The server provides a function that allows users to instantly order the suggested dishes through a delivery service, enabling users to consume healthy meals with minimal effort based on optimized recipes.

[0532] Multi-device compatible

[0533] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0534] Individual proposals

[0535] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0536] Examples and prompts

[0537] For example, if the user is "vegan" and "looking to lose weight," they might send a prompt to the generative AI like this:

[0538] User Profile:

[0539] Food preference: Vegan

[0540] Health goal: weight loss

[0541] Generate proposals:

[0542] Please suggest healthy vegan recipes that fit this profile, preferably ones that are nutritionally balanced and support weight loss.

[0543] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals, with the aim of improving the quality of their meals as part of their health management.

[0544] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0545] Step 1: User Registration

[0546] When a user creates a new account, the server receives the user's name, email address, password, dietary preferences, allergy information, health goals, etc. A user profile is generated based on this information and stored in a database. The input is the user's personal information and dietary settings, and the output is a notification that the user profile has been registered.

[0547] Step 2: Log in

[0548] A user enters an email address and password on a login screen. The server receives this and checks it against an existing user profile in its database. If there is a match, the login is successful and the user is granted access to their account. The input is the user's authentication information, and the output is a notification of login success or failure.

[0549] Step 3: Find a recipe

[0550] After logging in, the server searches the database for cooking recipes based on the user profile information. It filters recipes that match the user's dietary preferences and allergy information and presents them to the user. The input is the user profile information, and the output is the filtered list of recipes.

[0551] Step 4: Optimize your nutritional balance

[0552] The server calculates nutritional information for the selected recipe and suggests optimal nutritional balance by adjusting ingredients and portions based on the user's health goals (e.g., weight loss). The input is the selected recipe and the user's health goals, and the output is the optimized nutritional information and suggested recipe modifications.

[0553] Step 5: Show the cooking process

[0554] The device uses an image generation means to display the cooking process and the finished product. Images, videos, and even audio guidance help the user follow the steps accurately. The input is optimized recipe information, and the output is displayed images and videos of the cooking process steps.

[0555] Step 6: Order delivery

[0556] The server allows the user to order the selected recipe via a delivery service. Once the user confirms the order, the server sends the order information via the delivery service's API and processes the food for delivery. The input is the user's order information, and the output is a delivery order confirmation and tracking information.

[0557] Step 7: Multi-device compatibility

[0558] The server provides information optimized for the type of device the user is using, providing a consistent user experience across different devices such as smartphones, smartwatches, and eyeglass displays. The input is device information and user profile information, and the output is display information optimized for the device.

[0559] Step 8: Personalized recipe suggestions

[0560] The server uses a generative AI model based on past user selection data and current profile information to automatically generate new recipe and ingredient suggestions. For example, it can suggest the most popular recipes over a certain period of time or new, untried recipes to the user. The input is the user's past selection data and profile information, and the output is individually generated new recipe suggestions.

[0561] Specifically, the server sends the prompt "User profile: Food preference: Vegan, Health goal: Weight loss" to the generative AI model, which then suggests healthy vegan recipes.

[0562] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0563] This invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy meal. This system can further improve the user's dining experience by combining it with an emotion engine that recognizes the user's emotions. Detailed embodiments of the system are described below.

[0564] User Registration and Login

[0565] A user accesses the system and enters information to create an account (such as name, email address, password, dietary preferences, allergy information, and health goals). Once the information is complete, the user presses the "Register" button. The server receives the information sent by the user, generates a user profile based on that information, and stores it in a database. The user can then log in and use services based on their individual information.

[0566] Recipe Search and Filtering

[0567] The server searches a database for recipes based on the user's profile information, and the resulting recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0568] Optimizing nutritional balance

[0569] The server calculates the nutritional information for the searched recipe. If the goal is weight loss, it will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0570] Cooking process support

[0571] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0572] Multi-device compatible

[0573] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[0574] Individual proposals

[0575] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0576] Use of emotion engine

[0577] The app is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and vocal tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice based on that information.

[0578] Specifically, if the user is feeling stressed, the emotion engine will suggest recipes and steps that have a relaxing effect. For example, it will suggest recipes that are easy to make in a short time and contain ingredients that will improve the user's mood. In addition, the emotion engine will monitor the user's emotions while cooking and provide appropriate voice advice and encouraging messages.

[0579] Specific examples

[0580] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." The user logs into the system, and the server searches for vegan and gluten-free recipes based on the user profile. The user then selects a recipe, and the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, allowing the user to follow the steps accurately. The server also provides an optimized display for the user's device (e.g., a smartwatch).

[0581] Furthermore, the emotion engine can recognize the user's emotions and, for example, if the user is feeling stressed, it can suggest dishes that have a relaxing effect. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages to help the user enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also gain emotional satisfaction in the process.

[0582] The processing flow will be explained below.

[0583] Step 1:

[0584] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals to create an account. The user presses the "Register" button.

[0585] Step 2:

[0586] The server receives the information sent by the user, generates a profile based on this information, stores it in a database, and sends the user a registration confirmation message.

[0587] Step 3:

[0588] A user enters their email address and password and presses the login button to access the system. The server verifies the login information and loads the profile information.

[0589] Step 4:

[0590] The server searches the database for suitable recipes based on the user profile, filters the recipes that match the user's criteria, and sends the results to the device.

[0591] Step 5:

[0592] The device displays a filtered list of recipes, and the user selects one from the displayed list.

[0593] Step 6:

[0594] After a user selects a recipe, the server calculates the recipe's details and nutritional balance, and then makes recommendations based on the results to help with health goals such as weight loss.

[0595] Step 7:

[0596] The server sends the optimized recipe information to the device, which then displays it to the user, showing each step of the recipe with images and videos, and providing audio advice.

[0597] Step 8:

[0598] The emotion engine analyzes the user's facial expressions and voice tone to recognize the user's emotional state. The server receives the data from the emotion engine and determines the user's current emotional state.

[0599] Step 9:

[0600] The server uses an emotion engine to suggest recipes and ingredients that suit the user's emotional state. For example, if the user is feeling stressed, it suggests dishes that have a relaxing effect.

[0601] Step 10:

[0602] The device starts cooking according to the recipe selected by the user, and displays each step in real time, while the emotion engine monitors the user's emotional state.

[0603] Step 11:

[0604] Based on the data from the emotion engine, the device provides appropriate voice advice and encouraging messages to the user while cooking. The content and tone of the advice are adjusted according to the user's emotional state.

[0605] Step 12:

[0606] After the user has finished cooking, the server will suggest new recipes and ingredients based on the user's past selection data, taking into account the user's preferences and emotional state.

[0607] Step 13:

[0608] If the user accepts the new suggestion, the process of recipe search, nutritional balance optimization, process display, and support from the emotion engine is repeated again. This cycle allows users to continuously enjoy new cooking experiences and nutritionally balanced meals.

[0609] Example 2

[0610] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0611] In today's busy lifestyles, it is difficult for users to maintain a healthy diet. It is also difficult to select individual meals and manage nutritional balance, and existing systems do not adequately support recipe selection and cooking according to emotional states. Therefore, there is a need for a system that can effectively improve users' eating habits and provide meal suggestions and cooking support according to their emotions.

[0612] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0613] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; and [means for recognizing the user's emotions using an emotion engine and adjusting the content of cooking advice.] This not only enables users to easily select and cook individual, nutritionally balanced meals, but also allows them to receive appropriate support according to their emotional state.

[0614] "User profile information" refers to personal information such as dietary preferences, allergy information, and health goals that a user registers in the system.

[0615] A "server" is a computer system that processes the entire system and manages data, and is a device that plays a role in executing various means.

[0616] A "cooking recipe" is information that describes the steps and ingredients for making a particular dish.

[0617] "Nutritional balance" refers to the proper distribution of nutrients such as calories, protein, fat, and carbohydrates.

[0618] "Image generation means" refers to a technology for generating images of cooking steps and finished recipes, and visually presenting them to the user.

[0619] "User Device" refers to the equipment used by a user to access the system, including smartphones, smartwatches, eyeglass displays, etc.

[0620] "Past user selections" refers to historical data of recipes and ingredients selected by the user in the past.

[0621] An "emotion engine" is a technology that analyzes a user's facial expressions and vocal tone to recognize their emotional state.

[0622] "Cooking advice" refers to information that guides or instructs the user when cooking.

[0623] MODE FOR CARRYING OUT THE INVENTION

[0624] This invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. In particular, by combining it with an emotion engine that recognizes the user's emotions, the user's dining experience can be further improved. Detailed embodiments of the system are described below.

[0625] User Registration and Login

[0626] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals. When the user presses the "Register" button, the device sends the entered information to the server. The server generates a user profile based on the received information and stores it in a database. The user can then log in and use individual services.

[0627] Recipe Search and Filtering

[0628] The server searches the database for recipes based on the user's profile information. For example, if the user is vegan and prefers gluten-free meals, it will retrieve only recipes that match those criteria.

[0629] Optimizing nutritional balance

[0630] The server calculates the nutritional information for the searched recipe, examining each nutrient component, such as calories, protein, fat, and carbohydrates. Based on the user's health goals (e.g., weight loss), it adjusts the amounts of ingredients and suggests optimal nutritional balance. For example, it can adjust the amounts of certain ingredients to reduce calorie intake.

[0631] Cooking process support

[0632] The device uses an image generation means to display the cooking process and a finished product. Images and videos are displayed for each step, and audio advice is also provided. This allows even beginners and busy users to complete the dish with confidence.

[0633] Multi-device compatible

[0634] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[0635] Individual proposals

[0636] The server uses a generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current user profile. For example, it can suggest recipes that the user has chosen most over a certain period of time, or new recipes that the user has not yet tried.

[0637] Use of emotion engine

[0638] The device is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and voice tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice content based on that information. Specifically, if the user is feeling stressed, it can suggest recipes and steps that have a relaxing effect. For example, it can suggest relaxing dishes that can be made easily and quickly, or recipes that include ingredients that will lift the user's mood. In addition, the emotion engine monitors the user's emotions while cooking and provides appropriate voice advice and encouraging messages.

[0639] Specific examples

[0640] For example, consider a user named "User123" who prefers vegan and gluten-free diets and is trying to lose weight. After logging in to the system, the server searches for vegan and gluten-free recipes based on the user profile. After the user selects a recipe, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device supports the user's cooking process by displaying images and providing audio guidance. Furthermore, the emotion engine recognizes the user's emotions and, for example, can suggest dishes with a relaxing effect if the user is feeling stressed. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages, helping the user to enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also feel emotionally fulfilled in the process.

[0641] Example prompts for generative AI models

[0642] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[0643] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0644] Step 1:

[0645] User Registration

[0646] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals.

[0647] The terminal transmits the input information to the server.

[0648] The server generates a user profile based on the received information and stores it in a database.

[0649] Input: User information (name, email address, password, food preferences, allergy information, health goals)

[0650] Output: User profile stored in the database

[0651] Step 2:

[0652] User Login

[0653] The user logs in using the email address and password they registered.

[0654] The server verifies the credentials and retrieves the user profile.

[0655] Input: Email address, Password

[0656] Output: Authentication result, user profile

[0657] Step 3:

[0658] Recipe Search and Filtering

[0659] The server searches the database for cooking recipes based on the user profile information.

[0660] Extract only recipes that match the user's dietary preferences and allergy information.

[0661] Input: User profile information

[0662] Output: Filtered recipe list

[0663] Step 4:

[0664] Optimizing nutritional balance

[0665] The server calculates the nutritional information for the retrieved recipe.

[0666] Based on the user's health goals (e.g., weight loss), it adjusts ingredient amounts and suggests optimal nutritional balance.

[0667] Input: filtered list of recipes, user's health goals

[0668] Output: Nutritionally balanced recipes

[0669] Step 5:

[0670] Cooking process support

[0671] The terminal uses an image generating means to display the cooking process and a finished image.

[0672] Each step is accompanied by images and videos of the process, and audio guidance is also provided.

[0673] Input: Nutritionally balanced recipe

[0674] Output: Recipe process images, videos, audio advice

[0675] Step 6:

[0676] Multi-device compatible

[0677] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc.

[0678] Input: User's device information

[0679] Output: Information display optimized for each device

[0680] Step 7:

[0681] Individual proposal generation

[0682] The server uses the generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current profile.

[0683] Input: Past user selection data, user profile

[0684] Output: New recipe and ingredient suggestions

[0685] Step 8:

[0686] Use of emotion engine

[0687] The emotion engine recognizes the user's emotions, for example by analyzing the user's facial expressions and tone of voice.

[0688] The server collects the user's emotional state in real time and adjusts recipes and cooking advice based on that information.

[0689] Input: User's facial expression data, voice tone data

[0690] Output: User's emotional state, recipes and advice based on the emotion

[0691] Prompt Sentence Examples

[0692] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[0693] (Application example 2)

[0694] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0695] In modern society, it is difficult for users to easily enjoy healthy meals. In particular, it is complex to consider the nutritional balance and food preferences of each individual user, and the cooking process is difficult, which places a heavy burden on beginners and busy users. Furthermore, food choices and motivation to cook are often influenced by stress and emotional fluctuations, so a system that can simultaneously solve these issues is needed.

[0696] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0697] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for providing cooking advice based on the user's emotional state using an emotion engine that recognizes the user's emotions]; and [means for providing audio advice and encouraging messages to the user in cooperation with a smartphone, smart glasses, or smart watch.] This allows users to receive personalized meal suggestions and support for cooking steps, and encouragement and advice according to their emotional state, making it easy to enjoy healthy, nutritionally balanced meals.

[0698] "User profile information" is data about an individual user, such as dietary preferences, allergy information, and health goals.

[0699] The "means for searching for recipes" is a function for extracting optimal recipes based on user profile information.

[0700] The "means for optimizing the nutritional balance of a cooking recipe" is a function for adjusting the nutritional components of a cooking recipe in accordance with the user's health goals, thereby providing an optimal nutritional balance.

[0701] The "means for displaying cooking steps using image generation means" is a function for visually displaying each cooking step and the finished product, using images and videos.

[0702] "Means for providing information optimized for user devices" refers to a function that provides information in a format that is optimal for various devices used by users (smartphones, smart glasses, smart watches, etc.).

[0703] The "means for proposing new recipes based on past user selections" is a function that analyzes the user's past selection history and appropriately suggests new cooking recipes.

[0704] An "emotion engine" is a technology for recognizing and analyzing a user's emotional state, primarily by analyzing facial expressions and vocal tone.

[0705] The "means for providing cooking advice" is a function for providing appropriate advice and guidance to the user while cooking.

[0706] "Smartphones, smart glasses, and smart watches" are devices that users use to receive information, and each can provide data in different formats.

[0707] "Voice advice" is a function that provides instructions and advice to the user through voice.

[0708] An "encouraging message" is a message intended to encourage and motivate the user.

[0709] This invention is a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system registers user profile information, searches for cooking recipes, optimizes the nutritional balance of those recipes, and displays the cooking steps to support cooking. It also provides optimized information to the user's device and uses an emotion engine to provide advice based on the user's emotional state.

[0710] The server first registers user profile information. The user enters their dietary preferences, allergy information, health goals, etc., which are then saved in a database. The server then searches for the most suitable cooking recipe based on the user profile information and provides it to the user. After a recipe is selected, the server optimizes the nutritional balance of the recipe and makes suggestions that meet the user's health goals.

[0711] Furthermore, the device displays each cooking step and a picture of the finished product using images and audio. This information is optimized for the device used by the user, such as a smartphone, smart glasses, or smart watch. At this point, the cooking steps are displayed visually in an easy-to-understand manner using an image generation means, making cooking easy even for beginners or busy users.

[0712] The server also analyzes the user's past selection history and suggests new recipes, using a generative AI model to generate prompts and suggest optimal dishes based on the user's past selection data and current profile.

[0713] The emotion engine recognizes the user's emotional state and monitors it in real time while cooking. For example, if the user is feeling stressed, the emotion engine will provide relaxing dishes or encouraging messages. The smartphone, smart glasses, and smartwatch work together to provide voice advice, helping users to enjoy cooking more.

[0714] As a specific example, let's say a user is vegan, prefers gluten-free meals, and is trying to lose weight. In this case, the user's profile information is registered, and the server searches for and optimizes vegan and gluten-free recipes. If the user is feeling stressed, the server will suggest recipes with a relaxing effect, such as "banana smoothie" or "salad," and while cooking, it will provide encouraging voice messages such as "It's okay, it's hard, but you're almost there!" This allows users to easily enjoy healthy, nutritionally balanced meals.

[0715] An example prompt is:

[0716] "User profile: Vegan, gluten-free, goal: weight loss, current emotion: stressed. Generate a meal plan that considers these preferences and goals, and includes relaxation-inducing recipes."

[0717] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0718] Step 1: Register your user profile information

[0719] The server receives profile information from the user, such as dietary preferences, allergy information, health goals, etc. Based on this input information, the server registers and saves the user's profile in a database.

[0720] Input: User's dietary preferences, allergy information, health goals

[0721] Data processing and calculation: Formatting this information so that it can be stored in a database

[0722] Output: User profile saved message

[0723] Step 2: Find a recipe

[0724] The server references the user profile information, searches the database for relevant recipes, and filters out only those that match the user's preferences and allergy information.

[0725] Input: User profile information

[0726] Data processing and calculation: Execute a database query to retrieve recipes that match the criteria

[0727] Output: A list of matching recipes

[0728] Step 3: Optimize your nutritional balance

[0729] The server calculates the nutritional information of the searched recipes and optimizes them to meet the user's health goals, suggesting optimal nutritional balance by adjusting the amounts of ingredients used, etc.

[0730] Input: list of cooking recipes, user's health goals

[0731] Data processing and calculation: Analysis and adjustment of nutritional components

[0732] Output: Optimized nutritional balance recipe

[0733] Step 4: Show the cooking process

[0734] The device displays each step of the optimized recipe received from the server using images, videos, and audio, and is optimized to display appropriately regardless of the device the user is using.

[0735] Input: Optimized nutritional balance recipe

[0736] Data processing and calculation: Image generation and audio data distribution preparation

[0737] Output: Images, videos, and audio showing the cooking process

[0738] Step 5: Recognizing user emotions

[0739] The device analyzes the user's facial expressions and voice tone and uses an emotion engine to recognize the user's emotions in real time.

[0740] Input: User's face photo, voice data

[0741] Data processing and calculation: Extraction of emotional states using emotion analysis algorithms

[0742] Output: User's emotional state (e.g., stress, joy, sadness)

[0743] Step 6: Offering emotionally-based advice

[0744] The server provides the user with appropriate cooking recipes and cooking advice based on the user's emotional state obtained from the emotion engine.

[0745] Input: User emotional state, optimized recipe, user profile information

[0746] Data processing and calculation: Analysis of past selection data and generation of prompt sentences using a generative AI model

[0747] Output: Emotion-based cooking recipes and cooking advice

[0748] Step 7: Providing audio advice and encouraging messages

[0749] The device receives voice advice and encouraging messages from the server and provides them to the user. This can be a smartphone, smart glasses, or smart watch.

[0750] Input: Audio advice and encouraging messages sent from the server

[0751] Data processing and calculation: Preparing audio data for device distribution

[0752] Output: Playback of voice advice and encouraging messages

[0753] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0754] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0755] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0756] [Third embodiment]

[0757] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0758] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0759] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0760] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0761] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0762] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0763] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0764] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0765] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0766] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0767] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0768] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0769] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0770] User Registration and Login

[0771] A user accesses the system and creates an account. The system receives information input from the user, such as name, email address, password, dietary preferences, allergy information, and health goals. Based on this information, an individual user profile is generated and stored in a database. After registration, the user can log in and use services based on their individual information.

[0772] Recipe Search and Filtering

[0773] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0774] Optimizing nutritional balance

[0775] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0776] Cooking process support

[0777] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0778] Multi-device compatible

[0779] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0780] Individual proposals

[0781] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0782] Specific examples

[0783] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches for "vegan" and "gluten-free" recipes based on the user profile. When the user selects one of the recipes, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, helping the user to accurately follow the steps. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's past choices, allowing the user to always enjoy new cooking experiences.

[0784] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals.

[0785] The processing flow will be explained below.

[0786] Step 1:

[0787] The user accesses the system and enters the information required to create an account (name, email address, password, dietary preferences, allergy information, health goals). Once the information is complete, the user presses the "Register" button.

[0788] Step 2:

[0789] The server receives the information sent by the user, generates a user profile based on it, and stores it in a database. Once registration is complete, the server sends a confirmation message to the user.

[0790] Step 3:

[0791] The user enters login information (email address and password) and clicks the login button to access the system.

[0792] Step 4:

[0793] The server verifies the user's login information and retrieves the corresponding user profile from the database. If authentication is successful, the user is redirected to the home screen.

[0794] Step 5:

[0795] Users enter filtering criteria such as the desired food category from the search menu and press the search button. For example, they can specify criteria such as "vegan" or "gluten-free."

[0796] Step 6:

[0797] The server searches the database for matching recipes based on the user's filtering criteria and profile information, and the search results are filtered to include only recipes that meet the user's criteria.

[0798] Step 7:

[0799] The terminal displays the recipe search results acquired from the server to the user, who then selects a specific recipe from the displayed list of recipes.

[0800] Step 8:

[0801] When a user selects a recipe, the server calculates the nutritional value along with the details of the selected recipe. Once the calculation of the nutritional value is complete, the result is displayed to the user.

[0802] Step 9:

[0803] Based on the user's health goals (e.g., weight loss), the server optimizes the nutritional balance of the selected recipe, adjusting ingredient amounts as needed and generating new suggestions.

[0804] Step 10:

[0805] The device uses an image generation means to display each step of the cooking process, specifically visually showing the process in the form of images or videos, and simultaneously providing audio advice.

[0806] Step 11:

[0807] The user starts cooking, and the device displays each step in real time and provides voice instructions to guide the user, ensuring that cooking proceeds smoothly.

[0808] Step 12:

[0809] The server detects the type of device information the user uses (smartphone, smartwatch, eyeglass display) and provides information in a display format optimized for each device.

[0810] Step 13:

[0811] After the user has finished cooking, the server will suggest new recipes and ingredients based on their previous selections, for example, new vegan recipes or untried gluten-free recipes.

[0812] Step 14:

[0813] If the user accepts the new suggestion, the process of recipe search, optimization of nutritional balance, display of cooking steps, etc. is repeated again. This cycle allows the user to continuously enjoy new cooking experiences.

[0814] Example 1

[0815] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0816] Today's consumers expect easy and convenient ways to prepare healthy meals amid their busy lifestyles. However, finding nutritionally balanced recipes that fit their individual dietary preferences, allergies, and health goals, and understanding the cooking process, can be challenging. Systems that suggest new cooking experiences based on past choices are also lacking. Providing a consistent user experience across multiple devices is also a key challenge.

[0817] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0818] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for generating new recipes based on prompt sentences using a generative AI model]; and [means for providing audio advice to the user. This allows users to easily enjoy nutritionally balanced meals and receive powerful support for achieving their set health goals.

[0819] "User profile information" refers to personal information that a user registers with the system, such as name, email address, dietary preferences, allergy information, and health goals.

[0820] A "cooking recipe" is information that includes specific steps and information on ingredients for a user to cook.

[0821] "Nutrition optimization" is the process of adjusting recipe ingredients and cooking methods to meet a user's health goals and nutritional needs.

[0822] "Image generation means" refers to technology for displaying each step of the cooking process and the finished product as images or videos.

[0823] A "user device" is a device used by a user to access the system, such as a smartphone, tablet, smartwatch, or eyeglass display.

[0824] "Past user selections" refers to historical information about recipes and ingredients selected by the user using the system.

[0825] A "generative AI model" is an artificial intelligence model that takes prompt text as input and generates new recipes and suggestions.

[0826] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[0827] "Audio advice" is audio guidance provided to help the user accurately follow the cooking process.

[0828] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[0829] User Registration and Login

[0830] A user accesses the system and enters information such as their name, email address, password, dietary preferences, allergy information, and health goals on a dedicated sign-up page. This generates an individual user profile, which the server stores in a secure database. For example, a MySQL database can be used to securely manage user information. Next, the user enters their login information on the sign-in page to access the system. The server checks the authentication information against the database and, if there is a match, authenticates the user.

[0831] Recipe Search and Filtering

[0832] The server retrieves the logged-in user's profile information from the database. For example, it retrieves the profile data using the Django framework. Next, the server searches the database for cooking recipes based on the user's profile information. This is done efficiently using SQL queries or NoSQL databases. The server then filters the retrieved recipes based on the user's allergies and dietary preferences.

[0833] Optimizing nutritional balance

[0834] The server calculates the nutritional information for the filtered recipes by summing up the nutritional data for each ingredient and displaying it. It uses a Python library to perform the calculations and optimizes the amount of ingredients used based on the user's health goals (e.g., weight loss).

[0835] Cooking process support

[0836] The device uses image generation tools to display the cooking process and the finished product. For example, a mobile app using React Native displays images and videos for each step. It also uses a speech synthesis API (e.g., Google Text-to-Speech) to provide audio guidance for each step, helping the user to follow the steps accurately.

[0837] Multi-device compatible

[0838] The server collects information about the device used by the user and provides information optimized for each device. For example, it analyzes user agent information and provides the optimal interface for each device, such as a smartphone, smartwatch, or eyeglass display.

[0839] Individual proposals

[0840] The server retrieves the user's past recipe selection data from a database and uses it to suggest new recipes and ingredients. Furthermore, it uses a generative AI model to generate new recipes based on the prompt. For example, it uses OpenAI's generative AI model to generate new recipes and suggest them to the user.

[0841] Specific examples

[0842] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches and filters "vegan" and "gluten-free" recipes based on the user profile. When the user selects one recipe from the list, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step in sequence with images and audio guidance to help the user follow the steps accurately. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's previous selections, allowing the user to enjoy constantly new cooking experiences.

[0843] Example prompts for generative AI models

[0844] "Please suggest vegan and gluten-free recipes for those looking to lose weight."

[0845] With this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their health goals.

[0846] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0847] Step 1:

[0848] A user accesses the system and enters the required information (name, email address, password, dietary preferences, allergy information, health goals, etc.) on the sign-up page. The entered information is sent to the server by clicking the submit button. (Input) User information. (Output) Creation of a user profile.

[0849] Step 2:

[0850] The server generates an individual user profile based on the received user information and stores it in a secure database. For example, a MySQL database is used. This database stores all information about registered users. (Input) User information. (Output) User profile stored in a secure database.

[0851] Step 3:

[0852] A user logs in by entering their email address and password on the sign-in page. (Input) Email address and password. (Output) Login request.

[0853] Step 4:

[0854] The server retrieves the user's information from the database and compares it with the entered password. If they match, the user is authenticated and the login is successful. (Input) Login information and database information. (Output) Authentication result.

[0855] Step 5:

[0856] The server retrieves the profile information of the logged-in user from the database. For example, it retrieves the profile data using the Django framework. (Input) User ID. (Output) Profile information.

[0857] Step 6:

[0858] The server searches the database for recipes that match the user's food preferences based on the user's profile information. The search is performed efficiently using SQL queries and NoSQL databases. (Input) User profile information. (Output) Unfiltered list of recipes.

[0859] Step 7:

[0860] The server filters the retrieved recipes based on the user's allergy information and health goals. For example, it extracts only vegan and gluten-free recipes. (Input) Allergy information and health goals. (Output) Filtered list of cooking recipes.

[0861] Step 8:

[0862] The server calculates the nutritional information for each ingredient in the filtered recipe and optimizes it to meet the user's health goals. It uses a Python library to aggregate the nutritional information and adjust the ingredient amounts as needed. (Input) Filtered cooking recipe. (Output) Optimized nutritional information and ingredient amounts.

[0863] Step 9:

[0864] Based on the recipe selected by the user, the device uses image generation means to display each cooking step and a finished image. A mobile app using React Native displays images and videos corresponding to each step. (Input) Selected cooking recipe. (Output) Display of cooking steps using images and videos.

[0865] Step 10:

[0866] The device uses a speech synthesis API (such as Google Text-to-Speech) to provide audio advice for each step. (Input) Cooking process information. (Output) Audio advice.

[0867] Step 11:

[0868] The server collects information about the device used by the user and provides information optimized for each device. It analyzes the user agent information and generates different interfaces for smartphones, smartwatches, etc. (Input) Device information. (Output) Optimized interface.

[0869] Step 12:

[0870] The server retrieves the user's past recipe selection data from the database and uses this data to suggest new recipes and ingredients. (Input) Past selection data. (Output) New recipe suggestions.

[0871] Step 13:

[0872] The server uses a generative AI model to generate a new recipe based on the prompt. The generated recipe is displayed optimized for the user's profile information. (Input) Prompt and user profile information. (Output) The generated new recipe.

[0873] For example, when the prompt "Please suggest vegan and gluten-free recipes for weight loss" is entered, the server uses a generative AI model to generate appropriate recipes and suggest them to the user.

[0874] (Application example 1)

[0875] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0876] In recent years, with the rise in health consciousness, a nutritionally balanced diet has become increasingly important. However, it is difficult to consistently practice this diet due to the time-consuming nature of busy lives. While there are many dietary management applications available, few provide recipes tailored to individual user preferences, allergies, and other factors. Furthermore, it is often difficult to find the time to actually make the suggested recipes. There is a need for a system that can solve these problems and support healthy eating habits.

[0877] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0878] In this invention, the server includes means for registering user profile information, means for searching for cooking recipes based on the user profile information, means for optimizing the nutritional balance of the cooking recipes, means for displaying cooking steps using an image generation means, means for providing the optimized information to a user device, means for suggesting new recipes based on past user selections, and means for enabling the suggested dishes to be instantly ordered via a delivery service, thereby enabling users to easily create and order nutritionally balanced meals tailored to their individual health goals.

[0879] "User profile information" refers to data that includes personal information about a user, such as dietary preferences, allergy information, and health goals.

[0880] A "cooking recipe" is information that shows the ingredients, steps, and processes required to make a certain dish.

[0881] "Nutritional balance" refers to the appropriate proportions of each nutrient (calories, protein, fat, carbohydrates, vitamins, etc.) based on specific health goals and dietary restrictions.

[0882] "Image generation means" refers to a technical means for displaying the cooking process and the finished product as images or videos.

[0883] "User device" refers to a terminal used by a user, such as a smartphone, smart glasses, or head-mounted display.

[0884] "Past user selections" refers to data on recipes and ingredients that the user has previously selected or used.

[0885] A "delivery service" is a service that delivers food ordered by a user to a specified location.

[0886] A "server" is a computer system that performs various processes such as managing user profile information, searching for recipes, and optimizing nutritional balance.

[0887] The present invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system functions in cooperation with a server, a terminal, and a user device. First, a specific embodiment will be described.

[0888] User Registration and Login

[0889] A user accesses the system and creates an account. The server receives information input from the user, registering their name, email address, password, dietary preferences, allergy information, health goals, etc. Based on this information, an individual user profile is generated and stored in the database. After registering, the user can log in and use services based on their individual information.

[0890] Recipe Search and Filtering

[0891] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0892] Optimizing nutritional balance

[0893] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0894] Cooking process support

[0895] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help the user follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0896] Delivery collaboration

[0897] The server provides a function that allows users to instantly order the suggested dishes through a delivery service, enabling users to consume healthy meals with minimal effort based on optimized recipes.

[0898] Multi-device compatible

[0899] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[0900] Individual proposals

[0901] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0902] Examples and prompts

[0903] For example, if the user is "vegan" and "looking to lose weight," they might send a prompt to the generative AI like this:

[0904] User Profile:

[0905] Food preference: Vegan

[0906] Health goal: weight loss

[0907] Generate proposals:

[0908] Please suggest healthy vegan recipes that fit this profile, preferably ones that are nutritionally balanced and support weight loss.

[0909] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals, with the aim of improving the quality of their meals as part of their health management.

[0910] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0911] Step 1: User Registration

[0912] When a user creates a new account, the server receives the user's name, email address, password, dietary preferences, allergy information, health goals, etc. A user profile is generated based on this information and stored in a database. The input is the user's personal information and dietary settings, and the output is a notification that the user profile has been registered.

[0913] Step 2: Log in

[0914] A user enters an email address and password on a login screen. The server receives this and checks it against an existing user profile in its database. If there is a match, the login is successful and the user is granted access to their account. The input is the user's authentication information, and the output is a notification of login success or failure.

[0915] Step 3: Find a recipe

[0916] After logging in, the server searches the database for cooking recipes based on the user profile information. It filters recipes that match the user's dietary preferences and allergy information and presents them to the user. The input is the user profile information, and the output is the filtered list of recipes.

[0917] Step 4: Optimize your nutritional balance

[0918] The server calculates nutritional information for the selected recipe and suggests optimal nutritional balance by adjusting ingredients and portions based on the user's health goals (e.g., weight loss). The input is the selected recipe and the user's health goals, and the output is the optimized nutritional information and suggested recipe modifications.

[0919] Step 5: Show the cooking process

[0920] The device uses an image generation means to display the cooking process and the finished product. Images, videos, and even audio guidance help the user follow the steps accurately. The input is optimized recipe information, and the output is displayed images and videos of the cooking process steps.

[0921] Step 6: Order delivery

[0922] The server allows the user to order the selected recipe via a delivery service. Once the user confirms the order, the server sends the order information via the delivery service's API and processes the food for delivery. The input is the user's order information, and the output is a delivery order confirmation and tracking information.

[0923] Step 7: Multi-device compatibility

[0924] The server provides information optimized for the type of device the user is using, providing a consistent user experience across different devices such as smartphones, smartwatches, and eyeglass displays. The input is device information and user profile information, and the output is display information optimized for the device.

[0925] Step 8: Personalized recipe suggestions

[0926] The server uses a generative AI model based on past user selection data and current profile information to automatically generate new recipe and ingredient suggestions. For example, it can suggest the most popular recipes over a certain period of time or new, untried recipes to the user. The input is the user's past selection data and profile information, and the output is individually generated new recipe suggestions.

[0927] Specifically, the server sends the prompt "User profile: Food preference: Vegan, Health goal: Weight loss" to the generative AI model, which then suggests healthy vegan recipes.

[0928] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0929] This invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy meal. This system can further improve the user's dining experience by combining it with an emotion engine that recognizes the user's emotions. Detailed embodiments of the system are described below.

[0930] User Registration and Login

[0931] A user accesses the system and enters information to create an account (such as name, email address, password, dietary preferences, allergy information, and health goals). Once the information is complete, the user presses the "Register" button. The server receives the information sent by the user, generates a user profile based on that information, and stores it in a database. The user can then log in and use services based on their individual information.

[0932] Recipe Search and Filtering

[0933] The server searches a database for recipes based on the user's profile information, and the resulting recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[0934] Optimizing nutritional balance

[0935] The server calculates the nutritional information for the searched recipe. If the goal is weight loss, it will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[0936] Cooking process support

[0937] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[0938] Multi-device compatible

[0939] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[0940] Individual proposals

[0941] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[0942] Use of emotion engine

[0943] The app is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and vocal tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice based on that information.

[0944] Specifically, if the user is feeling stressed, the emotion engine will suggest recipes and steps that have a relaxing effect. For example, it will suggest recipes that are easy to make in a short time and contain ingredients that will improve the user's mood. In addition, the emotion engine will monitor the user's emotions while cooking and provide appropriate voice advice and encouraging messages.

[0945] Specific examples

[0946] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." The user logs into the system, and the server searches for vegan and gluten-free recipes based on the user profile. The user then selects a recipe, and the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, allowing the user to follow the steps accurately. The server also provides an optimized display for the user's device (e.g., a smartwatch).

[0947] Furthermore, the emotion engine can recognize the user's emotions and, for example, if the user is feeling stressed, it can suggest dishes that have a relaxing effect. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages to help the user enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also gain emotional satisfaction in the process.

[0948] The processing flow will be explained below.

[0949] Step 1:

[0950] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals to create an account. The user presses the "Register" button.

[0951] Step 2:

[0952] The server receives the information sent by the user, generates a profile based on this information, stores it in a database, and sends the user a registration confirmation message.

[0953] Step 3:

[0954] A user enters their email address and password and presses the login button to access the system. The server verifies the login information and loads the profile information.

[0955] Step 4:

[0956] The server searches the database for suitable recipes based on the user profile, filters the recipes that match the user's criteria, and sends the results to the device.

[0957] Step 5:

[0958] The device displays a filtered list of recipes, and the user selects one from the displayed list.

[0959] Step 6:

[0960] After a user selects a recipe, the server calculates the recipe's details and nutritional balance, and then makes recommendations based on the results to help with health goals such as weight loss.

[0961] Step 7:

[0962] The server sends the optimized recipe information to the device, which then displays it to the user, showing each step of the recipe with images and videos, and providing audio advice.

[0963] Step 8:

[0964] The emotion engine analyzes the user's facial expressions and voice tone to recognize the user's emotional state. The server receives the data from the emotion engine and determines the user's current emotional state.

[0965] Step 9:

[0966] The server uses an emotion engine to suggest recipes and ingredients that suit the user's emotional state. For example, if the user is feeling stressed, it suggests dishes that have a relaxing effect.

[0967] Step 10:

[0968] The device starts cooking according to the recipe selected by the user, and displays each step in real time, while the emotion engine monitors the user's emotional state.

[0969] Step 11:

[0970] Based on the data from the emotion engine, the device provides appropriate voice advice and encouraging messages to the user while cooking. The content and tone of the advice are adjusted according to the user's emotional state.

[0971] Step 12:

[0972] After the user has finished cooking, the server will suggest new recipes and ingredients based on the user's past selection data, taking into account the user's preferences and emotional state.

[0973] Step 13:

[0974] If the user accepts the new suggestion, the process of recipe search, nutritional balance optimization, process display, and support from the emotion engine is repeated again. This cycle allows users to continuously enjoy new cooking experiences and nutritionally balanced meals.

[0975] Example 2

[0976] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0977] In today's busy lifestyles, it is difficult for users to maintain a healthy diet. It is also difficult to select individual meals and manage nutritional balance, and existing systems do not adequately support recipe selection and cooking according to emotional states. Therefore, there is a need for a system that can effectively improve users' eating habits and provide meal suggestions and cooking support according to their emotions.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0979] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; and [means for recognizing the user's emotions using an emotion engine and adjusting the content of cooking advice.] This not only enables users to easily select and cook individual, nutritionally balanced meals, but also allows them to receive appropriate support according to their emotional state.

[0980] "User profile information" refers to personal information such as dietary preferences, allergy information, and health goals that a user registers in the system.

[0981] A "server" is a computer system that processes the entire system and manages data, and is a device that plays a role in executing various means.

[0982] A "cooking recipe" is information that describes the steps and ingredients for making a particular dish.

[0983] "Nutritional balance" refers to the proper distribution of nutrients such as calories, protein, fat, and carbohydrates.

[0984] "Image generation means" refers to a technology for generating images of cooking steps and finished recipes, and visually presenting them to the user.

[0985] "User Device" refers to the equipment used by a user to access the system, including smartphones, smartwatches, eyeglass displays, etc.

[0986] "Past user selections" refers to historical data of recipes and ingredients selected by the user in the past.

[0987] An "emotion engine" is a technology that analyzes a user's facial expressions and vocal tone to recognize their emotional state.

[0988] "Cooking advice" refers to information that guides or instructs the user when cooking.

[0989] MODE FOR CARRYING OUT THE INVENTION

[0990] This invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. In particular, by combining it with an emotion engine that recognizes the user's emotions, the user's dining experience can be further improved. Detailed embodiments of the system are described below.

[0991] User Registration and Login

[0992] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals. When the user presses the "Register" button, the device sends the entered information to the server. The server generates a user profile based on the received information and stores it in a database. The user can then log in and use individual services.

[0993] Recipe Search and Filtering

[0994] The server searches the database for recipes based on the user's profile information. For example, if the user is vegan and prefers gluten-free meals, it will retrieve only recipes that match those criteria.

[0995] Optimizing nutritional balance

[0996] The server calculates the nutritional information for the searched recipe, examining each nutrient component, such as calories, protein, fat, and carbohydrates. Based on the user's health goals (e.g., weight loss), it adjusts the amounts of ingredients and suggests optimal nutritional balance. For example, it can adjust the amounts of certain ingredients to reduce calorie intake.

[0997] Cooking process support

[0998] The device uses an image generation means to display the cooking process and a finished product. Images and videos are displayed for each step, and audio advice is also provided. This allows even beginners and busy users to complete the dish with confidence.

[0999] Multi-device compatible

[1000] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[1001] Individual proposals

[1002] The server uses a generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current user profile. For example, it can suggest recipes that the user has chosen most over a certain period of time, or new recipes that the user has not yet tried.

[1003] Use of emotion engine

[1004] The device is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and voice tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice content based on that information. Specifically, if the user is feeling stressed, it can suggest recipes and steps that have a relaxing effect. For example, it can suggest relaxing dishes that can be made easily and quickly, or recipes that include ingredients that will lift the user's mood. In addition, the emotion engine monitors the user's emotions while cooking and provides appropriate voice advice and encouraging messages.

[1005] Specific examples

[1006] For example, consider a user named "User123" who prefers vegan and gluten-free diets and is trying to lose weight. After logging in to the system, the server searches for vegan and gluten-free recipes based on the user profile. After the user selects a recipe, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device supports the user's cooking process by displaying images and providing audio guidance. Furthermore, the emotion engine recognizes the user's emotions and, for example, can suggest dishes with a relaxing effect if the user is feeling stressed. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages, helping the user to enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also feel emotionally fulfilled in the process.

[1007] Example prompts for generative AI models

[1008] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[1009] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1010] Step 1:

[1011] User Registration

[1012] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals.

[1013] The terminal transmits the input information to the server.

[1014] The server generates a user profile based on the received information and stores it in a database.

[1015] Input: User information (name, email address, password, food preferences, allergy information, health goals)

[1016] Output: User profile stored in the database

[1017] Step 2:

[1018] User Login

[1019] The user logs in using the email address and password they registered.

[1020] The server verifies the credentials and retrieves the user profile.

[1021] Input: Email address, Password

[1022] Output: Authentication result, user profile

[1023] Step 3:

[1024] Recipe Search and Filtering

[1025] The server searches the database for cooking recipes based on the user profile information.

[1026] Extract only recipes that match the user's dietary preferences and allergy information.

[1027] Input: User profile information

[1028] Output: Filtered recipe list

[1029] Step 4:

[1030] Optimizing nutritional balance

[1031] The server calculates the nutritional information for the retrieved recipe.

[1032] Based on the user's health goals (e.g., weight loss), it adjusts ingredient amounts and suggests optimal nutritional balance.

[1033] Input: filtered list of recipes, user's health goals

[1034] Output: Nutritionally balanced recipes

[1035] Step 5:

[1036] Cooking process support

[1037] The terminal uses an image generating means to display the cooking process and a finished image.

[1038] Each step is accompanied by images and videos of the process, and audio guidance is also provided.

[1039] Input: Nutritionally balanced recipe

[1040] Output: Recipe process images, videos, audio advice

[1041] Step 6:

[1042] Multi-device compatible

[1043] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc.

[1044] Input: User's device information

[1045] Output: Information display optimized for each device

[1046] Step 7:

[1047] Individual proposal generation

[1048] The server uses the generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current profile.

[1049] Input: Past user selection data, user profile

[1050] Output: New recipe and ingredient suggestions

[1051] Step 8:

[1052] Use of emotion engine

[1053] The emotion engine recognizes the user's emotions, for example by analyzing the user's facial expressions and tone of voice.

[1054] The server collects the user's emotional state in real time and adjusts recipes and cooking advice based on that information.

[1055] Input: User's facial expression data, voice tone data

[1056] Output: User's emotional state, recipes and advice based on the emotion

[1057] Prompt Sentence Examples

[1058] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[1059] (Application example 2)

[1060] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1061] In modern society, it is difficult for users to easily enjoy healthy meals. In particular, it is complex to consider the nutritional balance and food preferences of each individual user, and the cooking process is difficult, which places a heavy burden on beginners and busy users. Furthermore, food choices and motivation to cook are often influenced by stress and emotional fluctuations, so a system that can simultaneously solve these issues is needed.

[1062] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1063] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for providing cooking advice based on the user's emotional state using an emotion engine that recognizes the user's emotions]; and [means for providing audio advice and encouraging messages to the user in cooperation with a smartphone, smart glasses, or smart watch.] This allows users to receive personalized meal suggestions and support for cooking steps, and encouragement and advice according to their emotional state, making it easy to enjoy healthy, nutritionally balanced meals.

[1064] "User profile information" is data about an individual user, such as dietary preferences, allergy information, and health goals.

[1065] The "means for searching for recipes" is a function for extracting optimal recipes based on user profile information.

[1066] The "means for optimizing the nutritional balance of a cooking recipe" is a function for adjusting the nutritional components of a cooking recipe in accordance with the user's health goals, thereby providing an optimal nutritional balance.

[1067] The "means for displaying cooking steps using image generation means" is a function for visually displaying each cooking step and the finished product, using images and videos.

[1068] "Means for providing information optimized for user devices" refers to a function that provides information in a format that is optimal for various devices used by users (smartphones, smart glasses, smart watches, etc.).

[1069] The "means for proposing new recipes based on past user selections" is a function that analyzes the user's past selection history and appropriately suggests new cooking recipes.

[1070] An "emotion engine" is a technology for recognizing and analyzing a user's emotional state, primarily by analyzing facial expressions and vocal tone.

[1071] The "means for providing cooking advice" is a function for providing appropriate advice and guidance to the user while cooking.

[1072] "Smartphones, smart glasses, and smart watches" are devices that users use to receive information, and each can provide data in different formats.

[1073] "Voice advice" is a function that provides instructions and advice to the user through voice.

[1074] An "encouraging message" is a message intended to encourage and motivate the user.

[1075] This invention is a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system registers user profile information, searches for cooking recipes, optimizes the nutritional balance of those recipes, and displays the cooking steps to support cooking. It also provides optimized information to the user's device and uses an emotion engine to provide advice based on the user's emotional state.

[1076] The server first registers user profile information. The user enters their dietary preferences, allergy information, health goals, etc., which are then saved in a database. The server then searches for the most suitable cooking recipe based on the user profile information and provides it to the user. After a recipe is selected, the server optimizes the nutritional balance of the recipe and makes suggestions that meet the user's health goals.

[1077] Furthermore, the device displays each cooking step and a picture of the finished product using images and audio. This information is optimized for the device used by the user, such as a smartphone, smart glasses, or smart watch. At this point, the cooking steps are displayed visually in an easy-to-understand manner using an image generation means, making cooking easy even for beginners or busy users.

[1078] The server also analyzes the user's past selection history and suggests new recipes, using a generative AI model to generate prompts and suggest optimal dishes based on the user's past selection data and current profile.

[1079] The emotion engine recognizes the user's emotional state and monitors it in real time while cooking. For example, if the user is feeling stressed, the emotion engine will provide relaxing dishes or encouraging messages. The smartphone, smart glasses, and smartwatch work together to provide voice advice, helping users to enjoy cooking more.

[1080] As a specific example, let's say a user is vegan, prefers gluten-free meals, and is trying to lose weight. In this case, the user's profile information is registered, and the server searches for and optimizes vegan and gluten-free recipes. If the user is feeling stressed, the server will suggest recipes with a relaxing effect, such as "banana smoothie" or "salad," and while cooking, it will provide encouraging voice messages such as "It's okay, it's hard, but you're almost there!" This allows users to easily enjoy healthy, nutritionally balanced meals.

[1081] An example prompt is:

[1082] "User profile: Vegan, gluten-free, goal: weight loss, current emotion: stressed. Generate a meal plan that considers these preferences and goals, and includes relaxation-inducing recipes."

[1083] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1084] Step 1: Register your user profile information

[1085] The server receives profile information from the user, such as dietary preferences, allergy information, health goals, etc. Based on this input information, the server registers and saves the user's profile in a database.

[1086] Input: User's dietary preferences, allergy information, health goals

[1087] Data processing and calculation: Formatting this information so that it can be stored in a database

[1088] Output: User profile saved message

[1089] Step 2: Find a recipe

[1090] The server references the user profile information, searches the database for relevant recipes, and filters out only those that match the user's preferences and allergy information.

[1091] Input: User profile information

[1092] Data processing and calculation: Execute a database query to retrieve recipes that match the criteria

[1093] Output: A list of matching recipes

[1094] Step 3: Optimize your nutritional balance

[1095] The server calculates the nutritional information of the searched recipes and optimizes them to meet the user's health goals, suggesting optimal nutritional balance by adjusting the amounts of ingredients used, etc.

[1096] Input: list of cooking recipes, user's health goals

[1097] Data processing and calculation: Analysis and adjustment of nutritional components

[1098] Output: Optimized nutritional balance recipe

[1099] Step 4: Show the cooking process

[1100] The device displays each step of the optimized recipe received from the server using images, videos, and audio, and is optimized to display appropriately regardless of the device the user is using.

[1101] Input: Optimized nutritional balance recipe

[1102] Data processing and calculation: Image generation and audio data distribution preparation

[1103] Output: Images, videos, and audio showing the cooking process

[1104] Step 5: Recognizing user emotions

[1105] The device analyzes the user's facial expressions and voice tone and uses an emotion engine to recognize the user's emotions in real time.

[1106] Input: User's face photo, voice data

[1107] Data processing and calculation: Extraction of emotional states using emotion analysis algorithms

[1108] Output: User's emotional state (e.g., stress, joy, sadness)

[1109] Step 6: Offering emotionally-based advice

[1110] The server provides the user with appropriate cooking recipes and cooking advice based on the user's emotional state obtained from the emotion engine.

[1111] Input: User emotional state, optimized recipe, user profile information

[1112] Data processing and calculation: Analysis of past selection data and generation of prompt sentences using a generative AI model

[1113] Output: Emotion-based cooking recipes and cooking advice

[1114] Step 7: Providing audio advice and encouraging messages

[1115] The device receives voice advice and encouraging messages from the server and provides them to the user. This can be a smartphone, smart glasses, or smart watch.

[1116] Input: Audio advice and encouraging messages sent from the server

[1117] Data processing and calculation: Preparing audio data for device distribution

[1118] Output: Playback of voice advice and encouraging messages

[1119] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1120] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1121] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1122] [Fourth embodiment]

[1123] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1124] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1125] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1126] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1127] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1128] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1129] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1130] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1131] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1132] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1133] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1134] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1135] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1136] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[1137] User Registration and Login

[1138] A user accesses the system and creates an account. The system receives information input from the user, such as name, email address, password, dietary preferences, allergy information, and health goals. Based on this information, an individual user profile is generated and stored in a database. After registration, the user can log in and use services based on their individual information.

[1139] Recipe Search and Filtering

[1140] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[1141] Optimizing nutritional balance

[1142] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[1143] Cooking process support

[1144] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[1145] Multi-device compatible

[1146] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[1147] Individual proposals

[1148] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[1149] Specific examples

[1150] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches for "vegan" and "gluten-free" recipes based on the user profile. When the user selects one of the recipes, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, helping the user to accurately follow the steps. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's past choices, allowing the user to always enjoy new cooking experiences.

[1151] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals.

[1152] The processing flow will be explained below.

[1153] Step 1:

[1154] The user accesses the system and enters the information required to create an account (name, email address, password, dietary preferences, allergy information, health goals). Once the information is complete, the user presses the "Register" button.

[1155] Step 2:

[1156] The server receives the information sent by the user, generates a user profile based on it, and stores it in a database. Once registration is complete, the server sends a confirmation message to the user.

[1157] Step 3:

[1158] The user enters login information (email address and password) and clicks the login button to access the system.

[1159] Step 4:

[1160] The server verifies the user's login information and retrieves the corresponding user profile from the database. If authentication is successful, the user is redirected to the home screen.

[1161] Step 5:

[1162] Users enter filtering criteria such as the desired food category from the search menu and press the search button. For example, they can specify criteria such as "vegan" or "gluten-free."

[1163] Step 6:

[1164] The server searches the database for matching recipes based on the user's filtering criteria and profile information, and the search results are filtered to include only recipes that meet the user's criteria.

[1165] Step 7:

[1166] The terminal displays the recipe search results acquired from the server to the user, who then selects a specific recipe from the displayed list of recipes.

[1167] Step 8:

[1168] When a user selects a recipe, the server calculates the nutritional value along with the details of the selected recipe. Once the calculation of the nutritional value is complete, the result is displayed to the user.

[1169] Step 9:

[1170] Based on the user's health goals (e.g., weight loss), the server optimizes the nutritional balance of the selected recipe, adjusting ingredient amounts as needed and generating new suggestions.

[1171] Step 10:

[1172] The device uses an image generation means to display each step of the cooking process, specifically visually showing the process in the form of images or videos, and simultaneously providing audio advice.

[1173] Step 11:

[1174] The user starts cooking, and the device displays each step in real time and provides voice instructions to guide the user, ensuring that cooking proceeds smoothly.

[1175] Step 12:

[1176] The server detects the type of device information the user uses (smartphone, smartwatch, eyeglass display) and provides information in a display format optimized for each device.

[1177] Step 13:

[1178] After the user has finished cooking, the server will suggest new recipes and ingredients based on their previous selections, for example, new vegan recipes or untried gluten-free recipes.

[1179] Step 14:

[1180] If the user accepts the new suggestion, the process of recipe search, optimization of nutritional balance, display of cooking steps, etc. is repeated again. This cycle allows the user to continuously enjoy new cooking experiences.

[1181] Example 1

[1182] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1183] Today's consumers expect easy and convenient ways to prepare healthy meals amid their busy lifestyles. However, finding nutritionally balanced recipes that fit their individual dietary preferences, allergies, and health goals, and understanding the cooking process, can be challenging. Systems that suggest new cooking experiences based on past choices are also lacking. Providing a consistent user experience across multiple devices is also a key challenge.

[1184] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1185] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for generating new recipes based on prompt sentences using a generative AI model]; and [means for providing audio advice to the user. This allows users to easily enjoy nutritionally balanced meals and receive powerful support for achieving their set health goals.

[1186] "User profile information" refers to personal information that a user registers with the system, such as name, email address, dietary preferences, allergy information, and health goals.

[1187] A "cooking recipe" is information that includes specific steps and information on ingredients for a user to cook.

[1188] "Nutrition optimization" is the process of adjusting recipe ingredients and cooking methods to meet a user's health goals and nutritional needs.

[1189] "Image generation means" refers to technology for displaying each step of the cooking process and the finished product as images or videos.

[1190] A "user device" is a device used by a user to access the system, such as a smartphone, tablet, smartwatch, or eyeglass display.

[1191] "Past user selections" refers to historical information about recipes and ingredients selected by the user using the system.

[1192] A "generative AI model" is an artificial intelligence model that takes prompt text as input and generates new recipes and suggestions.

[1193] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[1194] "Audio advice" is audio guidance provided to help the user accurately follow the cooking process.

[1195] The present invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy diet. Detailed embodiments of the system will be described below.

[1196] User Registration and Login

[1197] A user accesses the system and enters information such as their name, email address, password, dietary preferences, allergy information, and health goals on a dedicated sign-up page. This generates an individual user profile, which the server stores in a secure database. For example, a MySQL database can be used to securely manage user information. Next, the user enters their login information on the sign-in page to access the system. The server checks the authentication information against the database and, if there is a match, authenticates the user.

[1198] Recipe Search and Filtering

[1199] The server retrieves the logged-in user's profile information from the database. For example, it retrieves the profile data using the Django framework. Next, the server searches the database for cooking recipes based on the user's profile information. This is done efficiently using SQL queries or NoSQL databases. The server then filters the retrieved recipes based on the user's allergies and dietary preferences.

[1200] Optimizing nutritional balance

[1201] The server calculates the nutritional information for the filtered recipes by summing up the nutritional data for each ingredient and displaying it. It uses a Python library to perform the calculations and optimizes the amount of ingredients used based on the user's health goals (e.g., weight loss).

[1202] Cooking process support

[1203] The device uses image generation tools to display the cooking process and the finished product. For example, a mobile app using React Native displays images and videos for each step. It also uses a speech synthesis API (e.g., Google Text-to-Speech) to provide audio guidance for each step, helping the user to follow the steps accurately.

[1204] Multi-device compatible

[1205] The server collects information about the device used by the user and provides information optimized for each device. For example, it analyzes user agent information and provides the optimal interface for each device, such as a smartphone, smartwatch, or eyeglass display.

[1206] Individual proposals

[1207] The server retrieves the user's past recipe selection data from a database and uses it to suggest new recipes and ingredients. Furthermore, it uses a generative AI model to generate new recipes based on the prompt. For example, it uses OpenAI's generative AI model to generate new recipes and suggest them to the user.

[1208] Specific examples

[1209] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." After the user logs in to the system, the server searches and filters "vegan" and "gluten-free" recipes based on the user profile. When the user selects one recipe from the list, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step in sequence with images and audio guidance to help the user follow the steps accurately. The server also provides a display optimized for the user's device (e.g., a smartwatch). Furthermore, the server suggests new recipes based on the user's previous selections, allowing the user to enjoy constantly new cooking experiences.

[1210] Example prompts for generative AI models

[1211] "Please suggest vegan and gluten-free recipes for those looking to lose weight."

[1212] With this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their health goals.

[1213] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1214] Step 1:

[1215] A user accesses the system and enters the required information (name, email address, password, dietary preferences, allergy information, health goals, etc.) on the sign-up page. The entered information is sent to the server by clicking the submit button. (Input) User information. (Output) Creation of a user profile.

[1216] Step 2:

[1217] The server generates an individual user profile based on the received user information and stores it in a secure database. For example, a MySQL database is used. This database stores all information about registered users. (Input) User information. (Output) User profile stored in a secure database.

[1218] Step 3:

[1219] A user logs in by entering their email address and password on the sign-in page. (Input) Email address and password. (Output) Login request.

[1220] Step 4:

[1221] The server retrieves the user's information from the database and compares it with the entered password. If they match, the user is authenticated and the login is successful. (Input) Login information and database information. (Output) Authentication result.

[1222] Step 5:

[1223] The server retrieves the profile information of the logged-in user from the database. For example, it retrieves the profile data using the Django framework. (Input) User ID. (Output) Profile information.

[1224] Step 6:

[1225] The server searches the database for recipes that match the user's food preferences based on the user's profile information. The search is performed efficiently using SQL queries and NoSQL databases. (Input) User profile information. (Output) Unfiltered list of recipes.

[1226] Step 7:

[1227] The server filters the retrieved recipes based on the user's allergy information and health goals. For example, it extracts only vegan and gluten-free recipes. (Input) Allergy information and health goals. (Output) Filtered list of cooking recipes.

[1228] Step 8:

[1229] The server calculates the nutritional information for each ingredient in the filtered recipe and optimizes it to meet the user's health goals. It uses a Python library to aggregate the nutritional information and adjust the ingredient amounts as needed. (Input) Filtered cooking recipe. (Output) Optimized nutritional information and ingredient amounts.

[1230] Step 9:

[1231] Based on the recipe selected by the user, the device uses image generation means to display each cooking step and a finished image. A mobile app using React Native displays images and videos corresponding to each step. (Input) Selected cooking recipe. (Output) Display of cooking steps using images and videos.

[1232] Step 10:

[1233] The device uses a speech synthesis API (such as Google Text-to-Speech) to provide audio advice for each step. (Input) Cooking process information. (Output) Audio advice.

[1234] Step 11:

[1235] The server collects information about the device used by the user and provides information optimized for each device. It analyzes the user agent information and generates different interfaces for smartphones, smartwatches, etc. (Input) Device information. (Output) Optimized interface.

[1236] Step 12:

[1237] The server retrieves the user's past recipe selection data from the database and uses this data to suggest new recipes and ingredients. (Input) Past selection data. (Output) New recipe suggestions.

[1238] Step 13:

[1239] The server uses a generative AI model to generate a new recipe based on the prompt. The generated recipe is displayed optimized for the user's profile information. (Input) Prompt and user profile information. (Output) The generated new recipe.

[1240] For example, when the prompt "Please suggest vegan and gluten-free recipes for weight loss" is entered, the server uses a generative AI model to generate appropriate recipes and suggest them to the user.

[1241] (Application example 1)

[1242] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1243] In recent years, with the rise in health consciousness, a nutritionally balanced diet has become increasingly important. However, it is difficult to consistently practice this diet due to the time-consuming nature of busy lives. While there are many dietary management applications available, few provide recipes tailored to individual user preferences, allergies, and other factors. Furthermore, it is often difficult to find the time to actually make the suggested recipes. There is a need for a system that can solve these problems and support healthy eating habits.

[1244] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1245] In this invention, the server includes means for registering user profile information, means for searching for cooking recipes based on the user profile information, means for optimizing the nutritional balance of the cooking recipes, means for displaying cooking steps using an image generation means, means for providing the optimized information to a user device, means for suggesting new recipes based on past user selections, and means for enabling the suggested dishes to be instantly ordered via a delivery service, thereby enabling users to easily create and order nutritionally balanced meals tailored to their individual health goals.

[1246] "User profile information" refers to data that includes personal information about a user, such as dietary preferences, allergy information, and health goals.

[1247] A "cooking recipe" is information that shows the ingredients, steps, and processes required to make a certain dish.

[1248] "Nutritional balance" refers to the appropriate proportions of each nutrient (calories, protein, fat, carbohydrates, vitamins, etc.) based on specific health goals and dietary restrictions.

[1249] "Image generation means" refers to a technical means for displaying the cooking process and the finished product as images or videos.

[1250] "User device" refers to a terminal used by a user, such as a smartphone, smart glasses, or head-mounted display.

[1251] "Past user selections" refers to data on recipes and ingredients that the user has previously selected or used.

[1252] A "delivery service" is a service that delivers food ordered by a user to a specified location.

[1253] A "server" is a computer system that performs various processes such as managing user profile information, searching for recipes, and optimizing nutritional balance.

[1254] The present invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system functions in cooperation with a server, a terminal, and a user device. First, a specific embodiment will be described.

[1255] User Registration and Login

[1256] A user accesses the system and creates an account. The server receives information input from the user, registering their name, email address, password, dietary preferences, allergy information, health goals, etc. Based on this information, an individual user profile is generated and stored in the database. After registering, the user can log in and use services based on their individual information.

[1257] Recipe Search and Filtering

[1258] The server searches a database for recipes based on the user's profile information. The searched recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[1259] Optimizing nutritional balance

[1260] The server calculates the nutritional information for the searched recipe. For example, if the goal is weight loss, the server will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[1261] Cooking process support

[1262] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help the user follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[1263] Delivery collaboration

[1264] The server provides a function that allows users to instantly order the suggested dishes through a delivery service, enabling users to consume healthy meals with minimal effort based on optimized recipes.

[1265] Multi-device compatible

[1266] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, and eyeglass displays, allowing users to receive consistent support regardless of the device they use.

[1267] Individual proposals

[1268] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[1269] Examples and prompts

[1270] For example, if the user is "vegan" and "looking to lose weight," they might send a prompt to the generative AI like this:

[1271] User Profile:

[1272] Food preference: Vegan

[1273] Health goal: weight loss

[1274] Generate proposals:

[1275] Please suggest healthy vegan recipes that fit this profile, preferably ones that are nutritionally balanced and support weight loss.

[1276] Through this system, users can easily enjoy nutritionally balanced meals and receive powerful support to achieve their set health goals, with the aim of improving the quality of their meals as part of their health management.

[1277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1278] Step 1: User Registration

[1279] When a user creates a new account, the server receives the user's name, email address, password, dietary preferences, allergy information, health goals, etc. A user profile is generated based on this information and stored in a database. The input is the user's personal information and dietary settings, and the output is a notification that the user profile has been registered.

[1280] Step 2: Log in

[1281] A user enters an email address and password on a login screen. The server receives this and checks it against an existing user profile in its database. If there is a match, the login is successful and the user is granted access to their account. The input is the user's authentication information, and the output is a notification of login success or failure.

[1282] Step 3: Find a recipe

[1283] After logging in, the server searches the database for cooking recipes based on the user profile information. It filters recipes that match the user's dietary preferences and allergy information and presents them to the user. The input is the user profile information, and the output is the filtered list of recipes.

[1284] Step 4: Optimize your nutritional balance

[1285] The server calculates nutritional information for the selected recipe and suggests optimal nutritional balance by adjusting ingredients and portions based on the user's health goals (e.g., weight loss). The input is the selected recipe and the user's health goals, and the output is the optimized nutritional information and suggested recipe modifications.

[1286] Step 5: Show the cooking process

[1287] The device uses an image generation means to display the cooking process and the finished product. Images, videos, and even audio guidance help the user follow the steps accurately. The input is optimized recipe information, and the output is displayed images and videos of the cooking process steps.

[1288] Step 6: Order delivery

[1289] The server allows the user to order the selected recipe via a delivery service. Once the user confirms the order, the server sends the order information via the delivery service's API and processes the food for delivery. The input is the user's order information, and the output is a delivery order confirmation and tracking information.

[1290] Step 7: Multi-device compatibility

[1291] The server provides information optimized for the type of device the user is using, providing a consistent user experience across different devices such as smartphones, smartwatches, and eyeglass displays. The input is device information and user profile information, and the output is display information optimized for the device.

[1292] Step 8: Personalized recipe suggestions

[1293] The server uses a generative AI model based on past user selection data and current profile information to automatically generate new recipe and ingredient suggestions. For example, it can suggest the most popular recipes over a certain period of time or new, untried recipes to the user. The input is the user's past selection data and profile information, and the output is individually generated new recipe suggestions.

[1294] Specifically, the server sends the prompt "User profile: Food preference: Vegan, Health goal: Weight loss" to the generative AI model, which then suggests healthy vegan recipes.

[1295] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1296] This invention provides a system that allows users to easily enjoy a nutritionally balanced and healthy meal. This system can further improve the user's dining experience by combining it with an emotion engine that recognizes the user's emotions. Detailed embodiments of the system are described below.

[1297] User Registration and Login

[1298] A user accesses the system and enters information to create an account (such as name, email address, password, dietary preferences, allergy information, and health goals). Once the information is complete, the user presses the "Register" button. The server receives the information sent by the user, generates a user profile based on that information, and stores it in a database. The user can then log in and use services based on their individual information.

[1299] Recipe Search and Filtering

[1300] The server searches a database for recipes based on the user's profile information, and the resulting recipes are filtered to match the user's dietary preferences and allergies. For example, if a user prefers a vegan and gluten-free diet, only recipes that match these preferences will be displayed.

[1301] Optimizing nutritional balance

[1302] The server calculates the nutritional information for the searched recipe. If the goal is weight loss, it will suggest recipe modifications that take into account calories and nutritional components. Specifically, it will suggest the optimal nutritional balance to match the user's health goals by adjusting the amounts of ingredients used, etc.

[1303] Cooking process support

[1304] The device uses an image generation means to display the cooking process and a picture of the finished product. The cooking process is divided into individual steps, and images and videos of each step are displayed. Audio advice is also provided to help users follow the steps accurately. This allows even beginners and busy users to complete the dish with confidence.

[1305] Multi-device compatible

[1306] The server references the device information used by the user and provides information optimized for each device, such as smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[1307] Individual proposals

[1308] The server uses generative AI to automatically generate new recipes and ingredient suggestions based on past user selection data and current user profiles. For example, it may suggest the most popular recipes over a certain period of time or new, untried recipes to users. This allows users to always enjoy new cooking experiences and nutritionally balanced meals.

[1309] Use of emotion engine

[1310] The app is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and vocal tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice based on that information.

[1311] Specifically, if the user is feeling stressed, the emotion engine will suggest recipes and steps that have a relaxing effect. For example, it will suggest recipes that are easy to make in a short time and contain ingredients that will improve the user's mood. In addition, the emotion engine will monitor the user's emotions while cooking and provide appropriate voice advice and encouraging messages.

[1312] Specific examples

[1313] For example, consider a user "User123" who prefers a "vegan" and "gluten-free" diet and is aiming to "lose weight." The user logs into the system, and the server searches for vegan and gluten-free recipes based on the user profile. The user then selects a recipe, and the server calculates the recipe's nutritional information and optimizes it for weight loss. The device displays each cooking step through images and audio guidance, allowing the user to follow the steps accurately. The server also provides an optimized display for the user's device (e.g., a smartwatch).

[1314] Furthermore, the emotion engine can recognize the user's emotions and, for example, if the user is feeling stressed, it can suggest dishes that have a relaxing effect. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages to help the user enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also gain emotional satisfaction in the process.

[1315] The processing flow will be explained below.

[1316] Step 1:

[1317] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals to create an account. The user presses the "Register" button.

[1318] Step 2:

[1319] The server receives the information sent by the user, generates a profile based on this information, stores it in a database, and sends the user a registration confirmation message.

[1320] Step 3:

[1321] A user enters their email address and password and presses the login button to access the system. The server verifies the login information and loads the profile information.

[1322] Step 4:

[1323] The server searches the database for suitable recipes based on the user profile, filters the recipes that match the user's criteria, and sends the results to the device.

[1324] Step 5:

[1325] The device displays a filtered list of recipes, and the user selects one from the displayed list.

[1326] Step 6:

[1327] After a user selects a recipe, the server calculates the recipe's details and nutritional balance, and then makes recommendations based on the results to help with health goals such as weight loss.

[1328] Step 7:

[1329] The server sends the optimized recipe information to the device, which then displays it to the user, showing each step of the recipe with images and videos, and providing audio advice.

[1330] Step 8:

[1331] The emotion engine analyzes the user's facial expressions and voice tone to recognize the user's emotional state. The server receives the data from the emotion engine and determines the user's current emotional state.

[1332] Step 9:

[1333] The server uses an emotion engine to suggest recipes and ingredients that suit the user's emotional state. For example, if the user is feeling stressed, it suggests dishes that have a relaxing effect.

[1334] Step 10:

[1335] The device starts cooking according to the recipe selected by the user, and displays each step in real time, while the emotion engine monitors the user's emotional state.

[1336] Step 11:

[1337] Based on the data from the emotion engine, the device provides appropriate voice advice and encouraging messages to the user while cooking. The content and tone of the advice are adjusted according to the user's emotional state.

[1338] Step 12:

[1339] After the user has finished cooking, the server will suggest new recipes and ingredients based on the user's past selection data, taking into account the user's preferences and emotional state.

[1340] Step 13:

[1341] If the user accepts the new suggestion, the process of recipe search, nutritional balance optimization, process display, and support from the emotion engine is repeated again. This cycle allows users to continuously enjoy new cooking experiences and nutritionally balanced meals.

[1342] Example 2

[1343] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1344] In today's busy lifestyles, it is difficult for users to maintain a healthy diet. It is also difficult to select individual meals and manage nutritional balance, and existing systems do not adequately support recipe selection and cooking according to emotional states. Therefore, there is a need for a system that can effectively improve users' eating habits and provide meal suggestions and cooking support according to their emotions.

[1345] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1346] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; and [means for recognizing the user's emotions using an emotion engine and adjusting the content of cooking advice.] This not only enables users to easily select and cook individual, nutritionally balanced meals, but also allows them to receive appropriate support according to their emotional state.

[1347] "User profile information" refers to personal information such as dietary preferences, allergy information, and health goals that a user registers in the system.

[1348] A "server" is a computer system that processes the entire system and manages data, and is a device that plays a role in executing various means.

[1349] A "cooking recipe" is information that describes the steps and ingredients for making a particular dish.

[1350] "Nutritional balance" refers to the proper distribution of nutrients such as calories, protein, fat, and carbohydrates.

[1351] "Image generation means" refers to a technology for generating images of cooking steps and finished recipes, and visually presenting them to the user.

[1352] "User Device" refers to the equipment used by a user to access the system, including smartphones, smartwatches, eyeglass displays, etc.

[1353] "Past user selections" refers to historical data of recipes and ingredients selected by the user in the past.

[1354] An "emotion engine" is a technology that analyzes a user's facial expressions and vocal tone to recognize their emotional state.

[1355] "Cooking advice" refers to information that guides or instructs the user when cooking.

[1356] MODE FOR CARRYING OUT THE INVENTION

[1357] This invention provides a system that allows users to easily enjoy nutritionally balanced and healthy meals. In particular, by combining it with an emotion engine that recognizes the user's emotions, the user's dining experience can be further improved. Detailed embodiments of the system are described below.

[1358] User Registration and Login

[1359] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals. When the user presses the "Register" button, the device sends the entered information to the server. The server generates a user profile based on the received information and stores it in a database. The user can then log in and use individual services.

[1360] Recipe Search and Filtering

[1361] The server searches the database for recipes based on the user's profile information. For example, if the user is vegan and prefers gluten-free meals, it will retrieve only recipes that match those criteria.

[1362] Optimizing nutritional balance

[1363] The server calculates the nutritional information for the searched recipe, examining each nutrient component, such as calories, protein, fat, and carbohydrates. Based on the user's health goals (e.g., weight loss), it adjusts the amounts of ingredients and suggests optimal nutritional balance. For example, it can adjust the amounts of certain ingredients to reduce calorie intake.

[1364] Cooking process support

[1365] The device uses an image generation means to display the cooking process and a finished product. Images and videos are displayed for each step, and audio advice is also provided. This allows even beginners and busy users to complete the dish with confidence.

[1366] Multi-device compatible

[1367] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc. This allows users to receive consistent support regardless of the device they use.

[1368] Individual proposals

[1369] The server uses a generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current user profile. For example, it can suggest recipes that the user has chosen most over a certain period of time, or new recipes that the user has not yet tried.

[1370] Use of emotion engine

[1371] The device is equipped with an emotion engine that recognizes the user's emotions. Emotion recognition is performed, for example, by analyzing the user's facial expressions and voice tone. The server collects the user's emotional state (happy, sad, stressed, etc.) in real time and adjusts the recipe and cooking advice content based on that information. Specifically, if the user is feeling stressed, it can suggest recipes and steps that have a relaxing effect. For example, it can suggest relaxing dishes that can be made easily and quickly, or recipes that include ingredients that will lift the user's mood. In addition, the emotion engine monitors the user's emotions while cooking and provides appropriate voice advice and encouraging messages.

[1372] Specific examples

[1373] For example, consider a user named "User123" who prefers vegan and gluten-free diets and is trying to lose weight. After logging in to the system, the server searches for vegan and gluten-free recipes based on the user profile. After the user selects a recipe, the server calculates the recipe's nutritional information and optimizes it for weight loss. The device supports the user's cooking process by displaying images and providing audio guidance. Furthermore, the emotion engine recognizes the user's emotions and, for example, can suggest dishes with a relaxing effect if the user is feeling stressed. During cooking, the emotion engine analyzes the user's emotions in real time and provides appropriate advice and encouraging messages, helping the user to enjoy cooking more. In this way, users can not only enjoy healthy, nutritionally balanced meals, but also feel emotionally fulfilled in the process.

[1374] Example prompts for generative AI models

[1375] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[1376] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1377] Step 1:

[1378] User Registration

[1379] A user accesses the system and enters their name, email address, password, dietary preferences, allergy information, and health goals.

[1380] The terminal transmits the input information to the server.

[1381] The server generates a user profile based on the received information and stores it in a database.

[1382] Input: User information (name, email address, password, food preferences, allergy information, health goals)

[1383] Output: User profile stored in the database

[1384] Step 2:

[1385] User Login

[1386] The user logs in using the email address and password they registered.

[1387] The server verifies the credentials and retrieves the user profile.

[1388] Input: Email address, Password

[1389] Output: Authentication result, user profile

[1390] Step 3:

[1391] Recipe Search and Filtering

[1392] The server searches the database for cooking recipes based on the user profile information.

[1393] Extract only recipes that match the user's dietary preferences and allergy information.

[1394] Input: User profile information

[1395] Output: Filtered recipe list

[1396] Step 4:

[1397] Optimizing nutritional balance

[1398] The server calculates the nutritional information for the retrieved recipe.

[1399] Based on the user's health goals (e.g., weight loss), it adjusts ingredient amounts and suggests optimal nutritional balance.

[1400] Input: filtered list of recipes, user's health goals

[1401] Output: Nutritionally balanced recipes

[1402] Step 5:

[1403] Cooking process support

[1404] The terminal uses an image generating means to display the cooking process and a finished image.

[1405] Each step is accompanied by images and videos of the process, and audio guidance is also provided.

[1406] Input: Nutritionally balanced recipe

[1407] Output: Recipe process images, videos, audio advice

[1408] Step 6:

[1409] Multi-device compatible

[1410] The server references information about the device the user is using and provides information optimized for smartphones, smartwatches, eyeglass displays, etc.

[1411] Input: User's device information

[1412] Output: Information display optimized for each device

[1413] Step 7:

[1414] Individual proposal generation

[1415] The server uses the generative AI model to automatically generate new recipe and ingredient suggestions based on past user selection data and current profile.

[1416] Input: Past user selection data, user profile

[1417] Output: New recipe and ingredient suggestions

[1418] Step 8:

[1419] Use of emotion engine

[1420] The emotion engine recognizes the user's emotions, for example by analyzing the user's facial expressions and tone of voice.

[1421] The server collects the user's emotional state in real time and adjusts recipes and cooking advice based on that information.

[1422] Input: User's facial expression data, voice tone data

[1423] Output: User's emotional state, recipes and advice based on the emotion

[1424] Prompt Sentence Examples

[1425] "Search for vegan and gluten-free recipes and suggest ways to optimize their nutritional balance for weight loss. Also, use an emotion engine to suggest relaxing dishes for when the user is feeling stressed."

[1426] (Application example 2)

[1427] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1428] In modern society, it is difficult for users to easily enjoy healthy meals. In particular, it is complex to consider the nutritional balance and food preferences of each individual user, and the cooking process is difficult, which places a heavy burden on beginners and busy users. Furthermore, food choices and motivation to cook are often influenced by stress and emotional fluctuations, so a system that can simultaneously solve these issues is needed.

[1429] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1430] In this invention, the server includes: [means for registering user profile information]; [means for searching for cooking recipes based on the user profile information]; [means for optimizing the nutritional balance of cooking recipes]; [means for displaying cooking steps using image generation means]; [means for providing optimized information to a user device]; [means for suggesting new recipes based on past user selections]; [means for providing cooking advice based on the user's emotional state using an emotion engine that recognizes the user's emotions]; and [means for providing audio advice and encouraging messages to the user in cooperation with a smartphone, smart glasses, or smart watch.] This allows users to receive personalized meal suggestions and support for cooking steps, and encouragement and advice according to their emotional state, making it easy to enjoy healthy, nutritionally balanced meals.

[1431] "User profile information" is data about an individual user, such as dietary preferences, allergy information, and health goals.

[1432] The "means for searching for recipes" is a function for extracting optimal recipes based on user profile information.

[1433] The "means for optimizing the nutritional balance of a cooking recipe" is a function for adjusting the nutritional components of a cooking recipe in accordance with the user's health goals, thereby providing an optimal nutritional balance.

[1434] The "means for displaying cooking steps using image generation means" is a function for visually displaying each cooking step and the finished product, using images and videos.

[1435] "Means for providing information optimized for user devices" refers to a function that provides information in a format that is optimal for various devices used by users (smartphones, smart glasses, smart watches, etc.).

[1436] The "means for proposing new recipes based on past user selections" is a function that analyzes the user's past selection history and appropriately suggests new cooking recipes.

[1437] An "emotion engine" is a technology for recognizing and analyzing a user's emotional state, primarily by analyzing facial expressions and vocal tone.

[1438] The "means for providing cooking advice" is a function for providing appropriate advice and guidance to the user while cooking.

[1439] "Smartphones, smart glasses, and smart watches" are devices that users use to receive information, and each can provide data in different formats.

[1440] "Voice advice" is a function that provides instructions and advice to the user through voice.

[1441] An "encouraging message" is a message intended to encourage and motivate the user.

[1442] This invention is a system that allows users to easily enjoy nutritionally balanced and healthy meals. This system registers user profile information, searches for cooking recipes, optimizes the nutritional balance of those recipes, and displays the cooking steps to support cooking. It also provides optimized information to the user's device and uses an emotion engine to provide advice based on the user's emotional state.

[1443] The server first registers user profile information. The user enters their dietary preferences, allergy information, health goals, etc., which are then saved in a database. The server then searches for the most suitable cooking recipe based on the user profile information and provides it to the user. After a recipe is selected, the server optimizes the nutritional balance of the recipe and makes suggestions that meet the user's health goals.

[1444] Furthermore, the device displays each cooking step and a picture of the finished product using images and audio. This information is optimized for the device used by the user, such as a smartphone, smart glasses, or smart watch. At this point, the cooking steps are displayed visually in an easy-to-understand manner using an image generation means, making cooking easy even for beginners or busy users.

[1445] The server also analyzes the user's past selection history and suggests new recipes, using a generative AI model to generate prompts and suggest optimal dishes based on the user's past selection data and current profile.

[1446] The emotion engine recognizes the user's emotional state and monitors it in real time while cooking. For example, if the user is feeling stressed, the emotion engine will provide relaxing dishes or encouraging messages. The smartphone, smart glasses, and smartwatch work together to provide voice advice, helping users to enjoy cooking more.

[1447] As a specific example, let's say a user is vegan, prefers gluten-free meals, and is trying to lose weight. In this case, the user's profile information is registered, and the server searches for and optimizes vegan and gluten-free recipes. If the user is feeling stressed, the server will suggest recipes with a relaxing effect, such as "banana smoothie" or "salad," and while cooking, it will provide encouraging voice messages such as "It's okay, it's hard, but you're almost there!" This allows users to easily enjoy healthy, nutritionally balanced meals.

[1448] An example prompt is:

[1449] "User profile: Vegan, gluten-free, goal: weight loss, current emotion: stressed. Generate a meal plan that considers these preferences and goals, and includes relaxation-inducing recipes."

[1450] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1451] Step 1: Register your user profile information

[1452] The server receives profile information from the user, such as dietary preferences, allergy information, health goals, etc. Based on this input information, the server registers and saves the user's profile in a database.

[1453] Input: User's dietary preferences, allergy information, health goals

[1454] Data processing and calculation: Formatting this information so that it can be stored in a database

[1455] Output: User profile saved message

[1456] Step 2: Find a recipe

[1457] The server references the user profile information, searches the database for relevant recipes, and filters out only those that match the user's preferences and allergy information.

[1458] Input: User profile information

[1459] Data processing and calculation: Execute a database query to retrieve recipes that match the criteria

[1460] Output: A list of matching recipes

[1461] Step 3: Optimize your nutritional balance

[1462] The server calculates the nutritional information of the searched recipes and optimizes them to meet the user's health goals, suggesting optimal nutritional balance by adjusting the amounts of ingredients used, etc.

[1463] Input: list of cooking recipes, user's health goals

[1464] Data processing and calculation: Analysis and adjustment of nutritional components

[1465] Output: Optimized nutritional balance recipe

[1466] Step 4: Show the cooking process

[1467] The device displays each step of the optimized recipe received from the server using images, videos, and audio, and is optimized to display appropriately regardless of the device the user is using.

[1468] Input: Optimized nutritional balance recipe

[1469] Data processing and calculation: Image generation and audio data distribution preparation

[1470] Output: Images, videos, and audio showing the cooking process

[1471] Step 5: Recognizing user emotions

[1472] The device analyzes the user's facial expressions and voice tone and uses an emotion engine to recognize the user's emotions in real time.

[1473] Input: User's face photo, voice data

[1474] Data processing and calculation: Extraction of emotional states using emotion analysis algorithms

[1475] Output: User's emotional state (e.g., stress, joy, sadness)

[1476] Step 6: Offering emotionally-based advice

[1477] The server provides the user with appropriate cooking recipes and cooking advice based on the user's emotional state obtained from the emotion engine.

[1478] Input: User emotional state, optimized recipe, user profile information

[1479] Data processing and calculation: Analysis of past selection data and generation of prompt sentences using a generative AI model

[1480] Output: Emotion-based cooking recipes and cooking advice

[1481] Step 7: Providing audio advice and encouraging messages

[1482] The device receives voice advice and encouraging messages from the server and provides them to the user. This can be a smartphone, smart glasses, or smart watch.

[1483] Input: Audio advice and encouraging messages sent from the server

[1484] Data processing and calculation: Preparing audio data for device distribution

[1485] Output: Playback of voice advice and encouraging messages

[1486] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1487] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1488] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1489] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1490] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1491] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1492] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1493] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1494] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1495] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1496] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1497] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1498] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1499] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1500] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1501] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1502] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1503] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1504] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1505] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1506] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1507] The following is further disclosed regarding the above embodiment.

[1508] (Claim 1)

[1509] [means for registering user profile information;

[1510] [means for searching for cooking recipes based on user profile information;

[1511] [Methods for optimizing the nutritional balance of cooking recipes,

[1512] [a means for displaying cooking steps using an image generating means;

[1513] [Means for providing information optimized for a user device;

[1514] [Means for suggesting new recipes based on past user selections; and

[1515] A system including:

[1516] (Claim 2)

[1517] The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

[1518] (Claim 3)

[1519] [The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product in images and audio.

[1520] "Example 1"

[1521] (Claim 1)

[1522] [means for registering user profile information;

[1523] [means for searching for cooking recipes based on user profile information;

[1524] [Methods for optimizing the nutritional balance of cooking recipes,

[1525] [a means for displaying cooking steps using an image generating means;

[1526] [Means for providing information optimized for a user device;

[1527] [Means for suggesting new recipes based on past user selections; and

[1528] [Means for generating a new recipe based on a prompt sentence using a generative AI model;

[1529] [means for providing audio advice to a user;

[1530] A system including:

[1531] (Claim 2)

[1532] The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

[1533] (Claim 3)

[1534] [The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product in images and audio.

[1535] "Application Example 1"

[1536] (Claim 1)

[1537] [means for registering user profile information;

[1538] [means for searching for cooking recipes based on user profile information;

[1539] [Methods for optimizing the nutritional balance of cooking recipes,

[1540] [a means for displaying cooking steps using an image generating means;

[1541] [Means for providing information optimized for a user device;

[1542] [Means for suggesting new recipes based on past user selections; and

[1543] [Means for making the suggested dishes instantly available for ordering via a delivery service;

[1544] A system including:

[1545] (Claim 2)

[1546] The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

[1547] (Claim 3)

[1548] [The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product in images and audio.

[1549] "Example 2: Combining Emotion Engines"

[1550] (Claim 1)

[1551] [means for registering user profile information;

[1552] [means for searching for cooking recipes based on user profile information;

[1553] [Methods for optimizing the nutritional balance of cooking recipes,

[1554] [a means for displaying cooking steps using an image generating means;

[1555] [Means for providing information optimized for a user device;

[1556] [Means for suggesting new recipes based on past user selections; and

[1557] [Means for recognizing the user's emotions using an emotion engine and adjusting the cooking advice content;

[1558] A system including:

[1559] (Claim 2)

[1560] The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

[1561] (Claim 3)

[1562] [The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product in images and audio.

[1563] "Application example 2 when combining emotion engines"

[1564] (Claim 1)

[1565] [means for registering user profile information;

[1566] [means for searching for cooking recipes based on user profile information;

[1567] [Methods for optimizing the nutritional balance of cooking recipes,

[1568] [a means for displaying cooking steps using an image generating means;

[1569] [Means for providing information optimized for a user device;

[1570] [Means for suggesting new recipes based on past user selections; and

[1571] [means for providing cooking advice based on the user's emotional state using an emotion engine that recognizes the user's emotions;

[1572] [Means for providing voice advice and encouraging messages to users in conjunction with a smartphone, smart glasses, or smart watch;

[1573] A system including:

[1574] (Claim 2)

[1575] The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

[1576] (Claim 3)

[1577] [The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product in images and audio. [Explanation of symbols]

[1578] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for registering user profile information; means for retrieving cooking recipes based on user profile information; A way to optimize the nutritional balance of cooking recipes, means for displaying cooking steps using image generation means; means for providing information optimized for a user device; a means for suggesting new recipes based on past user selections; A system including:

2. 10. The system of claim 1, wherein the means for registering user profile information comprises means for receiving information including the user's dietary preferences, allergy information, and health goals.

3. 2. The system according to claim 1, wherein the image generating means includes means for displaying the cooking process and the finished product with images and audio.

Images